A method for quantitatively detecting mica content in sand based on spectral analysis
Patent Information
- Application Number
- CN202610844390.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-11
- Publication Date
- 2026-10-27
AI Technical Summary
[0002]砂作为混凝土的重要细骨料，其云母含量直接影响混凝土的强度、耐久性和工作性能；现行国家标准《建设用砂》GB/T 14684规定了砂中云母含量的检测方法，即人工挑选法：在放大镜下用钢针将砂样中的云母颗粒逐一挑出并称重计算；该方法虽然直观，但存在效率极低、人为误差大和对操作者视力伤害严重等问题，且无法有效检测机制砂石粉中的游离云母；近年来，红外光谱、近红外光谱和拉曼光谱等光谱分析技术被尝试应用于砂中云母的快速检测；然而，现有光谱检测方法普遍面临两大技术瓶颈：一是云母和伊利石等黏土矿物在光谱特征上高度相似，即存在“同谱异物”现象，导致定性定量结果易混淆；二是实验室高精度光谱仪建立的检测模型无法直接迁移到现场便携式光谱仪上使用，一旦光谱仪型号、环境温度湿度或砂样来源发生变化，模型预测精度急剧下降，需要重新采集大量现场有标注样本进行建模，而现场样本的云母含量需通过人工挑选法标注，成本极高且难以大规模获取
[0022]本发明的有益效果：1）跨场景泛化能力提升：针对现有机器学习方法无法解决跨仪器、跨场景模型泛化难题和且严重依赖大规模标注训练数据的问题，本发明通过源域与目标域特征提取器共同对抗域判别器的对抗训练，迫使模型提取域不变特征，并采用固定源域大样本（完全标注）与目标域小样本（仅需源域标注样本数量的1%~10%）的迁移学习架构，实现了实验室高精度光谱仪模型向现场便携式光谱仪的低成本迁移，大幅降低了对目标域标注数据的依赖；2）在线自适应进化能力：针对现有域自适应方案为静态迁移和模型固化后无法随新数据进化而导致长期部署性能衰减的不足，本发明采用固定的源域训练集与动态累积的目标域训练集周期性重新执行对抗训练和微调，使模型能够持续适应现场数据分布的变化，在长期部署中始终保持最优检测性能；3）特征分布全局对齐：引入最大均值差异损失作为额外分布对齐约束，与对抗损失协同优化，进一步消除了源域与目标域之间的特征分布差异；4）预测结果可信评估：针对现有方法缺乏预测结果可靠性量化评估机制和异常样本仍输出不可信预测值的不足，本发明采用蒙特卡洛dropout输出云母含量预测值及其95%置信区间，当置信区间过宽时自动标记低置信度结果并建议复测，为工程质量控制提供了可量化的决策依据，避免了不可信检测结果带来的工程风险。
Smart Images
Figure CN122991339A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of building material testing technology, specifically to a method for quantitatively detecting mica content in sand based on spectral analysis, and particularly to an intelligent detection method for mica content in sand that combines adversarial domain adaptive evolutionary transfer learning with spectral analysis. Background Technology
[0002] As an important fine aggregate in concrete, sand's mica content directly affects the strength, durability, and workability of concrete; the current national standard "Construction Sand" GB / T 14684 specifies a method for detecting mica content in sand, namely the manual selection method: under a magnifying glass, mica particles in the sand sample are picked out one by one with a steel needle and weighed. Although this method is intuitive, it suffers from extremely low efficiency, large human error, and serious damage to the operator's eyesight. Moreover, it cannot effectively detect free mica in machine-made sand and gravel powder. In recent years, spectral analysis techniques such as infrared spectroscopy, near-infrared spectroscopy, and Raman spectroscopy have been attempted to be applied to the rapid detection of mica in sand. However, existing spectral detection methods generally face two major technical bottlenecks: first, mica and clay minerals such as illite have highly similar spectral characteristics, i.e., there is a "same spectrum, different substance" phenomenon, which makes it easy to confuse qualitative and quantitative results; second, the detection model established by a high-precision laboratory spectrometer cannot be directly transferred to a portable spectrometer in the field. Once the spectrometer model, ambient temperature and humidity, or sand sample source changes, the model prediction accuracy drops sharply, requiring the collection of a large number of labeled samples in the field for modeling. However, the mica content of the field samples needs to be labeled by manual selection, which is extremely costly and difficult to obtain on a large scale.
[0003] To address the aforementioned issues, existing technologies have attempted to combine machine learning and spectral analysis, such as using partial least squares regression, support vector machines, and simple neural networks for modeling. However, these methods still cannot solve the problem of model generalization across instruments and scenarios, and heavily rely on large-scale labeled training data. Some studies have attempted to introduce generative adversarial networks for domain adaptive transfer, but existing solutions are mostly static transfers, meaning that once the source domain is aligned to the target domain, the model becomes fixed and cannot evolve and update online with the continuous collection of new field data, leading to a gradual decline in model performance during long-term deployment. In addition, existing methods lack a quantitative evaluation mechanism for the reliability of prediction results. When encountering anomalous samples that differ significantly from the training distribution, the model may still output unreliable predictions, posing a potential risk to engineering quality. Summary of the Invention
[0004] The purpose of this invention is to overcome the above-mentioned problems existing in the prior art; to this end, this invention provides a method for quantitatively detecting the mica content in sand based on spectral analysis, the method comprising: Step S1: Obtain the spectral data of sand samples from the source domain and the corresponding mica content labeling data to construct the source domain training set; obtain the spectral data of sand samples from the target domain and some labeled sand sample data to construct the target domain training set; the source domain training set is a pre-constructed fixed data set whose content remains unchanged throughout the entire method lifecycle; the target domain training set consists of spectral data of sand samples collected from the field, including a small number of labeled samples and a large number of unlabeled samples, used to train the model to adapt to the field detection environment; Step S2: Construct a domain adaptive evolutionary transfer network, the network including: a source domain feature extractor F s The first spectral feature vector is used to extract the source domain sand sample spectral data; the target domain feature extractor F t The first part of the code is used to extract the second spectral feature vector of the sand sample spectral data in the target domain; the second part is used to extract the second spectral feature vector; the third part is used to extract the second spectral feature vector; the fourth part is used to extract the second spectral feature vector; the fifth part is used to extract the second spectral feature vector; the sixth part is used to extract the second spectral feature vector; the seventh part is used to extract the second spectral feature vector; the eighth part is used to extract the second spectral feature vector; the ninth part is used to extract the second spectral feature vector; the eleventh ... S The target domain feature extractor F t The source domain discriminator F is subjected to adversarial training with the domain discriminator D; wherein the source domain feature extractor F s and the target feature extractor F t The source and target domains act as game partners with the domain discriminator D, making the feature vector distributions of the source and target domains tend to be consistent; during training, the source domain training set and the current target domain training set are used; in step S4, after the adversarial training reaches equilibrium, the source domain feature extractor F is fixed. s and the target domain feature extractor F t The parameters of the mica content regression predictor R are fine-tuned on labeled data in the target domain; S5, during the collection of new target domain samples, newly collected spectral data are continuously added to the target domain training set; an evolutionary update strategy is adopted, using a fixed source domain training set and an updated target domain training set to periodically re-execute steps S3 to S4 to achieve online adaptive evolution of the model; wherein, no new data is added to the source domain training set during the evolutionary update process, and the initial state is always maintained; S6, the spectral data of the sand sample to be tested is input into the trained mica content regression predictor R, and the quantitative result of mica content is output; the sand sample to be tested and the sand samples in the target domain training set belong to the same field distribution, but the spectral data of the sand sample to be tested is not included in the target domain training set.
[0005] Specifically, the source domain feature extractor F s and target domain feature extractor F tSharing the same backbone network structure, the backbone network is a one-dimensional convolutional neural network; the domain discriminator D uses a gradient inversion layer to implement adversarial training. Let the input spectral sample x, after passing through the feature extractor F, yield a feature vector z = F(x). The gradient inversion layer maintains identity transformation during forward propagation and automatically multiplies the gradient by -λ during backward propagation, where λ is the gradient inversion coefficient; the loss function for adversarial training is:
[0006] Where, x s For the source domain spectral sample, x t For the target domain spectral sample, D s For the source domain data distribution, D t For the target domain data distribution, F s (x) s F is the source domain feature vector. t (x) t ) represents the feature vector of the target domain, and D(·) represents the output of the domain discriminator.
[0007] Specifically, the gradient reversal coefficient λ adopts an adaptive dynamic adjustment strategy, and the expression for λ is:
[0008] Where p is the proportion of the current training round to the total training rounds, p∈[0,1]; α is a hyperparameter controlling the steepness of the curve, α∈[5,10]; λ is close to 0 in the early stage of training and close to 1 in the later stage of training.
[0009] Specifically, the source domain feature extractor F s and the target domain feature extractor F t During the adversarial training phase, Maximum Mean Difference (MMD) is introduced as an additional distribution alignment constraint, and the total loss function is:
[0010] Where, λ adv The larger the model, the more emphasis is placed on domain alignment; To combat losses, λ MMD MMD loss weighting coefficients 2 For Z s and Z t The square of the maximum mean difference between them, Z s Let F be the set of features of the source domain. s (x) s )}; Z t Let F be the set of features in the target domain. t (x) t )}.
[0011] Specifically, the mica content regression predictor R is a multilayer perceptron network, whose output layer is a single neuron that uses a linear activation function to output continuous predicted values of mica content. Where z is the feature vector output by the feature extractor F; the training loss function of the mica content regression predictor R is:
[0012] Where, N t，labeled The number of spectral samples with mica content labels in the target domain. For the i-th target domain spectral sample, Let θ represent the true mica content of the i-th target, β be the regularization coefficient, and take a value between 0.001 and 0.01; R This represents a one-dimensional vector formed by concatenating all trainable parameters of the regression predictor R. Let be the squared L2 norm of the regression predictor parameters, and ∑ be the summation formula.
[0013] Specifically, the number of spectral samples with mica content labels in the source domain training set is N. s，labeled The number N of spectral samples with mica content labels in the target domain training set. t，labeled The two satisfy the relationship: 0.01·N s，labeled ≤N t，labeled ≤0.1·N s，labeled .
[0014] Specifically, the triggering conditions for the evolution update strategy are at least one of the following: (1) the cumulative number of newly collected unlabeled samples in the target domain reaches 20 to 100; (2) the average relative deviation between the predicted and verified values of mica content in the target domain exceeds 5% to 15%; (3) the model deployment time exceeds 7 to 30 days; after triggering, all the labeled and unlabeled data accumulated in the current target domain are merged and steps S3 to S4 are re-executed together with the fixed source domain training set.
[0015] Specifically, the present invention also includes manually labeling samples with the highest information content from newly acquired unlabeled spectral data in the target domain using an active learning strategy, thereby expanding the labeled dataset in the target domain; the active learning adopts a sampling strategy based on prediction entropy, and the uncertainty score of the sample is defined as:
[0016] Where p(c|x) t The model predicts the probability that the spectral sample belongs to mica category c, where C is the number of mica subspecies, including at least muscovite, illite, and biotite. The top 1 to 10 samples with the highest scores in each round are labeled, and the labeled samples are added to the target domain labeled dataset.
[0017] Specifically, the spectral data of both the source and target domains undergo a unified preprocessing procedure, including: denoising, baseline correction, normalization, and wavelength alignment; wherein, normalization employs a standard normal variable transformation.
[0018] Where, x （i） Let i be the original spectral vector of the i-th sample. x is the mean of the corresponding sample spectral vectors. （i） - This represents the original spectrum minus its mean, resulting in a vector; m is the total number of wavelength points, and j is the index of the wavelength point, from 1 to m. Let be the original intensity value of the spectrum of the i-th sand sample at the j-th wavelength point. The result of the standard normal variable SNV transformation of the i-th sand sample spectrum is a vector containing the values of all wavelength points.
[0019] Specifically, the output of the mica content regression predictor R also includes the prediction confidence interval, and the prediction uncertainty is estimated using Monte Carlo dropout: repeated T mc The next forward propagation yields a set of predicted values { , , ..., The final predicted value is:
[0020] The prediction standard deviation is:
[0021] in, This is the k-th predicted value; the 95% confidence interval is [ -1.96σ pred , +1.96σ pred When the confidence interval width exceeds the preset threshold, the system automatically marks the detection result as low confidence.
[0022] The beneficial effects of this invention are as follows: 1) Improved cross-scene generalization ability: Addressing the problem that existing machine learning methods cannot solve the generalization difficulties of models across instruments and scenes and heavily rely on large-scale labeled training data, this invention uses adversarial training between source domain and target domain feature extractors against a domain discriminator, forcing the model to extract domain-invariant features. It also employs a transfer learning architecture with a fixed large sample size (fully labeled) in the source domain and a small sample size in the target domain (requiring only 1%~10% of the source domain labeled samples), achieving low-cost transfer of high-precision laboratory spectrometer models to portable field spectrometers, significantly reducing dependence on labeled target domain data; 2) Online adaptive evolution capability: Addressing the shortcomings of existing domain adaptive solutions, which involve static transfer and the inability of models to evolve with new data after model solidification, leading to long-term performance degradation, this invention uses fixed source domain training. 1) Periodically re-perform adversarial training and fine-tuning on the target domain training set and dynamically accumulated set, enabling the model to continuously adapt to changes in the distribution of field data and maintain optimal detection performance in long-term deployment; 2) Global alignment of feature distribution: The maximum mean difference loss is introduced as an additional distribution alignment constraint, which is optimized in conjunction with the adversarial loss to further eliminate the feature distribution difference between the source domain and the target domain; 3) Credibility assessment of prediction results: In view of the shortcomings of existing methods in lacking a quantitative evaluation mechanism for the reliability of prediction results and outputting unreliable prediction values for abnormal samples, this invention uses Monte Carlo dropout to output the predicted value of mica content and its 95% confidence interval. When the confidence interval is too wide, low confidence results are automatically marked and retesting is recommended, providing a quantifiable decision basis for engineering quality control and avoiding engineering risks caused by unreliable detection results. Attached Figure Description
[0023] Figure 1 This is a flowchart of a method for quantitatively detecting mica content in sand based on spectral analysis according to the present invention. Detailed Implementation
[0024] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the embodiments of the present invention are only one specific embodiment, and any modifications and simple substitutions made by those skilled in the art based on the embodiments of the present invention are within the protection scope of the present invention.
[0025] like Figure 1The diagram shows a flowchart of a method for quantitatively detecting mica content in sand based on spectral analysis according to the present invention. The flowchart includes: Step S1, acquiring spectral data of sand samples from the source domain and corresponding mica content labeling data to construct a source domain training set; acquiring spectral data of sand samples from the target domain and some labeled sand sample data to construct a target domain training set; the source domain training set is a pre-constructed fixed data set whose content remains unchanged throughout the entire method lifecycle; the target domain training set consists of spectral data of sand samples collected from the field, including a small number of labeled samples and a large number of unlabeled samples, used to train the model to adapt to the field detection environment; Step S2, constructing a domain adaptive evolutionary transfer network, the network including: a source domain feature extractor F... s The first spectral feature vector is used to extract the source domain sand sample spectral data; the target domain feature extractor F t The first part of the code is used to extract the second spectral feature vector of the sand sample spectral data in the target domain; the second part is used to extract the second spectral feature vector; the third part is used to extract the second spectral feature vector; the fourth part is used to extract the second spectral feature vector; the fifth part is used to extract the second spectral feature vector; the sixth part is used to extract the second spectral feature vector; the seventh part is used to extract the second spectral feature vector; the eighth part is used to extract the second spectral feature vector; the ninth part is used to extract the second spectral feature vector; the eleventh ... S The target domain feature extractor F t The source domain discriminator F is subjected to adversarial training with the domain discriminator D; wherein the source domain feature extractor F s and the target feature extractor F t The source and target domains act as game partners with the domain discriminator D, making the feature vector distributions of the source and target domains tend to be consistent; during training, the source domain training set and the current target domain training set are used; in step S4, after the adversarial training reaches equilibrium, the source domain feature extractor F is fixed. s and the target domain feature extractor F t The parameters of the mica content regression predictor R are fine-tuned on labeled data in the target domain; S5, during the collection of new target domain samples, newly collected spectral data are continuously added to the target domain training set; an evolutionary update strategy is adopted, using a fixed source domain training set and an updated target domain training set to periodically re-execute steps S3 to S4 to achieve online adaptive evolution of the model; wherein, no new data is added to the source domain training set during the evolutionary update process, and the initial state is always maintained; S6, the spectral data of the sand sample to be tested is input into the trained mica content regression predictor R, and the quantitative result of mica content is output; the sand sample to be tested and the sand samples in the target domain training set belong to the same field distribution, but the spectral data of the sand sample to be tested is not included in the target domain training set.
[0026] This invention first constructs a source domain training set as a high-precision fully labeled sample for laboratory use and a target domain training set as a portable, field-portable set of a small number of labeled samples and a large number of unlabeled samples. Then, it constructs a domain adaptive evolutionary transfer network consisting of a source domain feature extractor, a target domain feature extractor, a domain discriminator, and a regression predictor. Through adversarial training, the feature distributions of the source and target domains are made to converge. After fixing the feature extractor, the regression predictor is fine-tuned using only labeled data from the target domain. The network is then periodically retrained using the fixed source domain and dynamic target domain training sets to achieve online adaptive evolution. The purpose of this invention is to solve the problems of poor cross-instrument generalization ability, heavy reliance on large-scale labeled data, inability to evolve online through static transfer, and lack of reliability assessment of prediction results in existing spectral detection methods.
[0027] The technical effects of the present invention will be explained in detail below through a specific embodiment.
[0028] The embodiments of the present invention include: 1. training set construction; 2. network construction; 3. spectral data preprocessing; 4. adversarial training; 5. regression predictor fine-tuning; 6. online adaptive evolution; 7. prediction output.
[0029] Step 1: First, the first part of the embodiment, namely the construction of the training set, is explained; the spectral data of sand samples in the source domain and the corresponding mica content annotation data are obtained to construct the source domain training set; the source domain training set is a pre-constructed fixed dataset whose content remains unchanged throughout the entire method lifecycle; let N be the number of spectral samples with mica content annotation values in the source domain training set. s，labeled .
[0030] Subsequently, spectral data of sand samples in the target domain and some labeled sand sample data were acquired to construct a target domain training set. The target domain training set consists of spectral data of sand samples collected from the field. It includes a small number of labeled samples and a large number of unlabeled samples to train the model to adapt to the field detection environment. Let N be the number of spectral samples with mica content labels in the target domain training set. t，labeled .
[0031] The relationship between the number of labeled samples in the source domain training set and the target domain training set is as follows: 0.01·N s，labeled ≤N t，labeled ≤0.1·N s，labeled .
[0032] Step 2: Network construction; construct a domain adaptive evolutionary transfer network, the network including: a source domain feature extractor F. s The first spectral feature vector is used to extract the source domain sand sample spectral data; the target domain feature extractor F tThe first part is used to extract the second spectral feature vector of the sand sample spectral data in the target domain; the second part is used to distinguish whether the source of the input feature vector is the source domain or the target domain; and the third part is used to output the mica content regression predictor R based on the spectral feature vector.
[0033] Source Domain Feature Extractor F s and target domain feature extractor F t They share the same backbone network structure, which is a one-dimensional convolutional neural network.
[0034] Step 3: Spectral data preprocessing; both source and target domain spectral data undergo a unified preprocessing workflow, including: denoising, baseline correction, normalization, and wavelength alignment; among which, normalization uses standard normal variable transformation:
[0035] Where, x （i） Let i be the original spectral vector of the i-th sample. x is the mean of the corresponding sample spectral vectors. （i） - represents the centered vector obtained by subtracting the mean from the original spectrum; m is the total number of wavelength points, and j is the index of the wavelength point, from 1 to m; Let be the original intensity value of the spectrum of the i-th sand sample at the j-th wavelength point. The result of the standard normal variable SNV transformation of the i-th sand sample spectrum is a vector containing all wavelength point values.
[0036] Step 4: Adversarial Training; Employing a generative adversarial training strategy, the source domain feature extractor F... s Target Domain Feature Extractor F t The source domain discriminator D is subjected to adversarial training; wherein, the source domain feature extractor F s and the target domain feature extractor F t They work together with the domain discriminator D to play a game with each other, making the feature vector distributions of the source domain and the target domain tend to be consistent; during training, the source domain training set and the current target domain training set are used.
[0037] The domain discriminator D employs a gradient inversion layer to implement adversarial training. Let the input spectral sample x be processed by the feature extractor F to obtain the feature vector z = F(x). The gradient inversion layer maintains the identity transformation during forward propagation and automatically multiplies the gradient by -λ during backward propagation, where λ is the gradient inversion coefficient. The loss function for adversarial training is:
[0038] Where, x s For the source domain spectral sample, x t For the target domain spectral sample, Ds For the source domain data distribution, D t For the target domain data distribution, F s (x) s F is the source domain feature vector. t (x) t ) represents the feature vector of the target domain, and D(·) represents the output of the domain discriminator.
[0039] The gradient reversal coefficient λ adopts an adaptive dynamic adjustment strategy, and the expression for λ is:
[0040] Where p is the proportion of the current training round to the total training rounds, p∈[0,1]; α is a hyperparameter controlling the steepness of the curve, α∈[5,10]; λ is close to 0 in the early stage of training and close to 1 in the later stage of training.
[0041] The source domain feature extractor F of the present invention s and target domain feature extractor F t Introducing the maximum mean difference as an additional distribution alignment constraint during the adversarial training phase, the resulting total loss function is:
[0042] Where, λ adv To counteract the loss weighting coefficient, a value between 0.1 and 1 is taken, λ adv The larger the model, the more emphasis is placed on domain alignment; To combat losses, λ MMD MMD loss weighting coefficients 2 Z is the square of the maximum mean difference. s Let F be the set of features of the source domain. s (x) s )}; Z t Let F be the set of features in the target domain. t (x) t )}.
[0043] By minimizing the total loss function Simultaneously, the feature extractor and domain discriminator are optimized, enabling the feature extractor to extract domain-invariant features to deceive the domain discriminator while retaining information useful for mica content prediction.
[0044] Step 5: After the adversarial training reaches equilibrium, fix the source domain feature extractor F. s and target domain feature extractor F t The parameters of the mica content regression predictor R are fine-tuned on labeled data in the target domain.
[0045] The mica content regression predictor R is a multilayer perceptron network with a single neuron in its output layer. It uses a linear activation function to output continuous predicted values of mica content. Where z is the feature vector output by the feature extractor F; the training loss function of the regression predictor R is:
[0046] Where, N t，labeled The number of spectral samples with mica content labels in the target domain. For the i-th target domain spectral sample, Let θ represent the true mica content of the i-th target, β be the regularization coefficient, and take a value between 0.001 and 0.01; R This represents a one-dimensional vector formed by concatenating all trainable parameters of the regression predictor R. Let be the squared L2 norm of the regression predictor parameters, and ∑ be the summation formula.
[0047] Step 6: Online adaptive evolution; During the acquisition of new target domain samples, newly acquired spectral data are continuously added to the target domain training set; An evolutionary update strategy is adopted, using a fixed source domain training set and an updated target domain training set, and steps 4 and 5 are periodically re-executed to achieve online adaptive evolution of the model; wherein, no new data is added to the source domain training set during the evolutionary update process, and it always maintains the state at the time of initial construction.
[0048] The triggering conditions for the evolution update strategy are at least one of the following: (1) the cumulative number of newly collected unlabeled samples in the target domain reaches 20 to 100; (2) the average relative deviation between the predicted value and the validation value of mica content in the target domain exceeds 5% to 15%; (3) the model deployment time exceeds 7 to 30 days; after triggering, all the labeled and unlabeled data accumulated in the current target domain are merged and the third to fourth steps are re-executed together with the fixed source domain training set.
[0049] Furthermore, this method also employs an active learning strategy to select the most informative samples from newly acquired unlabeled spectral data in the target domain for manual annotation, thereby expanding the labeled dataset in the target domain. The active learning utilizes a sampling strategy based on prediction entropy, and the uncertainty score of a sample is defined as:
[0050] Where p(c|x) t The model predicts the probability that the spectral sample belongs to mica category c, where C is the number of mica subspecies, including at least muscovite, illite, and biotite. In each round, the top 1 to 10 samples with the highest scores are selected for labeling, and the labeled samples are added to the target domain labeled dataset.
[0051] Step 7: Prediction Output: Input the spectral data of the sand sample to be tested into the trained mica content regression predictor R, and output the quantitative result of mica content; the sand sample to be tested and the sand samples in the target domain training set belong to the same field distribution, but the spectral data of the sand sample to be tested is not included in the target domain training set.
[0052] The output of the mica content regression predictor R also includes the prediction confidence interval, and the prediction uncertainty is estimated using Monte Carlo dropout: repeated T mc The next forward propagation yields a set of predicted values { , , ..., The final predicted value is:
[0053] The prediction standard deviation is:
[0054] in, This is the k-th predicted value; the 95% confidence interval is [ -1.96σ pred , +1.96σ pred When the confidence interval width exceeds the preset threshold, the system automatically marks the detection result as low confidence; it automatically marks low confidence results and suggests retesting.
[0055] In summary, this invention constructs a fixed source domain training set and a dynamically accumulated target domain training set. It employs an adversarial training strategy where the source domain feature extractor and the target domain feature extractor jointly challenge the domain discriminator, forcing the model to extract domain-invariant features. Combined with the fact that only 1% to 10% of the source domain's labeled samples are needed for a small number of labeled target domain samples, effective fine-tuning of the regression predictor is achieved. Furthermore, by periodically re-executing adversarial training and fine-tuning using the fixed source domain and dynamic target domain training sets, online adaptive evolution of the model is realized. Simultaneously, Monte Carlo dropout is used to output predicted values and 95% confidence intervals. When the confidence interval is too wide, low-confidence results are automatically marked and retesting is recommended. This invention effectively solves the technical problems of existing sand mica spectral detection methods, such as poor cross-instrument and cross-scenario generalization ability, heavy reliance on large-scale labeled data, inability to evolve online through static migration, and lack of quantitative assessment of the reliability of prediction results. It significantly improves the practicality, stability, and reliability of the detection method and can be widely applied in the field of quality testing of construction sand.
Claims
1. A method for quantitatively detecting mica content in sand based on spectral analysis, characterized in that, The method includes the following steps: Step S1: Obtain the spectral data of sand samples from the source domain and the corresponding mica content labeling data to construct the source domain training set; obtain the spectral data of sand samples from the target domain and some labeled sand sample data to construct the target domain training set; the source domain training set is a pre-constructed fixed data set whose content remains unchanged throughout the entire method lifecycle to construct the target domain training set; the target domain training set consists of spectral data of sand samples collected from the field, which includes a small number of labeled samples and a large number of unlabeled samples, used to train the model to adapt to the field detection environment; Step S2, construct a domain adaptive evolutionary transfer network, the network including: a source domain feature extractor F s The first spectral feature vector is used to extract the source domain sand sample spectral data; the target domain feature extractor F t The first part is used to extract the second spectral feature vector of the sand sample spectral data in the target domain; the second part is used to distinguish whether the source feature vector comes from the source domain or the target domain; and the third part is used to output the mica content regression predictor R based on the spectral feature vector. Step S3: Employ a generative adversarial training strategy to train the source domain feature extractor F. S The target domain feature extractor F t The source domain discriminator F is subjected to adversarial training with the domain discriminator D; wherein the source domain feature extractor F s and the target feature extractor F t Together with the domain discriminator D, they act as game partners, making the feature vector distributions of the source domain and the target domain tend to be consistent; during training, the source domain training set and the current target domain training set are used; Step S4: After the adversarial training reaches equilibrium, fix the source domain feature extractor F. s and the target domain feature extractor F t The parameters of the mica content regression predictor R are fine-tuned on labeled data in the target domain. S5, during the acquisition of new target domain samples, the newly acquired spectral data is continuously added to the target domain training set; an evolutionary update strategy is adopted, using a fixed source domain training set and the updated target domain training set to periodically re-execute steps S3 to S4 to achieve online adaptive evolution of the model; wherein, the source domain training set does not add new data during the evolutionary update process and always maintains the state at the initial construction. S6, input the spectral data of the sand sample to be tested into the trained mica content regression predictor R, and output the quantitative result of mica content; the sand sample to be tested and the sand samples in the target domain training set belong to the same field distribution, but the spectral data of the sand sample to be tested is not included in the target domain training set.
2. The method according to claim 1, characterized in that, The source domain feature extractor F s and the target domain feature extractor F t Sharing the same backbone network structure, the backbone network is a one-dimensional convolutional neural network; the domain discriminator D uses a gradient inversion layer to implement adversarial training. Let the input spectral sample x, after passing through the feature extractor F, yield a feature vector z = F(x). The gradient inversion layer maintains an identity transformation during forward propagation and automatically multiplies the gradient by -λ during backward propagation, where λ is the gradient inversion coefficient; the loss function for adversarial training is: Where, x s For the source domain spectral sample, x t For the target domain spectral sample, D s For the source domain data distribution, D t For the target domain data distribution, F s (x) s F is the source domain feature vector. t (x) t ) represents the feature vector of the target domain, and D(·) represents the output of the domain discriminator.
3. The method according to claim 2, characterized in that, The gradient reversal coefficient λ adopts an adaptive dynamic adjustment strategy, and the expression for λ is: Where p is the proportion of the current training round to the total training rounds, p∈[0,1]; α is a hyperparameter controlling the steepness of the curve, α∈[5,10]; λ is close to 0 in the early stage of training and close to 1 in the later stage of training.
4. The method according to claim 2, characterized in that, The source domain feature extractor F s and the target domain feature extractor F t During the adversarial training phase, Maximum Mean Difference (MMD) is introduced as an additional distribution alignment constraint, and the total loss function is: Where, λ adv The larger the model, the more emphasis is placed on domain alignment; To combat losses, λ MMD MMD loss weighting coefficients 2 For Z s and Z t The square of the maximum mean difference between them, Z s Let F be the set of features of the source domain. s (x) s )}; Z t Let F be the set of features in the target domain. t (x) t )}.
5. The method according to claim 1, characterized in that, The mica content regression predictor R is a multilayer perceptron network, with its output layer consisting of a single neuron using a linear activation function, outputting continuous predicted values of mica content. Where z is the feature vector output by the feature extractor F; the training loss function of the mica content regression predictor R is: Where, N t，labeled The number of spectral samples with mica content labels in the target domain. For the i-th target domain spectral sample, Let θ represent the true mica content of the i-th target, β be the regularization coefficient, and take a value between 0.001 and 0.01; R This represents a one-dimensional vector formed by concatenating all trainable parameters of the regression predictor R. Let be the squared L2 norm of the regression predictor parameters, and ∑ be the summation formula.
6. The method according to claim 1, characterized in that, The number of spectral samples with mica content labels in the source domain training set is N. s，labeled The number N of spectral samples with mica content labels in the target domain training set. t，labeled The two satisfy the relationship: 0.01·N s，labeled ≤N t，labeled ≤0.1·N s，labeled .
7. The method according to claim 1, characterized in that, The triggering conditions for the evolution update strategy are at least one of the following: (1) the cumulative number of newly collected unlabeled samples in the target domain reaches 20 to 100; (2) the average relative deviation between the predicted value and the verified value of mica content in the target domain exceeds 5% to 15%; (3) the model deployment time exceeds 7 to 30 days. After triggering, all labeled and unlabeled data accumulated in the current target domain are merged and re-executed along with the fixed source domain training set.
8. The method according to claim 1, characterized in that, It also includes manually labeling samples with the highest information content from newly acquired unlabeled spectral data in the target domain using an active learning strategy to expand the labeled dataset in the target domain; the active learning adopts a sampling strategy based on prediction entropy, and the uncertainty score of the sample is defined as: Where p(c|x) t The model predicts the probability that the spectral sample belongs to mica category c, where C is the number of mica subspecies, including at least muscovite, illite, and biotite. The top 1 to 10 samples with the highest scores in each round are labeled, and the labeled samples are added to the target domain labeled dataset.
9. The method according to claim 1, characterized in that, The spectral data of both the source and target domains undergo a unified preprocessing procedure, including: denoising, baseline correction, normalization, and wavelength alignment; wherein, normalization employs a standard normal variable transformation. Where, x （i） Let i be the original spectral vector of the i-th sample. x is the mean of the corresponding sample spectral vectors. （i） - This represents the original spectrum minus its mean, resulting in a vector; m is the total number of wavelength points, and j is the index of the wavelength point, from 1 to m. Let be the original intensity value of the spectrum of the i-th sand sample at the j-th wavelength point. The result of the standard normal variable SNV transformation of the i-th sand sample spectrum is a vector containing the values of all wavelength points.
10. The method according to claim 1, characterized in that, The output of the mica content regression predictor R also includes a prediction confidence interval, and the prediction uncertainty is estimated using the Monte Carlo dropout method: repeated T mc The next forward propagation yields a set of predicted values { , , ..., The final predicted value is: The prediction standard deviation is: in, This is the k-th predicted value; the 95% confidence interval is [ -1.96σ pred , +1.96σ pred When the confidence interval width exceeds the preset threshold, the system automatically marks the detection result as low confidence.