A feature attention mechanism-deep network-based model maintenance method

Abstract

The application discloses a kind of based on feature attention mechanism-depth network's model maintenance method, belong to spectral analysis technical field.The method includes: first, by quantitatively analyzing the spectral difference between source domain and target domain samples, the significant difference features between two domain samples are identified;Second, add channel attention layer in the pre-trained deep network model, the feature recalibration is carried out to significant difference feature;Finally, using a small amount of new target domain samples combined with the model fine-tuning strategy of transfer learning, complete the model maintenance of cross-domain sample.The application can accurately locate the significant difference spectral band between different domain samples, and recalibrate the difference features through attention mechanism, combined with hierarchical fine-tuning strategy, effectively maintains the prediction performance of detection model in cross-domain scene.Compared with the traditional model maintenance method, the application has higher detection accuracy and stronger sample efficiency, and is not limited to specific spectral field and measured substance, and has strong universality.

Description

Technical Field

[0001] This invention belongs to the field of spectral analysis technology, specifically relating to a model maintenance method based on feature attention mechanism-deep network, which is applicable to model calibration and updating when the performance of spectral detection model degrades due to changes in the properties of the analyte sample or detection conditions. Background Technology

[0002] Spectroscopy is the scientific field that studies the interaction between matter (solids, liquids, and gases) and radiation. Substance content detection is a crucial research area in spectroscopy, using spectral analysis models to determine the composition or concentration of analytes or unknown substances. However, the stability and reliability of spectral analysis models largely depend on their accuracy. In practical applications, the sample properties of the analyte and the detection conditions significantly affect its data distribution and characteristics, leading to model performance degradation. Therefore, researching the impact of changes in sample properties and detection conditions on the model is of great significance for maintaining the performance of analytical models.

[0003] Currently, there are four main solutions to the problem of model performance degradation caused by the properties of the test sample and the detection conditions: (1) Global model method: The model is rebuilt after merging the samples of the source domain and the target domain. This method requires a large number of measured labels of the target domain samples, which is costly. When the sample properties change significantly, a single model is difficult to take into account the feature distribution of samples from different domains. (2) Model update method: The original model is corrected using a small number of target domain samples. Representative techniques include slope / bias correction (S / B correction) and recalibration. These methods usually only make linear adjustments to the model output and cannot handle nonlinear spectral shifts caused by changes in sample properties. The correction capability is limited. (3) Transfer learning method: The knowledge of the source domain is transferred to the target domain through domain adaptation or domain generalization techniques. Most existing transfer learning methods focus on the global alignment of feature distributions. They fail to identify the specific bands where the significant differences between the source domain and the target domain are located and lack a deep understanding of the spectral mechanism of the differences, resulting in low transfer efficiency. (4) Constraint optimization method: Apply regularization constraints during model training to enhance the model's robustness to domain changes. This type of method usually requires simultaneous access to source and target domain data and lacks targeted processing of differential features, making it difficult to achieve stable results in practical applications.

[0004] In summary, existing model maintenance methods generally have the following shortcomings: (1) They fail to effectively identify significant differences between source and target domain samples and lack in-depth analysis of how changes in sample attributes and detection conditions affect the spectrum; (2) They fail to actively integrate difference feature information into the model maintenance process and lack a targeted recalibration mechanism for difference features.

[0005] Therefore, there is an urgent need for a method that can quantitatively identify significant differences and guide model maintenance accordingly, so as to improve the robustness and prediction accuracy of spectral detection models in cross-domain scenarios. Summary of the Invention

[0006] To address the problems existing in existing technologies, this invention discloses a model maintenance method based on a feature attention mechanism and deep networks, belonging to the field of spectral analysis technology. The method includes: first, identifying significant difference features between source and target domain samples by quantitatively analyzing the spectral differences; second, adding a channel attention layer to a pre-trained deep network model to recalibrate the significant difference features; and finally, using a small number of new target domain samples combined with a transfer learning model fine-tuning strategy to complete model maintenance for cross-domain samples. This invention can accurately locate significant difference spectral bands between samples from different domains and perform targeted recalibration of difference features through an attention mechanism. Combined with a hierarchical fine-tuning strategy, it effectively maintains the predictive performance of the detection model in cross-domain scenarios. Compared with traditional model maintenance methods, this invention has higher detection accuracy and stronger sample efficiency, and is not limited to specific spectral domains or analytes, exhibiting strong universality.

[0007] On the one hand, this invention provides a model maintenance method based on a feature attention mechanism-deep network, the method comprising the following steps: S1: By quantitatively analyzing the spectral differences between the source domain sample and the target domain sample, significant difference features between the source domain sample and the target domain sample are identified; S2: Add a channel attention layer to the pre-trained deep network model, and use the channel attention layer to recalibrate the significant difference features; S3: Using a small number of target domain samples, combined with the model fine-tuning strategy of transfer learning, the deep network model that has completed feature recalibration is fine-tuned to complete the model maintenance for cross-domain samples.

[0008] In the field of spectral modeling, although some studies have attempted to introduce attention mechanisms to address cross-domain adaptation problems, these mechanisms are typically a global channel feature recalibration. This involves a general weighted enhancement of the original spectral features of the source domain samples, with the attention weights derived entirely from the source domain data. They are not specifically designed to address the significant differences between the source and target domains. They focus only on the discriminative features in the substance content prediction task, neglecting the specific bands representing the inter-domain distribution shift. Furthermore, most existing methods focus on the global alignment of feature distributions, failing to identify the specific bands containing the significant differences between the source and target domains, and lacking a deep understanding of the spectral mechanisms underlying these differences.

[0009] The proposed method, in step S1, directly and quantitatively identifies significant differences between the two domains at the raw spectral level by calculating the spectral ratio between the source and target domain samples and performing statistical analysis on their overall mean. More importantly, this identification result is not output as an independent conclusion, but rather directly serves as prior guidance information for the channel attention layer in step S2. The learnable parameter vector of the channel attention layer corresponds to the significant difference features. After being converted into attention weights through a mapping function, it performs targeted feature recalibration only on the significantly different feature bands, rather than applying indiscriminate weighting to all spectral features. This causal-driven mechanism of first quantitatively identifying significant difference features and then selectively recalibrating them enables the model to adaptively enhance the feature representation of the difference bands and suppress irrelevant interference, thereby effectively overcoming the nonlinear effects of cross-domain spectral shift and demonstrating a significant improvement.

[0010] In the final S3 step, the transfer learning strategy of only fine-tuning the task adapter (freezing the feature extractor) is combined, which can quickly complete the model maintenance with only a small number of target domain samples, significantly improving the cross-domain prediction accuracy while maintaining extremely high sample efficiency.

[0011] Furthermore, S1 specifically includes: S1.1: Calculate the spectral ratio between the source domain sample and the target domain sample to obtain the spectral ratio data of the target domain sample; S1.2: Perform statistical characteristic analysis on the spectral ratio data, and determine the spectral bands whose statistical index values ​​exceed the preset threshold as the significant difference features.

[0012] In some embodiments, the spectral ratio data is the population mean.

[0013] In spectral model maintenance, by calculating the spectral ratios between all samples in the source and target domains and performing statistical characteristic analysis on their overall mean, spectral bands whose statistical characteristics meet preset judgment criteria are directly identified as significantly different features. This is fundamentally different from traditional spectral model transfer methods: traditional methods typically use indirect means such as domain adaptation (e.g., maximum mean difference) or feature alignment to handle domain differences, processing the model's feature space rather than the original spectral data, resulting in poor interpretability; while this invention starts directly from the original spectral data, using spectral ratios as a quantification tool and statistical characteristics as the judgment criteria, achieving direct, quantitative, and interpretable identification of inter-domain difference bands. This processing paradigm is unique in the field of spectral model maintenance technology.

[0014] It is understood that this invention is not limited to the detection of specific spectral fields and specific substances, but is applicable to samples from different domains and their analytes that meet the requirement of significant differences between any spectral fields.

[0015] Furthermore, the statistical indicators used in the statistical characteristic analysis in S1.2 are at least one of deviation, variance, standard deviation, or coefficient of variation.

[0016] It is understood that, in addition to deviation values, those skilled in the art can also use statistical indicators such as variance, standard deviation, or coefficient of variation to measure the degree of fluctuation of the overall mean of spectral ratios in the wavelength dimension in order to identify significant differences.

[0017] Furthermore, in S1.2, the statistical index used for the overall mean of the spectral ratio in the wavelength dimension is the deviation value.

[0018] Furthermore, S2 specifically includes: S2.1: Select a deep network architecture based on the data dimension of the sample to be tested, and pre-train the deep network architecture using source domain samples to obtain the pre-trained deep network model; S2.2: Define the channel attention layer and introduce a learnable parameter vector corresponding to the significant difference feature into the channel attention layer. Convert the learnable parameter vector into attention weights through a mapping function and use the attention weights to recalibrate the features in the deep network model.

[0019] First, a deep network architecture is selected based on the data dimensions of the test samples, and pre-training is performed using source domain samples. This ensures a high degree of matching between the basic model and the data characteristics, laying a reliable foundation for feature extraction capabilities in subsequent transfer learning. Second, learnable parameter vectors corresponding to the significant difference features identified by S1 are introduced into the channel attention layer, and these vectors are converted into attention weights through a mapping function. This achieves targeted feature recalibration guided by prior difference information, enabling the model to adaptively enhance the feature response of key difference bands and suppress the influence of irrelevant bands. This method avoids the redundant computation caused by the indiscriminate processing of all channels in traditional attention methods, and overcomes the defect that purely data-driven attention may deviate from the direction of true difference, significantly improving the efficiency and accuracy of feature recalibration in cross-domain scenarios.

[0020] Furthermore, the mapping function is at least one of a linear function, an exponential function, a power function, or a logarithmic function, and the selection of the mapping function and its parameters is based on minimizing the prediction error of the pre-trained deep network model on the target domain test set.

[0021] Understandably, the selection of the mapping function and its parameters is based on minimizing the prediction error of the pre-trained deep network model on the new target domain test set, thus choosing the optimal mapping function and its parameters.

[0022] In some approaches, the parameters of the mapping function are optimized and determined within a preset space through a grid search.

[0023] In some embodiments, the mapping function is preferably a logarithmic function.

[0024] Furthermore, in S2.1, the typical deep network architecture is a one-dimensional deep network model that matches the characteristics of one-dimensional spectral data.

[0025] Furthermore, in S2.2, the dimension of the learnable parameter vector is consistent with the dimension of the spectral data of the sample to be tested.

[0026] Furthermore, S3 specifically includes: S3.1: Freeze all parameters related to the feature extractor in the pre-trained deep network model; S3.2: Using a small number of target domain samples, fine-tune all parameters related to the task adapter in the pre-trained deep network model; S3.3: Input the spectrum of the target domain sample into the fine-tuned deep network model and output the prediction result of the content of the substance to be measured.

[0027] Regarding the transfer fine-tuning strategy, this invention tightly integrates the freezing strategy with the differential features identified in S1 and the attention layer added in S2 in step S3, and only fine-tunes the task adapter (attention layer and fully connected layer), freezing all parameters in the feature extractor (convolutional layer and batch normalization layer). The differential information has been pre-injected into the model through attention weights, so there is no need to relearn in the fine-tuning stage.

[0028] In existing technologies, a certain scale of labeled target domain samples is usually required to obtain relatively stable transfer performance. However, this invention achieves optimal extrapolation prediction performance with only a 5%~10% update set ratio through the coordinated use of three steps: significant difference feature pre-identification, attention injection, and hierarchical freezing fine-tuning. For example, in the detection of jujube sugar content, using only about 13 target domain samples (10% update set ratio) for model maintenance significantly reduced the sugar content prediction error of "Zhanhua" jujubes from 3.68 °Brix when directly extrapolating to 0.86 °Brix, while simultaneously reducing the model from completely ineffective (…). R 2 v The efficiency was improved to explain 62% of the sugar content variation (from negative). This significant improvement in sample efficiency is the result of the synergistic effect of the three steps, yielding unexpected technical benefits.

[0029] Furthermore, the feature extractor includes convolutional layers and batch normalization layers, and the task adapter includes attention layers and fully connected layers.

[0030] Furthermore, the deep network model is any one of a one-dimensional convolutional neural network, a recurrent neural network, or a Transformer model.

[0031] On the other hand, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0032] In another aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0033] In another aspect, the present invention provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method described above.

[0034] The beneficial effects of this invention are as follows: 1. By quantitatively analyzing the spectral ratio between source and target domain samples and combining it with statistical indicators (such as deviation and variance) to perform wavelength-dimensional characteristic analysis, the key bands that cause model performance degradation can be directly located from the original spectral data, and significant differences can be accurately identified. This provides interpretable physical / chemical information guidance for model maintenance and enhances model interpretability.

[0035] 2. A channel attention layer is introduced into the pre-trained deep network, and the significant difference features identified by S1 are mapped into learnable parameter vectors. These vectors are then converted into attention weights through a mapping function, enabling targeted recalibration of the difference bands. This avoids the redundant computation and overfitting risks associated with the traditional attention method that treats all channels indiscriminately. In the cross-variety detection task from the source domain "Dali" winter jujube to the target domain "Zhanhua" winter jujube, this method significantly reduces the root mean square error of validation (RMSEV) from 3.68 °Brix when directly extrapolating to 0.86 °Brix. R 2 v The accuracy of predictions improved dramatically, from -0.24 to 0.62.

[0036] 3. Employing a hierarchical transfer learning strategy that freezes the feature extractor (convolutional layers and batch normalization layers) and only fine-tunes the task adapter (attention layers and fully connected layers), this approach preserves the general spectral features learned by the feature extractor, given that S2 has already injected the difference information into the model through attention weights. Effective adaptation requires only a small number of target domain samples. This method achieves optimal model maintenance performance, reaching its best with only 5%–10% of samples, demonstrating a revolutionary advantage in sample efficiency.

[0037] 4. This invention freezes all parameters of the feature extractor (including convolutional layers and batch normalization layers) during the fine-tuning stage, ensuring that the stable and universal spectral feature representation ability learned by the pre-trained model from source domain samples is not compromised. This design effectively avoids the problems of basic feature degradation or overfitting caused by fine-tuning with a small number of target domain samples, enabling the model to retain the memory of effective features from the source domain while adapting to the new domain, achieving a balance between adapting to the new domain and not forgetting prior knowledge.

[0038] 5. Regardless of changes in sample attributes (such as variety, origin, maturity) or changes in detection conditions (such as instrument status, ambient temperature, optical path structure), as long as the differential features can be quantified and identified through spectral ratios, the model performance can be effectively maintained, and it has a very strong cross-scenario generalization ability.

[0039] 6. This invention employs a three-tiered progressive mechanism of positioning (S1), calibration (S2), and adaptation (S3) to establish a clear transmission path for discrepancies between modules. The overall effect far surpasses the simple superposition of single technologies, exhibiting significant synergistic benefits. It achieves superior extrapolation prediction performance compared to five traditional methods—global model, slope / bias correction, recalibration, deep network update, and semi-supervised parameterless calibration enhancement—when the update set ratio is ≥10%. Furthermore, it maintains optimal robustness across different sample sizes. This ability to achieve a leapfrog performance improvement across domains even with small sample sizes demonstrates significant technological advancement.

[0040] In summary, this invention balances accurate localization of differential features, targeted calibration, and low-cost model adaptation; it can identify significant differential features between samples from different domains, overcome the limitations of existing model maintenance methods, and better maintain the performance of detection models; it achieves superior results compared to existing technologies in terms of model prediction accuracy, sample efficiency, robustness, interpretability, and universality, providing an efficient, reliable, and widely applicable solution to the model maintenance problem in the field of spectral detection. Attached Figure Description

[0041] Figure 1 This is a flowchart of the present invention; Figure 2 This is a schematic diagram illustrating the principle of the method of the present invention; Figure 3 These are spectral diagrams of two winter jujube varieties from embodiments of the present invention; Figure 4 This is a sugar content distribution diagram of two winter jujube varieties in an embodiment of the present invention; Figure 5 This is a spectrum identification diagram showing the significant difference in spectral bands between two winter jujube varieties in this embodiment of the invention; Figure 6This refers to the optimal recalibration mapping function and its parameters obtained by learning the method for variety 2 samples in this embodiment of the invention. Figure 7 This is the prediction result of the 1D-CNN benchmark model built in this embodiment of the invention on the extrapolation test set of variety 2 samples; Figure 8 Yes, this is the prediction result of the model built using this method in this embodiment of the invention on the extrapolation test set of variety 2 samples; Figure 9 The prediction results of six model maintenance methods on the extrapolation test set of variety 2 samples in this embodiment of the invention; Figure 10 This is the sample efficiency analysis result of six model maintenance methods in the embodiments of the present invention under different update set ratios for variety 2 samples. Detailed Implementation

[0042] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be noted that the embodiments described below are intended to facilitate understanding of the present invention and are not intended to limit it in any way. The instruments and equipment used in this embodiment are all known products and were obtained by purchasing commercially available products.

[0043] Example 1: Model maintenance method based on feature attention mechanism-deep network provided by the present invention The model maintenance method based on feature attention mechanism-deep network provided in this embodiment has the following flowchart: Figure 1 As shown, the principle of the method is as follows: Figure 2 As shown, it includes the following steps: S1: The source domain sample of the substance to be tested is determined to be sample 1, and the target domain sample is sample 2. By quantitatively analyzing the spectral differences between sample 1 and sample 2, the significant difference characteristics between the two different sample 1 are identified. S1 specifically refers to: S1.1: Calculate the spectral ratio between all samples of variety 1 and variety 2, obtain the mean of the spectral ratio of the variety 2 samples, and obtain the overall mean of the spectral ratio of all variety 2 samples; S1.2: The statistical indicator used for the overall mean of the spectral ratios in the wavelength dimension is the deviation value. In specific implementation, the deviation value of the overall mean of the spectral ratios of sample variety 2 in the wavelength dimension is analyzed, and the spectral bands with larger deviation values ​​of the overall mean are identified as significant differences.

[0044] S2: Add a channel attention layer to the pre-trained one-dimensional convolutional neural network (1D-CNN) model to recalibrate features with significant differences. S2 specifically refers to: S2.1: Based on the one-dimensional spectral data of the sample to be tested, a typical 1D-CNN suitable for spectral data analysis is selected as the deep network architecture, and the 1D-CNN model is pre-trained using sample 1 of the substance to be tested. In specific implementation, the typical deep network architecture is a one-dimensional deep network model that matches the characteristics of the one-dimensional spectral data; The specific process of 1D-CNN model pre-training is as follows: A 1D-CNN model was constructed, comprising three convolutional-pooling modules (convolutional layers followed by ReLU activation functions and max-pooling layers), two fully connected layers, and finally, a single neuron outputting the predicted content of the analyte. In practice, the 1D-CNN model was trained using spectral data of sample 1 of variety and the content data of the analyte. The Adam optimizer was used for training, and hyperparameters were optimized within a preset space using grid search. To prevent overfitting, dropout, L2 norm regularized weight decay, and an early stopping strategy were employed (training automatically terminated when the validation set performance did not improve within eight consecutive iterations).

[0045] The hyperparameter grid search preset spaces are as follows: the kernel size search space for the three convolutional layers is [3, 5, 7, 9], the stride search space for the three convolutional layers is [1, 2, 3], the number of neurons in the hidden layer is [8, 16, 32, 64], the random inactivation rate search space is [0, 0.2, 0.5], the L2 regularization coefficient search space is [0.05, 0.1, 0.3], and the batch size search space is [16, 32, 64].

[0046] S2.2: Define a channel attention layer and introduce a learnable parameter vector corresponding to the significantly different features. This vector is then converted into attention weights through a mapping function, completing the recalibration of the significantly different features. In practice, the dimension of the learnable parameter vector is consistent with the dimension of the spectral data of the tested substance variety 2 samples. The mapping function is the optimal form among four common functions: linear, exponential, power, and logarithmic. The selection of the optimal mapping function and its parameters is based on minimizing the average root mean square error of validation produced by the pre-trained 1D-CNN model on the new variety 2 sample test set.

[0047] The specific process by which the mapping function transforms the learnable parameter vector into attention weights is as follows:

[0048] In the formula, w For attention weights, α For learnable parameter vectors, a and b For a specific parameter of the mapping function, its value range is: , .

[0049] The formulas for the four common mapping functions are as follows:

[0050] S3: Using a small number of new variety 2 samples combined with a transfer learning model fine-tuning strategy, the model maintenance for cross-variety samples is completed.

[0051] S3 specifically refers to: S3.1: Freeze all parameters related to the feature extractor in the pre-trained 1D-CNN model. In practice, the feature extractor includes convolutional layers and batch normalization layers, and the learning rate of all its weights, biases, and batch normalization parameters is set to 0.

[0052] S3.2: Using a small number of new variety 2 samples, fine-tune all parameters related to the task adapter in the pre-trained 1D-CNN model. Specifically, the task adapter includes an attention layer and a fully connected layer. The parameters of the attention layer are... α The learning rate for all fully connected layer parameters is set to 1.

[0053] S3.3: Input the spectrum of the new variety 2 sample into the fine-tuned 1D-CNN model and output the sugar content prediction result of the variety 2 sample.

[0054] Example 2: Performance Maintenance of Sugar Content Detection Models for Different Winter Jujube Varieties This embodiment uses the method provided in Example 1 to verify the effectiveness of the method proposed in this invention by taking the sugar content (soluble solids content, °Brix) of different varieties of winter jujubes as an example in visible / near infrared spectroscopy detection.

[0055] 1. Experimental Samples and Spectral Acquisition This embodiment uses two jujube varieties: "Dali" (referred to as the source domain, variety 1) and "Zhanhua" (referred to as the target domain, variety 2), purchased from a local supermarket in Hangzhou, Zhejiang Province, China. There are 219 samples of "Dali" jujubes and 132 samples of "Zhanhua" jujubes. All samples are mature, undamaged, and uniform in appearance.

[0056] Spectroscopic acquisition was performed in transmission mode using a halogen tungsten lamp as the light source and a QE65 Pro spectrometer (Ocean Insight, USA). The acquired spectral range was 560-920 nm. Simultaneously, a digital refractometer (PR-101α, Atago Co., Ltd., Japan) was used to measure the sugar content of each sample as the true content of the analyte.

[0057] 2. Implementation Steps Step S1: Identify significant difference features First, calculate the spectral ratio between all samples in the source domain (variety 1) and the target domain (variety 2). Specifically, for each wavelength, calculate the ratio of the spectral value of each variety 2 sample to the average spectral value of all variety 1 samples at that wavelength, and then average the ratios of all variety 2 samples to obtain the overall mean of the spectral ratio.

[0058] like Figure 3 As shown, the spectral absorption peaks of samples from varieties 1 and 2 are concentrated at 670 nm and 750 nm. The spectral morphology of the two varieties is similar after 720 nm, but there are some differences before 720 nm. Figure 4 The sugar content distribution of two jujube varieties is shown. The sugar content range of variety 1 is 11.2~20.0 °Brix, and that of variety 2 is 12.7~18.6 °Brix. There is a significant difference in the sugar content distribution between the two varieties.

[0059] Then, the statistical characteristics of the overall mean in the wavelength range of 560–920 nm are analyzed. In this embodiment, the deviation value is used as a statistical indicator. The deviation value reflects the degree to which the ratio data deviates from 1; the larger the deviation value, the more significant the spectral difference between the two varieties at that wavelength.

[0060] By calculating the overall mean of the spectral ratio between sample 2 and sample 1, and analyzing the deviation of this result in the wavelength dimension, the spectral difference between the two sample varieties is obtained. Figure 5 This shows the distribution of the overall mean deviation along the wavelength dimension of the spectral ratio of variety 2 to variety 1 samples. Figure 5 It is evident that the deviation values ​​exhibit significant nonlinear fluctuations in the 560–720 nm band, indicating that the spectral response in this band is most significantly affected by variety variations. Therefore, the 560–720 nm band is identified as a significant difference between the source and target domains.

[0061] Step S2: Add channel attention layers and perform feature recalibration S2.1: Pre-trained deep network model Based on the one-dimensional spectral data characteristics of the sample to be tested in this embodiment, a one-dimensional convolutional neural network (1D-CNN) is selected as the deep network architecture.

[0062] A 1D-CNN model was used as the baseline model for sugar content detection. Traditional model maintenance methods such as global model, slope / bias correction, recalibration, deep network update, and semi-supervised parameterless calibration enhancement were used as baseline methods.

[0063] The constructed 1D-CNN model contains three convolutional-pooling modules (each module consists of a one-dimensional convolutional layer, a ReLU activation function layer, and a max pooling layer, followed by two fully connected layers, and finally a single neuron outputs the sugar content prediction value.

[0064] The 1D-CNN model was pre-trained using spectral data and corresponding measured sugar content values ​​from 219 samples of variety 1 (source domain). Training employed the Adam optimizer, with hyperparameters determined via grid search. The search space consisted of the following parameters: kernel size {3, 5, 7, 9}, stride {1, 2, 3}, number of hidden layer neurons {8, 16, 32, 64}, random inactivation rate {0, 0.2, 0.5}, L2 regularization coefficient {0.05, 0.1, 0.3}, and batch size {16, 32, 64}. To prevent overfitting, an early stopping strategy was employed (training was terminated if validation set performance did not improve within eight consecutive iterations). After training, a pre-trained 1D-CNN baseline model was obtained.

[0065] S2.2: Add a channel attention layer Add a channel attention layer before the fully connected layers of the pre-trained 1D-CNN model. This layer introduces a learnable parameter vector. α Its dimension is consistent with the dimension of spectral data (in this embodiment, it is the number of spectral channels).

[0066] Based on the significantly different characteristic bands (560~720nm) identified by S1, this learnable parameter vector α Elements within this band are initialized to non-zero values, while other bands are initialized to zero, in order to guide attention weights to focus on bands of difference.

[0067] Define a mapping function f Learnable parameter vectors α Converted into final attention weights w = f ( α In this embodiment, four common mapping functions were examined: linear functions, exponential functions, power functions, and logarithmic functions.

[0068] from Figure 6 The average RMSEV of the pre-trained model on the Variety 2 test set is shown when different mapping functions are used. The results indicate that when the logarithmic function is used... w = log( a + exp( b · α When the RMSEV of the 1D-CNN model reaches its minimum value of 0.946 °Brix, the optimal parameters are: a =1, b=2. Therefore, this logarithmic function is chosen as the optimal mapping function for this instance.

[0069] The attention layer will calculate the attention weights w By multiplying the input feature map channel by channel, feature recalibration is performed on significantly different features, which enhances the model's response to features in different bands and suppresses irrelevant information.

[0070] Step S3: Migration Fine-tuning S3.1: Freeze Feature Extractor Parameters In the model processed by S2, all parameters related to the feature extractor are frozen. In this embodiment, the feature extractor includes all convolutional layers and batch normalization layers. The freezing operation sets the learning rate of these parameters to 0, ensuring they remain unchanged during subsequent fine-tuning.

[0071] S3.2: Fine-tuning task adapter parameters Only the parameters related to the task adapter in the model are set to a learnable state. In this embodiment, the task adapter includes the newly added attention layer and all fully connected layers. These learnable parameters are fine-tuned using a small number of target domain (Variety 2) samples. In this embodiment, 10% (approximately 13 samples) of the 132 Variety 2 samples are randomly selected as the update set, and the remainder is used as the extrapolation test set.

[0072] S3.3: Model Prediction Input the spectral data of the extrapolation test set of Variety 2 into the fine-tuned model, and the model output is the sugar content prediction value.

[0073] 3. Experimental Results and Comparative Analysis 3.1 Model Maintenance Effectiveness This embodiment also established a 1D-CNN benchmark model using samples from variety 1, and performed extrapolation tests on samples from variety 2. The prediction results of the 1D-CNN benchmark model trained only on the source domain (variety 1) directly applied to the target domain (variety 2) test set are as follows: Figure 7 As shown.

[0074] from Figure 7 It can be seen that the 1D-CNN baseline model performs well on its prediction set, with a prediction set determination coefficient (...). R 2 p The validation set determination coefficient was 0.91, and the root mean square error (RMSEP) of the prediction set was 0.72 °Brix. However, when the model was directly applied to the extrapolated test set of variety 2 samples, the predictive performance deteriorated sharply, and its validation set determination coefficient (... R 2 v The value is -0.24, and the RMSEV is 3.68 °Brix. The model is completely invalid.

[0075] After maintaining the performance of the 1D-CNN benchmark model using the method proposed in this invention, the model's ability to extrapolate and predict samples of variety 2 is significantly improved, achieving [the desired improvement]. R 2 v The value is 0.62, and the RMSEV is 0.86 °Brix. Figure 8 As shown in the figure. The results indicate that the method of the present invention successfully solves the problem of model failure caused by variety changes.

[0076] 3.2 Comparison with traditional methods This embodiment also uses five traditional model maintenance methods—global model, slope / bias correction, recalibration, deep network update, and semi-supervised parameterless calibration enhancement—as baseline methods to perform extrapolation predictions on variety 2 samples, in order to verify the effectiveness of the proposed method. The results are as follows: Figure 9 As shown.

[0077] The results are as follows Figure 9 As shown, the five traditional model maintenance methods all exhibit good performance on both the calibration set and the prediction set, with their calibration set determination coefficients (... R 2 c )and R 2 p The error rates are no less than 0.89 and 0.87, respectively, and the root mean square error of the correction set (RMSEC) and RMSEV are no more than 0.86 °Brix and 0.92 °Brix, respectively. In terms of extrapolation prediction performance, the global model has the best extrapolation prediction performance, followed closely by the method proposed in this invention. More importantly, the global model requires access to a large number of labeled samples in both the source and target domains simultaneously, resulting in extremely high costs; while this invention only requires a small number of target domain samples to achieve equivalent performance. Specifically, on variety 2 samples, the update set root mean square error (RMSEU) and RMSEV of all methods are controlled within 0.97 °Brix and 1.18 °Brix, respectively.

[0078] The extrapolation prediction performance of six model maintenance methods was tested at update set proportions of 5%, 10%, 15%, and 20% to evaluate their robustness under different sample sizes. The results are as follows: Figure 10 As shown. The method, global model, and slope / bias correction RMSEV provided by this invention are less affected by the updated sample size, while the extrapolation prediction performance of the recalibration method is highly dependent on the updated sample size. Furthermore, when the updated set proportion reaches 10% or more, the method provided by this invention exhibits the best prediction performance on the extrapolated test set, demonstrating good robustness and generalization ability.

[0079] As can be seen from this embodiment, the model maintenance method proposed in this invention achieves better performance and detection accuracy than traditional model maintenance methods in establishing prediction models for the sugar content of different varieties of winter jujubes, demonstrating that this invention has superior capabilities in model maintenance.

[0080] Example 3: Synergy and Indispensability of the Three Steps This embodiment uses the dataset from Embodiment 2, and the evaluation metrics include RMSEV and R 2 v By designing the following six sets of comparative experiments, the three core steps of the present invention were removed or replaced respectively, in order to quantitatively evaluate the independent contribution and synergistic effect of each step.

[0081] Experiment A (Complete Invention Scheme): Using S1 (spectral ratio identification of significant difference features) + S2 (adding channel attention layers for difference features, employing the optimal logarithmic mapping function) + S3 (freezing the feature extractor, only fine-tuning the task adapter). The update set ratio is 10%.

[0082] Experiment B (Removal of S1: Blind Attention Mechanism): Step S1 is not performed, i.e., significant difference bands are not pre-identified through spectral ratio analysis. In step S2, a standard channel attention layer (SENet structure) is directly added to the pre-trained network. Its learnable parameter vector is randomly initialized on all wavelengths and learned entirely by fine-tuning a small number of samples in the target domain. Step S3 remains unchanged.

[0083] Experiment C (Removing S2: No Attention Recalibration): Step S2 is not performed, i.e., channel attention layers are not inserted into the deep network. Only S1 (identifying significantly different bands) and S3 (layer freezing and fine-tuning) are performed. This setting aims to verify whether identifying only the different features without recalibrating them through an attention mechanism can effectively maintain the model.

[0084] Experiment D (Replace S3: Full Parameter Fine-Tuning): Perform S1 and S2, but instead of using the tiered freezing strategy in step S3, use the same small number of target domain samples to globally fine-tune all parameters of the entire network (including the feature extractor and task adapter).

[0085] Experiment E (S3 removed): With S1+S2, but completely skipping S3 (i.e., no fine-tuning). After adding the attention layer, the model parameters remain unchanged, directly predicting on the target domain test set. This experiment verifies whether S1+S2 alone can solve cross-domain problems without any model parameter adaptation.

[0086] Experiment F (replacing S1): In S1, instead of using spectral ratios to identify significant difference features, principal component analysis (PCA) was performed on the source and target domain samples respectively. The loading vector of the first principal component (PC1) was extracted, and the absolute difference between the loading vectors of the two domains in wavelength was calculated. The top 30% of wavelength bands with the largest difference were identified as significant difference features and used to identify the wavelengths that contribute the most to the domain difference.

[0087] Experiment G (replacing S2): Instead of adding a channel attention layer in S2, a domain discriminator (binary classification: source domain / target domain) is added after the pre-trained 1D-CNN feature extractor, and a gradient inversion layer is inserted. In the fine-tuning phase (S3), both the prediction task and the domain discrimination task (DANN) are optimized simultaneously. This method aims to achieve feature alignment through adversarial training.

[0088] All other aspects not mentioned are consistent with Example 2, and the experimental analysis results are shown in Table 1.

[0089] Table 1. Comparison of Experimental Results

[0090] As shown in Table 1, comparing Experiment A and Experiment B: when S1 was removed and the prior guidance of the attention mechanism lost its significant difference features (Experiment B), the RMSEV of the model plummeted from 0.86 °Brix to 2.53 °Brix. R 2 v The value decreased from 0.62 to -0.68. This indicates that the blind, data-driven attention mechanism struggles to accurately focus on the true inter-domain difference bands under small sample target domain conditions, resulting in a significant reduction in recalibration effectiveness and failing to fully unleash the potential of the attention mechanism. S1 provides precise spatial guidance for S2, and the combination of the two produces a synergistic effect greater than the sum of its parts.

[0091] Comparing Experiment A and Experiment C: When only significantly differentiating features are identified but recalibrated through an attention layer (Experiment C), model performance deteriorates significantly, with RMSEV increasing to 2.35 °Brix. R 2 v The result is negative (-0.51). This strongly demonstrates that simply knowing where the discrepancies occur (the output of S1) without explicitly recalibrating the deep features within the model through a learnable attention mechanism (S2) cannot effectively overcome the impact of inter-domain offset on prediction performance. S2 is the key bridge for transforming the information generated by S1 into the model.

[0092] Comparing Experiment A and Experiment D: When using the full parameter fine-tuning strategy (Experiment D), although it also has the guidance of S1 and S2, the performance is still lower than the complete scheme (RMSEV increases to 1.02 °Brix).R 2 v (0.56). This is because, with extremely limited samples in the target domain (10%), full parameter fine-tuning is prone to overfitting and can destroy the underlying spectral features (such as absorption peak shapes) learned by the pre-trained model in the source domain that are generally effective for sugar content prediction. The layered freezing strategy of S3 in this invention, by solidifying the feature extractor, perfectly preserves general knowledge and only fine-tunes the task adapter directly related to the differential features, achieving information preservation and updating. S3 ensures that the efforts of S1 and S2 are not destroyed by subsequent fine-tuning processes, and is a necessary guarantee for the synergistic effect of the three.

[0093] The RMSEV of experiment E (S1+S2 only, without any fine-tuning) is 2.04 °Brix. R 2 v The value is -0.31. This strongly demonstrates that simply identifying differential features and inserting an attention layer into the network without subsequent parameter fine-tuning (S3) cannot truly adapt the model to the target domain. The initial weights of the attention layer obtained through S2 (even if guided by S1) are still initialized on the source domain distribution. They must be fine-tuned through S3 with a small number of target domain samples to transfer the recalibration capability to the target domain. S3 is the key activation step that brings the model performance from "0" to "1".

[0094] Compared with Experiment A and Experiment F, although the alternative method can also find some difference bands, it has problems such as discontinuous bands, narrow range or positional shifts compared with the continuous and wide-range (560~720 nm) significant difference features identified by S1 of this invention, which significantly affects the detection accuracy.

[0095] Comparing Experiments A and G, although DANN is classic and effective in the field of domain adaptation, its performance is significantly worse than that of the present invention S2 under the condition of small sample size (10% of the target domain) in this task (RMSEV = 1.74 °Brix vs 0.86 °Brix). This is because DANN requires sufficient target domain samples to stabilize the adversarial training process, and it pursues globally domain-invariant features rather than specifically calibrating significantly different features. This leads to the key information of the prediction task (glycemia regression) being suppressed by the domain discriminator. The targeted recalibration of S2 is more suitable for the scenario of this invention than global domain alignment.

[0096] Therefore, the study in this embodiment demonstrates that the three steps of the present invention—S1 (identification of significant spectral differences), S2 (channel attention recalibration), and S3 (layered freeze fine-tuning)—are interdependent and indispensable. They form a complete technical loop; the absence or replacement of any step will lead to a significant decrease in the overall technical effect, confirming a clear synergistic effect among the three.

[0097] In summary, the model maintenance method based on feature attention mechanism-deep network proposed in this invention can overcome the limitations of existing model maintenance methods, better maintain the performance of detection models, and is not limited to the detection of specific spectral fields and specific substances, thus having strong universality.

[0098] It should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of them. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A model maintenance method based on feature attention mechanism-deep network, characterized in that, Includes the following steps: S1: By quantitatively analyzing the spectral differences between the source domain sample and the target domain sample, significant difference features between the source domain sample and the target domain sample are identified; S2: Add a channel attention layer to the pre-trained deep network model, and use the channel attention layer to recalibrate the significant difference features; S3: Using a small number of target domain samples, combined with the model fine-tuning strategy of transfer learning, the deep network model that has completed feature recalibration is fine-tuned to complete the model maintenance for cross-domain samples.

2. The method according to claim 1, characterized in that, S1 specifically includes: S1.1: Calculate the spectral ratio between the source domain sample and the target domain sample to obtain the spectral ratio data of the target domain sample; S1.2: Perform statistical characteristic analysis on the spectral ratio data, and determine the spectral bands whose statistical index values ​​exceed the preset threshold as the significant difference features.

3. The method according to claim 2, characterized in that, The statistical indicator in S1.2 is at least one of deviation, variance, standard deviation, or coefficient of variation.

4. The method according to claim 1, characterized in that, S2 specifically includes: S2.1: Select a deep network architecture based on the data dimension of the sample to be tested, and pre-train the deep network architecture using source domain samples to obtain the pre-trained deep network model; S2.2: Define the channel attention layer and introduce a learnable parameter vector corresponding to the significant difference feature into the channel attention layer. Convert the learnable parameter vector into attention weights through a mapping function and use the attention weights to recalibrate the features in the deep network model.

5. The method according to claim 4, characterized in that, The mapping function in S2.2 is at least one of a linear function, an exponential function, a power function, or a logarithmic function. The selection of the mapping function and its parameters is based on minimizing the prediction error of the pre-trained deep network model on the target domain test set.

6. The method according to claim 1, characterized in that, S3 specifically includes: S3.1: Freeze all parameters related to the feature extractor in the pre-trained deep network model; S3.2: Using a small number of target domain samples, fine-tune all parameters related to the task adapter in the pre-trained deep network model; S3.3: Input the spectrum of the target domain sample into the fine-tuned deep network model and output the prediction result of the content of the substance to be measured.

7. The method according to claim 6, characterized in that, The feature extractor in S3.1 includes a convolutional layer and a batch normalization layer, and the task adapter in S3.2 includes an attention layer and a fully connected layer.

8. The method according to claim 1, characterized in that, The deep network model is any one of a one-dimensional convolutional neural network, a recurrent neural network, or a Transformer model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images