Near infrared spectrum cross-device migration method and system based on comparative learning

Abstract

The invention belongs to the technical field of near-infrared cross-device migration, and particularly relates to a near-infrared spectrum cross-device migration method and system based on comparative learning. Acquiring a near infrared spectrum from the source equipment and inputting the near infrared spectrum into the trained migration model to obtain a spectrum close to the near infrared spectrum from the target equipment in overall distribution and local structure; the migration model is constructed based on a CUT framework, the Patch is defined as a local segment of a spectral signal in a wavelength dimension and is used for representing feature information of a local region, and the comparative learning loss used in the generator training process comprises application of the Patch-based comparative learning loss to a source domain spectrum from source equipment and application of the Patch-based comparative learning loss to a target domain spectrum from target equipment. Application of the Patch-based contrast learning loss to the target domain spectrum may prevent the generator from making unnecessary, destructive modifications to the target domain spectrum that have been the target domain spectrum. According to the method, cross-equipment high-quality spectrum alignment is realized under the condition of no standard sample, the deployment difficulty is reduced, and the migration stability is improved.

Description

Technical Field

[0001] This invention belongs to the field of near-infrared cross-device transfer technology, specifically relating to a near-infrared spectral cross-device transfer method and system based on contrastive learning. Background Technology

[0002] Near-infrared spectroscopy (NIR) is widely used in industries such as tobacco, agriculture, food, and pharmaceuticals due to its advantages of fast detection speed, high accuracy, ease of operation, and non-destructive measurement. It can be used to predict the qualitative and quantitative properties of samples. However, spectral data of the same sample collected on different devices often show inconsistent distributions. This difference usually stems from factors such as instrument hardware (e.g., detector type, light source characteristics), spectral acquisition parameters (e.g., wavelength range, resolution), and environmental conditions (e.g., temperature, humidity). Consequently, models built on one device become less adaptable to other devices.

[0003] To address the aforementioned issues, Calibration Transfer (CT) technology was proposed, the core of which is to reduce or eliminate offsets caused by differences in instruments. Spectral differences can be seen as the external manifestation of such deviations, while also reflecting the characteristics of different instruments. Based on whether or not they rely on standard samples, existing methods can be divided into two categories: (1) Standard sample-based methods, such as Direct Standardization (DS), Piecewise Direct Standardization (PDS), and Canonical Correlation Analysis (CCA). These methods require simultaneous measurement of the same sample on both primary and secondary instruments to construct a transfer function to reduce spectral differences. However, in practical applications, obtaining standard samples is often time-consuming, labor-intensive, and costly, and is limited by sample stability and instrument conditions, sometimes even impossible. (2) Standard sample-independent methods, such as Variable Penalty Dynamic Time Warping (CT-VPdtw), Multivariate Scatter Correction (MSC), and Finite Impulse Response (FIR) Filtering. Although such methods can reduce the difference between the source and target domain spectra to some extent, they are usually achieved by constructing a reference spectrum (such as using the arithmetic mean of the target domain sample spectra as a benchmark). This approach tends to ignore the full complexity of the spectral data and may not be able to fully preserve subtle spectral features and overall structural information, thus affecting the model's generalization ability and transfer performance.

[0004] Chinese invention patent application CN118603932A, published on September 6, 2024, discloses a standard-free model transfer method based on near-infrared spectroscopy. This method predicts the index value of the sample to be tested by correlating the calibration set spectrum and reference values ​​of the hybrid model. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for cross-device near-infrared spectroscopy transfer based on contrastive learning, in order to solve the problems of existing calibration transfer methods, such as dependence on standard samples, insufficient spectral distribution alignment accuracy, incomplete preservation of spectral information, and poor model adaptability across different devices.

[0006] To address the aforementioned technical problems, this invention provides a method for cross-device transfer of near-infrared spectroscopy based on contrastive learning, the method comprising:

[0007] The near-infrared spectrum from the source device is acquired and input into a trained transfer model. The trained transfer model is then used to generate a spectrum that is close to the near-infrared spectrum from the target device in terms of both overall distribution and local structure.

[0008] The transfer model is built on the CUT framework, and the patch-based contrastive learning loss used in the generator training process includes applying patch-based contrastive learning loss to the source domain spectrum from the source device and applying patch-based contrastive learning loss to the target domain spectrum from the target device. The patch is defined as a local segment of the spectral signal in the wavelength dimension, used to characterize the feature information of the local region.

[0009] Furthermore, the expression for the patch-based contrastive learning loss applied to the source domain spectrum from the source device is as follows:

[0010] ; ;

[0011] In the formula, Indicates the spectrum from the source region Applying patch-based contrastive learning loss, Represents a generator. This represents a projection network used to map local features output from the encoding layer to a contrastive learning space. Indicates the source region spectrum A single input sample was sampled. Indicates data distribution by source domain Extracted samples The loss is taken as the mathematical expectation; The generator's first... A specified layer; Indicates the first One patch; Indicates the first The first layer extracted from the specified layer The output features of each patch Indicates the first The actual data extracted from a specified layer for comparison. Exclude the first patch Negative sample features of other patches in a given patch Indicates the first The first layer extracted from the specified layer Input features of each patch; , and These represent the query vector, the input positive sample vector, and the input negative sample vector for the query patch, respectively. Indicates the first indivual ; Represents cross-entropy loss; This indicates the temperature hyperparameter.

[0012] Furthermore, the generative adversarial loss used in training the transfer model is the least squares generative adversarial loss.

[0013] Furthermore, the generator in the transfer model includes an encoding layer, a feature transformation layer, and a decoding layer; the encoding layer is used to extract multi-scale features of the input spectrum; the feature transformation layer includes multiple residual blocks, which are used to transfer encoded features to decoded features; and the decoding layer is used to output the final spectrum.

[0014] Furthermore, the coding layer comprises three coding units connected in sequence. The first coding unit comprises a one-dimensional convolution, instance normalization, and activation function connected in sequence, while the latter two coding units each comprise a one-dimensional convolution, instance normalization, activation function, and downsampling connected in sequence.

[0015] Furthermore, the residual block comprises sequentially connected one-dimensional convolution, instance normalization, activation function, one-dimensional convolution, and instance normalization, with the input of the first one-dimensional convolution and the output of the last instance normalization being skipped.

[0016] Furthermore, the decoding layer includes two decoding units and an output unit connected in sequence. Each decoding unit includes upsampling, one-dimensional convolution, instance normalization and activation function connected in sequence. The output unit includes one-dimensional convolution and activation function connected in sequence.

[0017] Furthermore, the discriminator in the transfer model includes four discriminator units connected in sequence and an output unit. The first discriminator unit includes a one-dimensional convolution, an activation function, and downsampling connected in sequence. The two middle discriminator units each include a one-dimensional convolution, instance normalization, activation function, and downsampling connected in sequence. The last discriminator unit includes a one-dimensional convolution, instance normalization, and activation function connected in sequence. The output unit includes a one-dimensional convolution.

[0018] Furthermore, the near-infrared spectrum obtained from the source device is a one-dimensional spectrum.

[0019] To address the aforementioned technical problems, the present invention also provides a near-infrared spectral cross-device transfer system based on contrastive learning, comprising a processor for executing a computer program to implement the steps of the near-infrared spectral cross-device transfer method based on contrastive learning described above.

[0020] Its beneficial effects are as follows: This invention applies the CUT framework, originally used in image domain transfer tasks, to near-infrared spectroscopy cross-device transfer tasks. This enables the constructed transfer model to effectively model spectral differences between different devices, addressing the dependence of existing calibration transfer methods on standard samples. The transfer model can convert near-infrared spectra acquired by the source device into data highly consistent with the target device's spectrum, thereby reducing spectral domain deviations caused by hardware differences and acquisition conditions. This achieves cross-device calibration transfer and solves problems such as insufficient spectral distribution alignment accuracy, incomplete retention of spectral information, and poor model adaptability across different devices, further improving the quality and stability of cross-device spectrum generation. Specifically, the loss function used to train the generator of this transfer model includes a patch-based contrastive learning loss, maximizing the mutual information between the source and generated spectra at corresponding positions within a multi-layer feature space. This effectively preserves the valid information of the source domain spectrum while achieving high-quality alignment between the generated spectrum and the true target domain spectrum. Unlike the patches in image translation, which originate from two-dimensional spatial locations, the patches in this invention are defined as local segments of spectral signals in the wavelength dimension, used to characterize the feature information of the local region. Furthermore, not only is a patch-based contrastive learning loss applied to the source domain spectrum from the source device, but also to the target domain spectrum from the target device. This is a self-supervised regularization that prevents the generator from making unnecessary and destructive modifications to the already target domain spectrum. It provides additional self-supervised signals to the transfer model, helping it to better learn the feature distribution of the target domain. This allows the generator to produce results closer to the target domain spectrum while retaining key information when converting the source domain spectrum, significantly improving the final generation effect. Attached Figure Description

[0021] Figure 1This is a flowchart of the near-infrared spectroscopy cross-device transfer method based on contrastive learning according to the present invention;

[0022] Figure 2 This is a schematic diagram of the overall framework of the CUT-based model of the present invention;

[0023] Figure 3 This is a network architecture diagram of the generator of the present invention;

[0024] Figure 4 This is a network architecture diagram of the discriminator of the present invention;

[0025] Figure 5(a) is a performance comparison chart of the CUT-based model and the DS method of the present invention under the RMSE evaluation index;

[0026] Figure 5(b) is a performance comparison chart of the CUT-based model and the DS method of the present invention under the MAPE evaluation index;

[0027] Figure 5(c) is a performance comparison chart of the CUT-based model and the DS method of the present invention under the PCC evaluation index;

[0028] Figure 5(d) is a performance comparison chart of the CUT-based model and the DS method of the present invention under the CS evaluation index;

[0029] Figure 6 This is a comparison diagram of the three-dimensional curves of the source domain, target domain, and generation domain spectra of the present invention;

[0030] Figure 7 This is a PCA projection diagram of the source domain, target domain, and generated sample of the present invention. Detailed Implementation

[0031] This invention, based on the existing image domain CUT model, combines the characteristics of one-dimensional near-infrared spectral data to apply a CUT-based transfer model to cross-device transfer tasks of near-infrared spectral data. This method can complete the transfer from the source domain to the target domain using only unpaired spectral samples, significantly reducing model deployment costs and implementation difficulty. It addresses the dependence of existing calibration transfer methods on standard samples and solves problems such as insufficient spectral distribution alignment accuracy, incomplete preservation of spectral information, and poor model adaptability across different devices. During the training of the transfer model, the original image domain PatchNCE mechanism is adapted to local (patch-wise) contrastive learning for one-dimensional spectral data. Furthermore, in this invention, a Patch is defined as a local segment of the spectral signal in the wavelength dimension, used to characterize the feature information of that local region. Not only is the source domain spectrum from the source device applied using Patch-based contrastive learning loss, but the target domain spectrum from the target device is also applied using Patch-based contrastive learning loss. This prevents the generator from making unnecessary and destructive modifications to the already target domain spectrum, effectively preserving the valid information of the source domain spectrum, and achieving high-quality alignment between the generated spectrum and the real target domain spectrum.

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings.

[0033] An implementation of a near-infrared spectral transfer method based on contrastive learning:

[0034] In recent years, the development of deep learning technology has provided new solutions for unsupervised cross-domain translation. Generative Adversarial Networks (GANs) have demonstrated good domain distribution alignment capabilities in unpaired image-to-image translation tasks, such as image processing. In these tasks, due to the lack of paired samples, cycle-consistency is often used to establish inter-domain correspondences, with Cycle-Consistent Generative Adversarial Network (CycleGAN) being one example. However, the bijective assumption upon which cycle-consistency is based may be too strict in practical applications and also incur high computational costs. In contrast, adopting a unidirectional generative strategy combined with a contrastive learning mechanism can effectively reduce computational overhead while maintaining consistency between spectral information and local structure. This characteristic provides a feasible and efficient solution for addressing the cross-device calibration translation problem in near-infrared spectroscopy (NIR). In some image domain transfer methods (such as the CUT model), a contrastive loss function, generally called PatchNCE loss, is used. This contrastive loss function typically compares different spatial locations within the same image. For each patch of the generated image, it brings positive sample pairs closer together and pushes negative sample pairs further apart, in order to learn an effective feature representation without manual annotation. This invention applies the CUT framework, originally used in image domain transfer tasks, to near-infrared spectroscopy cross-device transfer tasks to address the dependence of existing calibration transfer methods on standard samples. Moreover, it applies not only the source domain spectrum from the source device to the patch-based contrastive learning loss, but also the target domain spectrum from the target device, to help the transfer model better learn the feature distribution of the target domain.

[0035] This embodiment uses tobacco leaves as the object of detection, and takes the spectral data collected by two different models of near-infrared spectrometers as an example to illustrate the implementation process of a near-infrared spectral transfer method based on contrastive learning according to the present invention. The process is illustrated below. Figure 1 As shown. Of course, the object being detected by this method can be other objects as well, but the principle of the specific implementation process remains the same.

[0036] Step 1: Obtain the source domain dataset and the target domain dataset.

[0037] Near-infrared spectral data (both one-dimensional infrared spectral data) of tobacco samples were collected from device A (source device) and device B (target device), forming the source domain dataset X and the target domain dataset Y. Each spectrum contains 1557 wavelength points. After uniform preprocessing, one wavelength point was removed from the end of each sample to obtain a 1556-dimensional feature vector. This ensures the consistency of input and output dimensions, effectively avoids the tensor shape mismatch problem caused by the convolution kernel size, and also preserves the integrity of key spectral regions.

[0038] Step 2: Build a migration model based on the CUT framework.

[0039] Based on the Contrastive Unpaired Translation (CUT) framework proposed in image domain translation tasks, a transfer model is built to address the cross-device transfer requirements of one-dimensional near-infrared spectral data. A schematic diagram of the CUT framework is shown below. Figure 2 As shown, Figure 2 G in enc G represents the coding layer. dec The code represents the decoding layer, where `target` represents the target domain spectrum, `source` represents the source domain spectrum, `D` represents the discriminator, and `Generative Adversarial Loss` represents the generative adversarial loss. Compared to unpaired models such as CycleGAN, this invention abandons the bidirectional mapping under cycle consistency constraints and adopts a unidirectional mapping strategy to achieve spectral transformation from the source domain to the target domain. This design not only simplifies the training process and reduces memory usage and computational overhead, but also significantly shortens training time and improves the model's deployment efficiency in real-world industrial scenarios.

[0040] The transfer model consists of a generator (G) and a discriminator (D), and can be specifically referred to as a CUT-based model. The generator maps the source domain spectrum to a pseudo-spectrum that closely approximates the target domain, and the discriminator is used to distinguish between the real spectrum from the target domain and the pseudo-spectrum output by the generator.

[0041] Generator structure as follows Figure 3As shown, it consists of three parts: an encoding layer, a transformation layer, and a decoding layer. The encoding layer extracts multi-scale features of the input spectrum through multiple layers of one-dimensional convolution, instance normalization, and activation functions. Specifically, it includes three sequentially connected encoding units (the units are divided for ease of description). The first encoding unit includes a sequentially connected one-dimensional convolution (Conv1D), instance normalization (IN), and activation function (specifically, the ReLU activation function). The latter two encoding units each include a sequentially connected one-dimensional convolution (Conv1D), instance normalization (IN), activation function (specifically, the ReLU activation function), and downsampling (specifically, a one-dimensional convolution (Conv1D)). The feature transformation layer (Transformation) consists of multiple residual blocks (specifically 9 in this embodiment) to achieve efficient transfer of encoded features to decoded features. Each residual block contains sequentially connected one-dimensional convolution Conv1D, instance normalization IN, activation function (specifically ReLU activation function), one-dimensional convolution Conv1D, and instance normalization IN, with the input of the first one-dimensional convolution Conv1D and the output of the last instance normalization IN being skipped. Figure 3 The decoding layer uses one-dimensional convolution, instance normalization, and activation functions to gradually reconstruct the spectrum, and finally outputs the spectrum through a layer of convolution and activation functions. Specifically, it includes two decoding units and one output unit (the units are divided for ease of description). Each decoding unit includes upsampling, one-dimensional convolution (Conv1D), instance normalization (IN), and activation function (specifically, ReLU activation function) connected in sequence. The output unit includes one-dimensional convolution and activation function (specifically, Tanh activation function) connected in sequence.

[0042] The discriminator adopts the PatchGAN architecture, consisting of multiple layers of one-dimensional convolutions, instance normalization, and activation functions. It can determine the authenticity of the input data, thereby effectively constraining the output quality of the generator. For ease of description, this embodiment divides the discriminator structure into several discriminator units and one output unit, and its framework is as follows: Figure 4 As shown, it specifically includes four discrimination units and one output unit connected in sequence. Figure 4The five dashed boxes, from top to bottom, represent four discriminant units, with the last dashed box representing the output unit. The first discriminant unit consists of a sequentially connected one-dimensional convolution (Conv1D), an activation function (specifically, a Leaky ReLU activation function), and a downsampled function (specifically, a one-dimensional convolution (Conv1D)). The two middle discriminant units each consist of a sequentially connected one-dimensional convolution (Conv1D), an instance normalization (IN), an activation function (specifically, a Leaky ReLU activation function), and a downsampled function (specifically, a one-dimensional convolution (Conv1D)). The last discriminant unit consists of a sequentially connected one-dimensional convolution (Conv1D), an instance normalization (IN), and an activation function (specifically, a Leaky ReLU activation function). The output unit consists of a one-dimensional convolution (Conv1D).

[0043] Step 3: Use the dataset obtained in Step 1 to train the transfer learning model built in Step 2.

[0044] During the training of the transfer model, the loss function plays a crucial guiding role in the cross-device transfer task of near-infrared spectral data. It guides the generator to gradually approximate the features and distribution of real near-infrared spectral data, thus promoting the generator's learning and optimization. The loss function in this implementation includes two main types: generative adversarial loss and patch-based contrastive learning loss.

[0045] 1) Generate adversarial losses.

[0046] In this embodiment, the generative adversarial loss employs a least squares generative adversarial network (LSGAN) to encourage the generator to learn the distribution of the spectral data in the target domain (instrument B). This generates realistic and high-quality samples. The generator G and discriminator D are trained simultaneously. The generator aims to minimize the difference between the distributions of real and generated data, making it indistinguishable to the discriminator. The discriminator aims to accurately distinguish between real and generated data. The least-squares generative adversarial loss function is defined as follows:

[0047] (1)

[0048] (2)

[0049] In the formula, a and b represent the label values ​​of the discriminator for fake samples and real samples, respectively, and c represents the label value that the generator wants the discriminator to believe is a fake sample as real. This represents the discriminator loss based on LSGAN. The generator loss is based on LSGAN, where x represents the source domain spectral data and y represents the target domain spectral data. Represents the distribution of real data The expected value of the sample y obtained by downsampling is taken. Represents the input distribution of the source domain The expected value of the sample x obtained by downsampling is taken.

[0050] 2) Local contrast loss (PatchNCE, a spectral domain-based contrastive learning loss)

[0051] To preserve critical information of the input spectrum across devices, this implementation employs Patch-based contrastive learning loss (PatchNCE). The proposed local contrastive loss function is geared towards spectral signal processing, with the core objective of maintaining consistency of spectral features across devices. This loss achieves feature alignment by maximizing the similarity between the query patch (Q) and the positive sample patch (P), and minimizing the similarity with the negative sample patch (N). Unlike patches in image translation, which originate from two-dimensional spatial locations, the patch in this invention is defined as a local segment of the spectral signal in the wavelength dimension, used to characterize the feature information of that local region. Therefore, positive sample patches come from spectral segments at the same wavelength position, while negative sample patches come from segments of the spectrum at other wavelength positions.

[0052] In the implementation process, the first step is to generate the spectrum. A query patch Q is randomly selected, and the corresponding positive sample P comes from the source domain spectrum. At the same location, negative samples N come from The different positions in the vector. Q, P, and N are each mapped to a K-dimensional feature vector. , , , , and Let represent the query vector, the input positive sample vector, and the input negative sample vector of the query patch, respectively. The distance between them is scaled by the temperature hyperparameter τ (in this embodiment, τ is set to 0.07). Calculate the cross-entropy loss. :

[0053] (3)

[0054] In the formula, Indicates the first indivual , express The number of.

[0055] In the specific implementation, the generator is located at each specified layer. Output feature map via projection network The local features output from the encoding layer are mapped and projected onto a unified contrastive learning space, thereby ensuring that features at different layers and locations have a unified dimension and enhancing the discriminative power of contrastive learning, resulting in a feature stack. .in, , This represents the number of spatial locations in this layer; positive sample features are represented as... (Right now No. The first layer extracted from the specified layer The input features of each patch), and the negative sample features are represented as follows: (Right now No. The actual data extracted from a specified layer for comparison. Exclude the first patch The negative sample features of other patches in a given patch (excluding positive sample locations), where This represents the number of channels in each layer. Similarly, the feature stack of the output data is represented as... MLP networks project input and output patches into a shared embedding space.

[0056] (4)

[0057] In the formula, Indicates data distribution by source domain Extracted samples The loss is taken as the mathematical expectation. Represents a generator. Indicates the source region spectrum A sample of input data.

[0058] This loss function calculates a contrastive loss for each layer and each spatial patch. The loss constraint is the cross-device spectral input and generated output, ensuring that corresponding patches between the input and output spectra remain aligned in the feature space. It prompts the model to bring corresponding patches of the input and output data closer together, while moving mismatched patches away. Unlike in the image domain, PatchNCE is used in the spectral generator, aiming to maintain consistency between the input and output spectra at the same wavelength while constraining negative sample differences across wavelengths, thereby improving the accuracy and stability of cross-device spectral transfer.

[0059] 3) Final objective function.

[0060] This implementation also applies PatchNCE loss to the target domain Y samples. This is to prevent the generator from making unnecessary modifications to samples already in the target domain. The specific formula is not shown here.

[0061] The total loss of the generator is:

[0062] (5)

[0063] The total loss of the discriminator is:

[0064] (6)

[0065] By designing the loss function described above, the spectral distributions of the source and target domains can be effectively aligned, enabling high-quality calibration transfer of spectra across devices.

[0066] Based on the loss function designed above, the specific model training process is as follows:

[0067] First, a batch of spectral samples, denoted as real A, is randomly selected from the source domain near-infrared spectral dataset. Simultaneously, a batch of spectral samples, denoted as real B, is randomly selected from the target domain near-infrared spectral dataset. There is no pairing between the two. The generator receives both real A and real B as input: real A is transformed by the generator to obtain the synthetic spectrum fake B, achieving spectral transfer from the source domain to the target domain; real B is transformed by the generator to obtain idt B, used for feature consistency constraints in identity preservation mode. In the adversarial training phase, the discriminator receives fake B and real B respectively and outputs the result of determining whether the spectrum is real or fake. Based on the discrimination result, the loss of the fake samples in the generated spectrum and the loss of the real samples in the real spectrum are calculated to obtain the total loss of the discriminator. The generator's optimization process involves two parts: first, calculating the adversarial loss based on the discriminator's judgment of fake B; and second, updating the discriminator parameters through backpropagation to improve its discrimination capability. The first step is to encourage the generator to output a spectrum that more closely approximates the distribution of the real target domain. The second step is to use a feature extraction network F to sample local spectral fragments (patches) at the same positions from the multi-layer feature maps of real A and fake B, and to calculate a local contrast loss based on maximizing mutual information together with the corresponding feature fragments of real B and idt B. and This local contrast loss ensures that the generated spectrum retains effective information from the input spectrum at the local feature level, while preserving its original features during the target domain sample mapping process. The total generator loss is obtained by combining the generative adversarial loss and the local contrast loss. The generator parameters are updated via gradient backpropagation. After multiple rounds of iterative training, the generator is able to achieve high-quality cross-device spectral calibration under unpaired conditions.

[0068] Step 4: Model Reasoning and Application.

[0069] The method acquires newly collected near-infrared spectra from the source device and inputs them into a trained transfer model (actually, into a generator). The trained transfer model then generates spectra that approximate the near-infrared spectra from the target device in both overall distribution and local structure—this is the spectral data transferred to the target device. This generated spectrum can be directly input into the target device's existing analysis model. This method achieves cross-device calibration transfer of the spectral domain without the need for standard samples, significantly reducing calibration costs and improving applicability in scenarios where standard samples are scarce or devices are frequently changed.

[0070] This completes the entire method. The following experiments will demonstrate the effectiveness of the present invention.

[0071] In this embodiment, all near-infrared spectral samples undergo uniform clipping along the wavelength dimension during the data preprocessing stage. Specifically, one wavelength point is removed from the end of each sample, ensuring that both input and output data maintain 1556 wavelength points. This processing method ensures kernel size matching during convolution operations, avoiding computational errors caused by inconsistent tensor shapes, while fully preserving key waveband regions containing chemical information in the near-infrared spectrum, thereby guaranteeing the effectiveness and stability of model training.

[0072] In model training, the Adam optimization algorithm was used for parameter updates, with the decay rate of the first moment set to 0.5 and the decay rate of the second moment set to 0.999. This effectively helped the model achieve stable convergence quickly and obtain reasonable performance during training. In the initial training phase, the learning rates for both the generator and discriminator were set to 0.0002. A larger learning rate helps the model quickly explore the parameter space and find regions with lower loss values. Therefore, the learning rate was kept constant for the first 400 training epochs. In the remaining 400 training epochs, the learning rate was linearly decayed to help fine-tune the model parameters and gradually converge to the minimum of the loss function, thus ensuring more stable and accurate training. In the experiments, the number of patches was set to 128 to achieve a balance between computational efficiency and feature learning quality. The total number of training epochs was 800, and this training scale verified that the model could converge sufficiently while avoiding unnecessary computational costs.

[0073] To verify the cross-device calibration performance of this invention under standard-free conditions, the proposed CUT-based model was compared with two typical standard-free baseline methods: MSC (multivariate scattering correction) and FIR (finite impulse response filtering). On 188 paired test samples, four evaluation metrics were calculated: Pearson correlation coefficient (PCC), cosine similarity (CS), mean absolute percentage error (MAPE), and root mean square error (RMSE). The results are shown in Table 1. As can be seen from Table 1, compared with FIR, this invention outperforms FIR in 89.36% of the samples for PCC and RMSE, respectively, and in 85.64% and 85.11% for MAPE and CS, respectively. Compared with MSC, this invention achieves advantages of 86.70% and 82.98% for RMSE and MAPE, respectively, and 65.43% for CS, while its performance in PCC is close to that of the MSC method. These results indicate that this invention can achieve good spectral transfer performance under standard-free conditions.

[0074] Table 1

[0075]

[0076] Furthermore, the present invention is compared with the supervised calibration method DS (Direct Standardization) that relies on standard samples, and the results are as follows: Figures 5(a)~5(d) As shown, the RMSE and MAPE error metrics indicate that the proposed method performs slightly better than the DS method on most paired samples, suggesting a certain gap in accuracy. However, the PCC and CS similarity metrics are very close, both approaching 1, demonstrating extremely high consistency in spectral morphology preservation. This shows that even without relying on standard samples, the proposed method can still achieve spectral consistency calibration results close to those of supervised methods.

[0077] While the DS method can achieve high-precision inter-domain alignment using standard samples, it often requires additional data acquisition, making it impractical in real-world applications. In contrast, the method of this invention can be trained using only standard-free data from the source and target domains, providing a flexible solution for scenarios where standard samples are scarce or unavailable. In summary, although the overall performance of the method of this invention is slightly inferior to the DS method, it has significant advantages in data acquisition cost, applicable scenarios, and deployment flexibility. It can achieve effective spectral transfer under standard-free conditions, demonstrating good practicality and promising prospects for widespread application.

[0078] To more intuitively demonstrate the transfer effect, spectral curves were plotted for comparison of the five best-performing samples selected from the test set (see...). Figure 6 ), Figure 6 In this context, source X represents the source domain X, target Y represents the target domain Y, and X-generated Y represents the spectrum generated by X. It can be observed that the generated spectrum and the target spectrum are highly consistent across the entire spectral range, indicating that the method of this invention can effectively align spectral data under cross-device conditions.

[0079] To further analyze the distribution relationship between the source domain, the generator domain, and the target domain, this invention employs principal component analysis (PCA) to reduce the high-dimensional spectral data to a two-dimensional space for visualization (see...). Figure 7 This dimensionality reduction operation helps to intuitively assess how closely the generated spectrum approximates the target domain distribution in two-dimensional space, and the model's ability to reduce inter-domain differences under standard-free conditions. The analysis results show that PC1 and PC2 explain 78.18% and 17.84% of the total variance, respectively, with their combined explained variance reaching 96.02%, preserving most of the information from the original high-dimensional spectrum. Figure 7 As can be seen, the source sample (source X, blue), the generated sample (X Generated Y, red), and the real target sample (target Y, green) exhibit a clear inter-domain distribution relationship in two-dimensional space. Each blue point is connected to its corresponding green point by a dashed line, clearly demonstrating the inter-domain offset and model alignment effect. The generated sample (red) is distributed between the source and target domains, and is closer to the target domain, indicating that the method of this invention can simultaneously capture the spectral features of two domains and effectively reduce the inter-domain gap.

[0080] This phenomenon is highly consistent with the quantitative results (see Table 1). The visualization and quantitative results together verify that the method of this invention can effectively reduce inter-domain differences and achieve robust spectral consistency transfer without paired standard samples. Furthermore, to further verify the robustness and generalization ability of the model, a reverse calibration transfer experiment was conducted. The results show that the method of this invention also achieves excellent performance in reverse calibration transfer, further demonstrating the stability and universality of this invention under different transfer directions.

[0081] An implementation of a near-infrared spectral transfer system based on contrastive learning:

[0082] An embodiment of a near-infrared spectroscopy cross-device transfer system based on contrastive learning according to the present invention specifically includes a memory, a processor, an internal bus, and a computer program stored in the memory. The processor and the memory communicate and interact with each other via the internal bus. The processor executes the computer program to implement the steps of the method described in an embodiment of the near-infrared spectroscopy cross-device transfer method based on contrastive learning according to the present invention. The processor can be a microprocessor (MCU), a programmable logic device (FPGA), or other processing devices; the memory can be various types of memory that store information using electrical energy, such as RAM, ROM, etc., or other types of memory.

[0083] In summary, the present invention has the following characteristics:

[0084] 1) This invention proposes a standard-free near-infrared spectral calibration transfer method based on a CUT structure, combining adversarial generation with local feature contrastive learning for cross-device NIR spectroscopy tasks. This method does not rely on standard samples, thus maintaining consistency in waveform morphology and numerical accuracy while reducing inter-domain distribution differences, avoiding the high-cost standard sample acquisition required by existing supervised methods.

[0085] 2) This invention applies a local feature contrastive learning mechanism to cross-device near-infrared spectral calibration and transfer tasks. By establishing a positive and negative sample relationship of local features between the generated spectrum and the source spectrum, it effectively preserves the valid information of the source spectrum. That is, by maximizing the mutual information of corresponding positions of the input and output spectra through the local contrastive learning mechanism, it retains key information while aligning the spectral distributions of the source and target domains. This feature alignment strategy significantly improves the stability of the transferred spectrum in terms of morphology preservation and significantly outperforms non-standard methods such as MSC and FIR in terms of PCC, CS, and other metrics.

[0086] 3) In the CUT-based model framework, the generator continuously reduces the difference in distribution between the source and target domains through discriminator feedback, thus achieving calibration transfer. This mechanism effectively reduces error indicators such as RMSE and MAPE, ensuring that the generated spectrum not only closely approximates the target domain in morphology but also maintains high numerical accuracy.

[0087] 4) In the absence of standard samples, the Contrastive Unpaired Translation (CUT) model framework, originally used for image domain transfer tasks, is adapted and adjusted in terms of structure and parameters for the generator, discriminator, and contrast learning modules, taking into account the characteristics of one-dimensional near-infrared spectral data. Specifically, this involves targeted one-dimensional transformation, including adjustments to convolutional structures, sampling methods, and feature alignment mechanisms. This ensures the network effectively extracts and expresses spectral features, enabling it to effectively model spectral differences between different devices. This achieves effective adaptation of the model from two-dimensional image processing tasks to one-dimensional spectral data processing tasks. The model possesses powerful nonlinear feature modeling capabilities, capturing complex and nonlinear spectral differences between different instruments. Being entirely data-driven, the model requires no predefined transformation functions or reference spectra, thus exhibiting high applicability.

[0088] 5) Compared to the DS method, which relies on standard samples, this invention achieves similar PCC and CS metrics to the DS method without requiring additional standard samples. This significantly lowers the deployment threshold and implementation difficulty, reduces spectral domain deviations caused by hardware differences and acquisition conditions, and enables cross-device calibration transfer. The PCA visualization results provide intuitive verification that the spectra generated by this invention are more closely distributed in the two-dimensional feature space than the target domain samples, effectively reducing inter-domain differences and achieving stable spectral consistency transfer.

[0089] 6) This invention retains the unidirectional mapping advantage of CUT, removes cycle consistency constraints, significantly reduces training time and memory usage, and facilitates rapid deployment in industrial environments. This invention is suitable for rapid adaptation and reuse of near-infrared spectral models across different devices, offering greater flexibility, adaptability, and industrial application value.

Claims

1. A near-infrared spectroscopy cross-device transfer method based on contrast learning, characterized in that, The method comprises: acquiring near-infrared spectra from a source device and inputting the near-infrared spectra into a trained transfer model to generate spectra close to near-infrared spectra from a target device in overall distribution and local structure by using the trained transfer model; the transfer model is built based on a CUT framework, and a Patch-based contrastive learning loss used in a generator process of training the transfer model comprises applying a Patch-based contrastive learning loss to source domain spectra from the source device and applying a Patch-based contrastive learning loss to target domain spectra from the target device, and the Patch is defined as a local segment of spectral signals in a wavelength dimension, and is used to represent feature information of the local region.

2. The method of claim 1, wherein, An expression of the Patch-based contrastive learning loss applied to the source domain spectra from the source device is: ； ； In the formula, represents the source domain spectrum Apply Patch-based contrastive learning loss, represents the generator, represents a projection network for mapping local features output by the encoding layer to a contrastive learning space, represents the source domain spectrum Sampled input sample; represents the sample extracted according to the source domain data distribution Loss of sample Take mathematical expectation; represents the first specified layer of the generator; represents the first patch; represents the output feature of the first patch extracted by the first specified layer, represents the negative sample feature of the other patches excluding the first patch in the first patch actually used for contrast extracted by the first specified layer, represents the input feature of the first patch extracted by the first specified layer; , and respectively represent the query vector of the query patch, the input positive sample vector and the input negative sample vector, represents the first specified layer ; represents the cross-entropy loss; represents the temperature hyperparameter.​ 3. The method of claim 1, wherein, a generative adversarial loss used in the process of training the transfer model is a least squares generative adversarial loss.

4. The method of claim 1-3, wherein, The generator in the transfer model comprises an encoding layer, a feature transformation layer and a decoding layer; the encoding layer is used to extract multi-scale features of input spectra; the feature transformation layer comprises a plurality of residual blocks, and is used to realize transmission of encoded features to decoded features; and the decoding layer is used to output final spectra.

5. The method of claim 4, wherein the method is a contrastive learning based near-infrared spectroscopy cross-device transfer method. The encoding layer comprises three encoding units connected in sequence, the first encoding unit comprises one-dimensional convolution, instance normalization and an activation function connected in sequence, and the last two encoding units each comprise one-dimensional convolution, instance normalization, an activation function and down-sampling connected in sequence.

6. The method of claim 4, wherein the method is based on contrastive learning. The residual block comprises one-dimensional convolution, instance normalization, an activation function, one-dimensional convolution and instance normalization connected in sequence, and the input of the first one-dimensional convolution and the output of the last instance normalization are connected in a skip connection.

7. The method of claim 4, wherein the method is based on contrastive learning. The decoding layer comprises two decoding units and an output unit connected in sequence, each decoding unit comprises up-sampling, one-dimensional convolution, instance normalization and an activation function connected in sequence, and the output unit comprises one-dimensional convolution and an activation function connected in sequence.

8. The method of claim 1-3, wherein, The discriminator in the transfer model comprises four discrimination units and an output unit connected in sequence, the first discrimination unit comprises one-dimensional convolution, an activation function and down-sampling connected in sequence, the middle two discrimination units each comprise one-dimensional convolution, instance normalization, an activation function and down-sampling connected in sequence, the last discrimination unit comprises one-dimensional convolution, instance normalization and an activation function connected in sequence, and the output unit comprises one-dimensional convolution.

9. The method of claim 1-3, wherein, The acquired near-infrared spectra from the source device are one-dimensional spectra.

10. A near-infrared spectroscopy cross-device transfer system based on contrast learning, comprising a processor, characterized in that, The processor is configured to execute a computer program to implement the steps of the method of any one of claims 1-9.

Images