Rapid nondestructive testing system based on fusion of near infrared spectrum and Raman spectrum

Abstract

The invention is applicable to the field of spectrum detection and analysis, and particularly provides a rapid nondestructive detection system based on fusion of near infrared spectrum and Raman spectrum, which comprises a detection device and an intelligent terminal, the detection device comprises an upper-layer container, and a replaceable sample storage container is rotationally arranged in the upper-layer container; two openings are formed in the middle-layer container, and glass gates are arranged on the two openings; one opening corresponds to a data acquisition port of the near-infrared spectrometer, and the other opening corresponds to a data acquisition port of the Raman spectrometer; an edge end is arranged in the lower-layer container; a data processing unit is arranged in the edge end of the lower-layer container, and the data processing unit comprises a data acquisition control module, a data modeling module, a model self-adaption module, a reasoning prediction module and an upper-layer sample rotation control module. By combining the advantages of two spectrum technologies, the defects of a single technology in sample detection are overcome, and the detection accuracy and efficiency are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of spectral detection and analysis, and particularly relates to a rapid non-destructive detection system based on the fusion of near-infrared spectroscopy and Raman spectroscopy. Background Art

[0002] In the fields of agriculture, food safety and bioengineering, the safety and authenticity detection of agricultural product detection is of great significance.

[0003] Traditional detection methods mainly rely on biochemical analysis techniques such as polymerase chain reaction (PCR) and enzyme-linked immunosorbent assay (ELISA). Although these methods have high accuracy, they have limitations such as long detection cycle, high cost, complex operation, and high requirements for laboratory environment, and it is difficult to meet the needs of large-scale rapid detection. Therefore, there is an urgent need for an efficient, convenient and non-destructive detection technology to improve the screening efficiency of genetically modified agricultural products.

[0004] Spectral detection technology has become an important research direction in the field of agricultural product detection in recent years due to its non-destructive, fast and real-time analysis characteristics. Near-infrared spectroscopy (NIR) and Raman spectroscopy (Raman) technologies each have their own advantages in material composition analysis. NIR mainly reflects the vibration absorption information of organic components in the sample and is suitable for rapid detection of components such as moisture, protein, and oil in the sample; Raman is based on molecular vibration scattering and is sensitive to the molecular structure and chemical bond information of the sample, and can provide detailed molecular characteristics.

[0005] At present, there are certain limitations in single spectral technology, and the existing spectral detection systems usually have the following problems:

[0006] First, low detection efficiency: Traditional multi-spectral systems mostly adopt single-point sampling and it is difficult to achieve rapid batch detection;

[0007] Second, limited spectral feature extraction ability: Existing methods rely on traditional machine learning models and have weak learning ability for complex spectral features, resulting in limited detection accuracy;

[0008] Third, poor model generalization ability: The spectral features of different varieties and different growth environments are different, resulting in low model migration ability and it is difficult to be applicable to samples of different batches;

[0009] Fourth, more manual intervention: The spectral data acquisition and analysis process still rely on manual adjustment of light sources, sample positions and data processing parameters, affecting the automation degree and operability of detection. Summary of the Invention

[0010] The purpose of the embodiment of the present invention is to provide a rapid non-destructive detection system based on the fusion of near-infrared spectroscopy and Raman spectroscopy, aiming to solve the technical problems existing in the existing spectral detection systems, such as low detection efficiency, limited spectral feature extraction ability, poor model generalization ability, and more manual intervention.

[0011] To achieve the above object, the present invention provides the following technical solutions.

[0012] An embodiment of the present invention provides a rapid non-destructive detection system based on the fusion of near-infrared spectroscopy and Raman spectroscopy, including a detection device and an intelligent terminal; the intelligent terminal is a mobile phone terminal;

[0013] Among them, the detection device includes:

[0014] An upper container, in which a replaceable sample storage container is rotatably arranged; the top layer of the replaceable sample storage container has an openable screw cap, and the replaceable sample storage container is a partitioned structure, which is divided into eight regions by four diagonals. Each diagonal is divided into two radius lines at the center, and a sample slot is fixed in the middle of each radius line for placing the sample to be detected; the bottom layer of the replaceable sample storage container is a light-transmitting structure;

[0015] A middle container, on which two openings are provided, and glass gates are provided on both openings; the two openings are located on the same diameter of the middle container; one opening corresponds to the data acquisition port of the near-infrared spectrometer, and the other opening corresponds to the data acquisition port of the Raman spectrometer;

[0016] A lower container, in which an edge end is provided inside, and the edge end is connected to the near-infrared spectrometer and the Raman spectrometer respectively through two data lines; data interaction and command transmission are carried out between the edge end and the intelligent terminal. There is a data processing unit in the edge end of the lower container, and the data processing unit includes a data acquisition control module, a data modeling module, a model adaptation module, an inference prediction module, and an upper sample rotation control module. The upper sample rotation control module is used to control the replaceable sample storage container to rotate inside the upper container in increments of 45°.

[0017] Further, the openings provided on the middle container are located directly below the movement path of the sample slots when the replaceable sample storage container rotates, so that the acquisition ports of the near-infrared spectrometer and the Raman spectrometer can be aligned with the samples on the sample slots.

[0018] Further, both the near-infrared spectrometer and the Raman spectrometer are arranged inside the middle container for collecting spectral data of the samples in the upper container;

[0019] Both the near-infrared spectrometer and the Raman spectrometer are externally connected with data lines, and the data lines are connected to the edge end in the lower container;

[0020] The near-infrared spectrometer covers the wavelength range of 900nm - 1700nm;

[0021] The Raman spectrometer covers the wavelength range of 271nm - 2453nm, and the laser excitation wavelength of the Raman spectrometer is 785nm.

[0022] Furthermore, the data modeling module is used to process the near-infrared and Raman spectral data, including preprocessing, feature extraction, feature fusion, and optimizing the feature representation using a contrast learning algorithm; where:

[0023] In the preprocessing step, it includes Raman spectral preprocessing based on near-infrared spectra and near-infrared spectral preprocessing based on Raman spectra;

[0024] In the feature extraction step, a dual-modal encoder is used for feature extraction, including: extracting near-infrared spectral features using a near-infrared encoder , extracting Raman spectral features using a Raman encoder ; enhancing the spectral features through a feature enhancement module; optimizing the dual-modal information interaction by combining a global attention module and a cross-attention module;

[0025] In the feature fusion step, processing methods such as feature concatenation, weighted fusion, and feature selection for dimensionality reduction are adopted to optimize the feature representation; where the feature concatenation processing is expressed as: , where represents the spectral features after concatenation, is the spectral features after near-infrared spectral fusion, is the spectral features after Raman spectral fusion;

[0026] In the step of optimizing the feature representation using a contrast learning algorithm, data augmentation is first performed, and a positive and negative sample construction strategy based on contrast learning is adopted to expand the data distribution for different spectral modes; then the contrast loss is calculated, where the total loss of the model is expressed as:

[0027] ;

[0028] where L is the total loss value of the model, N is the total number of samples, is the feature vector of the i-th sample, is the feature vector of the sample positively correlated with the i-th sample; is the feature vector of the j-th sample, is the temperature parameter, used to control the smoothness of the contrast loss; represents the The feature vectors of the samples; in the model training stage, the gradient of the loss function model parameters is calculated using the backpropagation algorithm, and the model parameters are updated through the gradient; in the model optimization stage, the model weights are adjusted based on the loss function minimization strategy.

[0029] Further, the steps of Raman spectrum preprocessing based on near-infrared spectrum include background correction, noise removal, and feature alignment;

[0030] The steps of near-infrared spectrum preprocessing based on Raman spectrum include baseline correction, feature normalization, and feature alignment.

[0031] Further, in the step of optimizing the bimodal information interaction by combining the global attention module and the cross-attention module, the global attention is expressed as: ; ;

[0032] where Q, K, and V are the query, key, and value of the input features, d is the feature dimension, is the attention weight matrix, is the weighted global feature;

[0033] The cross-attention is expressed as:

[0034] ; ; ; ;

[0035] where, , , respectively represent the query, key, and value of the near-infrared spectrum, , , respectively represent the query, key, and value of the Raman spectrum, represents the attention weight matrix of the near-infrared spectrum, is the attention weight matrix of the Raman spectrum, is the spectral feature after the fusion of the near-infrared spectrum, is the spectral feature after the fusion of the Raman spectrum, and d represents the feature dimension.

[0036] Further, in the model adaptive module, the similarity between the feature vectors of the new data and the feature vectors stored in the model is calculated, and data comparison is performed based on the similarity;

[0037] According to the comparison result, the parameters of the model are dynamically adjusted; among them, the learning rate is adjusted based on the adaptive learning rate adjustment algorithm according to the gradient of the objective function, which is expressed as:

[0038] ;

[0039] Among them, is the current learning rate, is the decay factor, is the minimum value of the learning rate, is the sum of the squares of the gradients, is a constant;

[0040] Calculate the accuracy performance metric of the model on the validation set to verify the performance of the updated model, expressed as:

[0041] ;

[0042] Among them, represents the number of true positive examples, represents the number of true negative examples, represents the total number of samples;

[0043] When the model verification result is positive, save the updated model parameters.

[0044] Furthermore, in the inference and prediction module, use the pre-trained model to predict the real-time spectral data, including:

[0045] Obtain the real-time spectral data file and preprocess the data;

[0046] Use a convolutional neural network for model inference, expressed as: where, and distributions are the weights and biases of the model, is the normalized input data;

[0047] Evaluate the prediction results through accuracy and F1 score.

[0048] Compared with the prior art, the beneficial effects of the fast non-destructive detection system based on the fusion of near-infrared spectroscopy and Raman spectroscopy of the present invention are:

[0049] First, in the upper container of the present invention, the rotatable replaceable sample storage container is divided into eight regions by four diagonals. The replaceable sample storage container can rotate the sample in 45° increments to ensure the detection of multiple samples in sequence, significantly improving the detection efficiency and the degree of automation of sample processing;

[0050] Second, in the middle container of the present invention, both the near-infrared spectrometer and the Raman spectrometer are fixed inside the middle container for collecting the spectral data of the sample; the collected data is exported through a connecting wire; a glass shutter is provided at the opening for controlling the passage of the light beam at the end of shooting or measurement, which can ensure the unobstructed passage of the light beam during data collection, and can block the light during non-collection periods to avoid interference;

[0051] Thirdly, in the lower container of the present invention, there is an edge terminal including a built-in data processing module. The data processing module includes a data acquisition control module, a data modeling module, a model adaptation module, an inference prediction module, and an upper sample rotation control module, where: The data acquisition module is used to initialize the spectrometer and communication interface, configure the acquisition parameters, start data acquisition, and temporarily store the acquired spectral data in the storage unit; The data modeling module is used to preprocess the acquired near-infrared and Raman spectral data, including basis preprocessing, feature extraction, feature fusion, and optimizing the feature representation using the contrastive learning algorithm; The model adaptation module is used to dynamically adjust the model parameters using the newly acquired data, update the model through the online learning algorithm to adapt to the change of data distribution, and verify the performance of the updated model; The inference prediction module is used to perform inference prediction on the newly acquired data using the optimized model, output the prediction result and temporarily store it in the local cache; The above modules work together to ensure the smooth and efficient operation of the entire process from data acquisition to result output.

[0052] In summary, the rapid non-destructive detection system of the present invention combines the advantages of two spectral technologies, overcomes the deficiencies of a single technology in sample detection, and improves the accuracy and efficiency of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0054] Figure 1 It is the overall view of the detection device in the rapid non-destructive detection system based on the fusion of near-infrared spectroscopy and Raman spectroscopy of the present invention;

[0055] Figure 2 It is the exploded view of the detection device provided by the present invention;

[0056] Figure 3 It is the structural schematic diagram of the upper container in the detection device provided by the present invention;

[0057] Figure 4 It is the structural schematic diagram of the middle container in the detection device provided by the present invention;

[0058] Figure 5 It is the structural schematic diagram of the lower container in the detection device provided by the present invention;

[0059] Figure 6 It is the overall flowchart of the edge terminal module of the rapid non-destructive detection system provided by the present invention;

[0060] Figure 7Flow chart of near-infrared and Raman depth contrast learning modeling in the rapid non-destructive detection system provided by the present invention.

[0061] The reference signs are as follows:

[0062] 1. Upper container; 101. Replaceable sample storage container; 1011. Sample slot; 102. Screw cap;

[0063] 2. Middle container; 201. Near-infrared spectrometer; 202. Raman spectrometer;

[0064] 3. Lower container; 301. Edge end; 302. Liquid crystal display;

[0065] 4. Mobile phone terminal. Specific implementation mode

[0066] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0067] The following describes the specific implementation of the present invention in detail with reference to specific embodiments.

[0068] As Figures 1-4 shown, in an embodiment of the present invention, a rapid non-destructive detection system based on the fusion of near-infrared spectroscopy and Raman spectroscopy is provided. The rapid non-destructive detection system includes a detection device and an intelligent terminal;

[0069] Preferably, the intelligent terminal is the mobile phone terminal 4. The intelligent terminal can also be an electronic device such as a computer, which is not specifically limited;

[0070] Specifically, as Figures 1-3 shown, the detection device provided by the present invention includes an upper container 1. Inside the upper container 1, there is a replaceable sample storage container 101 for placing samples to be detected. The top layer of the replaceable sample storage container 101 has an openable screw cap 102. The openable manner can be achieved through a threaded connection method or a snap connection method; in addition, the samples to be detected in the embodiments of the present disclosure can be soybean seeds, corn seeds or seed powders, etc.;

[0071] Preferably, in the embodiment of the present invention, the replaceable sample storage container 101 is a partition structure. The partition structure is divided into eight regions by four diagonals. Each diagonal is divided into two radius lines at the center. Therefore, there are a total of eight radius lines for the four diagonals. A sample slot 1011 is fixed in the middle of each radius line. Each sample slot 1011 is used to place a sample, such as soybean and corn seed samples or sample powders, to ensure the uniform distribution of the samples in the container and improve the consistency of measurement.

[0072] Furthermore, the replaceable sample storage container 101 is rotatably arranged inside the upper-layer container 1, and the screw cap 102 is a black cap that can block external light, ensuring that the inside of the replaceable sample storage container 101 is in a dark environment, reducing the interference of ambient light, and thus ensuring the accuracy of spectral measurement; the replaceable sample storage container 101 for storing soybean and corn seeds or seed powder samples supports the interchange of powdery and seed-like samples, increasing the applicability and flexibility of the container.

[0073] Even further, the bottom layer of the replaceable sample storage container 101 is a light-transmissive structure, specifically made of a transparent material such as transparent glass, which allows the light beams of the near-infrared spectrometer 201 and the Raman spectrometer 202 to pass through smoothly, ensuring the accuracy of spectral data collection; while the top layer of the replaceable sample storage container 101 is equipped with a screw cap 102. After opening the screw cap 102, it is convenient to put in the sample, simplifying the sample loading process while maintaining the sealing of the container.

[0074] In addition, the bottom of the replaceable sample storage container 101 is equipped with a rotating device for rotating the replaceable sample storage container 101 in a certain time sequence. Those skilled in the art can use an electric motor with servo function to drive the rotation of the replaceable sample storage container 101 when configuring the rotating device, and no specific limitation is made here; in the embodiment of the present disclosure, the rotating device is used to rotate the replaceable sample storage container 101 in increments of 45°, facilitating the detection of samples in eight regions in sequence, and significantly improving the detection efficiency and the automation degree of sample processing.

[0075] Such as Figure 1 、 Figure 2 and Figure 4 As shown, the detection device provided by the embodiment of the present invention further includes a middle-layer container 2 with a circular design. The circular structure of the middle-layer container 2 matches the circular structures of the upper-layer container 1 and the lower-layer container 3, facilitating the connection between the top of the middle-layer container 2 and the upper-layer container 1, and the connection between the bottom of the lower-layer container 3 and the middle-layer container 2; preferably, the connection methods between the middle-layer container 2 and the upper-layer container 1 and the lower-layer container 3 can all adopt threaded connection structures.

[0076] Furthermore, two openings are provided on the middle-layer container 2, and the two openings are located on the same diameter of the middle-layer container 2; one of the openings corresponds to the data collection port of the near-infrared spectrometer 201, and the other opening corresponds to the data collection port of the Raman spectrometer 202;

[0077] Preferably, the opening provided on the middle layer container 2 is located directly below the movement path of the sample tank 1011 when the replaceable sample storage container 101 rotates, so that the collection ports of the near-infrared spectrometer 201 and the Raman spectrometer 202 can be aligned with the samples on the sample tank 1011.

[0078] Preferably, in the embodiments of the present invention, the near-infrared spectrometer 201 covers the wavelength band of 900nm - 1700nm; the Raman spectrometer 202 covers the wavelength band of 271nm - 2453nm, and the laser excitation wavelength of the Raman spectrometer 202 is 785nm.

[0079] Specifically, both the near-infrared spectrometer 201 and the Raman spectrometer 202 are provided inside the middle layer container 2 for collecting spectral data of the samples in the upper layer container 1.

[0080] Both the near-infrared spectrometer 201 and the Raman spectrometer 202 are externally connected with data lines, and the data lines are connected to the edge end 301 in the lower layer container 3 for realizing spectral data transmission.

[0081] Preferably, in the embodiments of the present invention, glass gates are provided on both openings, and the glass gates are used to control the passage of light beams during shooting or at the end; the glass gates can be controlled to open or close through buttons to ensure that the light beams can pass through unobstructed during data collection, while the light can be blocked during non-collection periods to avoid interference.

[0082] In an alternative implementation, the glass gate can adopt an opening and closing mechanism based on the iris principle in the prior art to realize the opening or closing of the glass gate; or a movable light blocking plate (such as controlling the telescopic movement of a micro telescopic rod to move the light blocking plate) can be used to control the passage of light, and the specific structure is not limited; the glass gate in the embodiments of the present disclosure is only used to block the light beam, and those skilled in the art can make reasonable settings according to needs.

[0083] Furthermore, in the embodiments of the present disclosure, by setting the near-infrared spectrometer 201 and the Raman spectrometer 202 to work in coordination with the rotating device of the replaceable sample storage container 101 of the upper layer container 1, when the sample can rotate in increments of 45°, the near-infrared spectrometer 201 and the Raman spectrometer 202 can sequentially detect multiple samples, thereby significantly improving the detection efficiency and the degree of automation of sample processing.

[0084] As Figure 1 、 Figure 2 and Figure 5 shown, the detection device provided by the present invention further includes a lower layer container 3. An edge end 301 is provided inside the lower layer container 3, and the edge end 301 is connected to the near-infrared spectrometer 201 and the Raman spectrometer 202 respectively through two data lines to realize the transmission of spectral data.

[0085] Optionally, a liquid crystal display 302 for real-time display is also provided on the lower container 3, which is used to synchronously display data during the acquisition process, enabling the operator to monitor the acquisition status and results in real time and make timely adjustments and optimizations.

[0086] Preferably, data interaction and instruction transmission are carried out between the edge terminal 301 and the intelligent terminal, supporting multiple communication methods such as Wi-Fi and Bluetooth to ensure the stability and real-time nature of data transmission.

[0087] Please refer to Figure 5 and Figure 6 , in the edge terminal 301 of the lower container 3, there is a data processing unit, which includes a data acquisition control module, a data modeling module, a model adaptation module, an inference prediction module, and an upper sample rotation control module. The upper sample rotation control module is used to control the replaceable sample storage container 101 to rotate in increments of 45°, realizing the rotation of the sample. These modules work together to ensure the smoothness and efficiency of the entire process from data acquisition to result output;

[0088] In the data acquisition control module of the present invention, first, the near-infrared spectrometer 201 and the Raman spectrometer 202 and their communication interfaces are initialized, and acquisition parameters such as time and resolution are configured, and the data acquisition process is started. The acquired spectral data is then temporarily stored in the data storage unit for subsequent processing and analysis;

[0089] In one implementation, the data modeling module is responsible for preprocessing the near-infrared and Raman spectral data, including preprocessing, feature extraction, feature fusion, and optimizing the feature representation using a contrastive learning algorithm, thereby laying a foundation for establishing an accurate data model;

[0090] In one implementation, the model adaptation module uses newly acquired data to dynamically adjust the model parameters, updates the model through an online learning algorithm to adapt to changes in the data distribution, and verifies the performance of the updated model to ensure that the prediction ability of the model continuously improves with the addition of new data;

[0091] In one implementation, the inference prediction module uses the optimized model to perform inference prediction on newly acquired data, outputs the prediction results and temporarily stores them in the local cache to provide support for the final decision;

[0092] In one implementation, the rotation control module is responsible for managing the acquisition order and status of the samples, controlling the rotation of the sample stage, and prompting the user to replace the sample after a set of samples is acquired, thereby ensuring the orderly progress of the acquisition process;

[0093] In addition, the edge device 301 is designed with a data storage unit for temporarily storing the collected spectral data and processed results, ensuring data security and accessibility; the data storage unit is used for data interaction and instruction transmission with the mobile phone 4, supporting multiple communication methods such as Wi-Fi and Bluetooth to ensure the stability and real-time nature of data transmission; a liquid crystal display is also provided on the lower layer container 3 for synchronous display during the acquisition process, enabling the operator to monitor the acquisition status and results in real time and make timely adjustments and optimizations.

[0094] Specifically, as Figure 6 and Figure 7 shown, in the data modeling module, a modeling process of near-infrared and Raman depth contrast learning is adopted, including the following steps:

[0095] Data input: The collected near-infrared and Raman raw spectral data;

[0096] Data preprocessing: The process of data preprocessing includes: Raman spectral preprocessing based on near-infrared spectra, performing background correction, noise removal, and feature alignment to reduce the systematic error between different spectral data sources;

[0097] Among them, the background correction process is expressed as: ;

[0098] The noise removal process is expressed as: ;

[0099] The feature alignment process is expressed as: ;

[0100] Among them, is the raw spectral data, is the average value of the background signal, is the spectral data after background correction, is the spectral data after denoising, is the spectral data after feature alignment, is the denoising algorithm, is the feature alignment algorithm;

[0101] Furthermore, the process of data preprocessing also includes: near-infrared spectral preprocessing based on Raman spectra, performing baseline correction, feature normalization, and feature alignment processing to improve the data quality of near-infrared spectra;

[0102] The process of data preprocessing provided by the embodiments of the present disclosure aims to enhance the data quality, extract more useful features, and thus improve the accuracy and reliability of subsequent analysis.

[0103] The modeling process also includes a feature extraction step. In the feature extraction step, a dual-modal encoder is adopted. The dual-modal encoder includes a near-infrared encoder (NIR encoder) and a Raman encoder (Raman encoder).

[0104] The near-infrared encoder is used to extract near-infrared spectral features, which is expressed as: ;

[0105] The Raman encoder is used to extract Raman spectral features, which is expressed as: ;

[0106] Among them, represents the encoder of the near-infrared spectrum, represents the Raman spectrum encoder; represents the preprocessed spectral features;

[0107] Through the Feature Enhancement Module (FEM), the representation ability of the features is improved, which is expressed as:

[0108] ;

[0109] Among them, F is the input spectral feature vector, W is the trainable weight matrix, b is the bias term, σ is the activation function, is the enhanced feature representation;

[0110] Furthermore, by combining the Global Attention Module (GAM) and the Cross Attention Module, the dual-modal information interaction is optimized to improve the feature fusion effect. Among them, the global attention is expressed as:

[0111] ; ;

[0112] Among them, Q, K, and V are the query, key, and value of the input features, is the transpose of the key, d is the feature dimension, is the attention weight matrix, is the weighted global feature;

[0113] The cross attention is expressed as:

[0114] ; ; ; ;

[0115] Among them, , , respectively represent the query, key, and value of the near-infrared spectrum, , , respectively represent the query, key, and value of the Raman spectrum, The attention weight matrix representing the near-infrared spectrum, is the attention weight matrix of the Raman spectrum, is the transpose of the near-infrared spectrum bond vector, is the transpose of the Raman spectrum bond vector, is the spectral feature after fusion of the near-infrared spectrum, is the spectral feature after fusion of the Raman spectrum, where d represents the feature dimension;

[0116] The modeling process also includes a feature fusion step. In the feature fusion step, processing methods such as feature concatenation, weighted fusion, and feature selection for dimensionality reduction are adopted to optimize the feature representation and improve the generalization ability of the model; among them, the feature concatenation processing is expressed as: , where in the formula, represents the spectral feature after concatenation, is the spectral feature after fusion of the near-infrared spectrum, is the spectral feature after fusion of the Raman spectrum;

[0117] The modeling process also includes a contrast learning step. In this step, first, data augmentation is performed, and a positive and negative sample construction strategy based on contrast learning is adopted to expand the data for different spectral mode data distributions to improve the robustness of the model; then, contrast loss calculation is performed. Among them, in the supervised contrast learning framework, the total loss of the model is expressed as:

[0118] ;

[0119] Among them, L is the total loss value of the model, N is the total number of samples, is the feature vector of the i-th sample, is the feature vector of the sample positively correlated with the i-th sample, is the feature vector of the j-th sample, is the temperature parameter, which is used to control the smoothness of the contrast loss, represents the th feature vector of the sample, can be i or other indices, representing any sample in the batch; in the model training stage, the gradient of the loss function model parameters is calculated using the backpropagation algorithm, and the model parameters are updated through the gradient; in the model optimization stage, the model weights are adjusted based on the loss function minimization strategy to improve the detection accuracy and generalization ability.

[0120] Furthermore, in the model adaptation module provided in the embodiments of the present disclosure, the model receives new data input and compares it with the existing features. The similarity between the new data feature vector and the feature vector stored in the model is calculated, expressed as: , where, represents the feature vector of the new data, Represents the existing feature vectors of the model;

[0121] According to the comparison results between the new data and the existing features, the parameters of the model are dynamically adjusted. Specifically, an adaptive learning rate adjustment algorithm is adopted and the learning rate is adjusted according to the gradient of the objective function. The formula is expressed as:

[0122] ;

[0123] where α t is the current learning rate, β is the decay factor, is the minimum value of the learning rate, v t-1 is the sum of the squares of the gradients, c is a constant; α t-1 represents the previous learning rate;

[0124] The performance of the updated model is verified by calculating other performance metrics such as the accuracy of the model on the validation set. The formula is expressed as:

[0125] ;

[0126] where Accuracy represents the accuracy metric, TP represents the number of true positives, TN represents the number of true negatives, and Total Samples represents the total number of samples;

[0127] Finally, when the model verification result is positive, the updated model parameters are saved, that is, if the verification result shows an improvement in the model performance, then the updated model parameters will be saved;

[0128] Through the above update strategy, the model can continuously adapt to new data and improve its prediction accuracy and robustness.

[0129] Furthermore, the inference and prediction module uses the pre-trained model to make fast and accurate predictions on real-time spectral data, obtains the real-time spectral data file, preprocesses the data, and performs model inference using a convolutional neural network, expressed as: Y = softmax(W T X norm + b); where W and b are the weights and biases of the model respectively, W T represents the transpose of the weights; X norm is the normalized input data; Y represents the output result of the inference;

[0130] Furthermore, the prediction results are evaluated by accuracy and F1 value; this evaluation step ensures the reliability of the model's prediction results and provides a basis for the final decision.

[0131] Furthermore, the APP provides a user interface for controlling the operation of the detection device, including sample rotation, data acquisition, model training, and result viewing; through the APP, users can monitor the collected spectral data and processing results in real time, including feature extraction, model inference, and prediction results; in addition, the APP supports a data export function, allowing users to export the collected spectral data and analysis results to local or cloud, ensuring the stability and real-time of data transmission.

[0132] The above solutions are only illustrative of a preferred example and are not limited thereto. When implementing the present invention, appropriate substitutions and / or modifications can be made according to the needs of users.

[0133] The number of devices and the processing scale described here are used to simplify the description of the present invention. Applications, modifications, and variations of the present invention will be apparent to those skilled in the art.

[0134] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the field, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the equivalent scope, the present invention is not limited to the specific details and the illustrated examples here.

Claims

1. Rapid nondestructive testing system based on the fusion of near infrared spectroscopy and Raman spectroscopy, including detection device and intelligent terminal; It is characterized in that The detection device includes: An upper container (1), wherein a replaceable sample storage container (101) is rotatably arranged inside the upper container (1); the top layer of the replaceable sample storage container (101) has a screw cover (102); the replaceable sample storage container (101) is a partition structure, and the partition structure is divided into eight areas by four diagonal lines, each diagonal line is divided into two radial lines along the center, and each radial line has a sample groove (1011) in the middle; the bottom layer of the replaceable sample storage container (101) is a light-transmitting structure; A middle container (2), wherein the middle container (2) is provided with two openings, and both openings are provided with glass gates; the two openings are located on the same diameter of the middle container (2); one opening corresponds to a data collection port of a near-infrared spectrometer (201), and the other opening corresponds to a data collection port of a Raman spectrometer (202); A lower container (3), wherein an edge end (301) is arranged inside the lower container (3); data interaction and instruction transmission are performed between the edge end (301) and the intelligent terminal, and the edge end (301) has a data processing unit, which includes a data acquisition control module, a data modeling module, a model adaptation module, an inference prediction module, and an upper sample rotation control module, wherein the upper sample rotation control module is used to control the replaceable sample storage container (101) to rotate in increments of 45° inside the upper container (1).

2. The rapid nondestructive testing system based on the fusion of near infrared spectroscopy and Raman spectroscopy according to claim 1 is characterized in that: The opening provided on the middle container (2) is located directly below the movement path of the sample slot (1011) when the replaceable sample storage container (101) rotates.

3. The rapid nondestructive testing system based on the fusion of near infrared spectroscopy and Raman spectroscopy according to claim 2 is characterized in that: The near infrared spectrometer (201) and the Raman spectrometer (202) are both arranged inside the middle container (2) and are used to collect spectral data of the sample in the upper container (1); Both the near-infrared spectrometer (201) and the Raman spectrometer (202) are externally connected to data lines, and the data lines are connected to the edge end (301) in the lower container (3); The near-infrared spectrometer (201) covers the 900nm-1700nm band; The Raman spectrometer (202) covers a wavelength range of 271 nm to 2453 nm, and the laser excitation wavelength of the Raman spectrometer (202) is 785 nm.

4. The rapid nondestructive testing system based on the fusion of near infrared spectroscopy and Raman spectroscopy according to claim 3 is characterized in that: The data modeling module is used to process near-infrared and Raman spectral data, including preprocessing, feature extraction, feature fusion, and optimization of feature representation using contrastive learning algorithms; among them: The pre-processing step includes Raman spectrum pre-processing based on near infrared spectrum and near infrared spectrum pre-processing based on Raman spectrum; In the feature extraction step, a dual-mode encoder is used for feature extraction, including: using a near-infrared encoder to extract near-infrared spectral features ; Extract Raman spectral features using Raman encoder ; Enhance the spectral features through the feature enhancement module; Combine the global attention module and the cross attention module to optimize the bimodal information interaction; In the step of feature fusion, feature splicing, weighted fusion, and feature selection and dimensionality reduction are used to optimize feature representation; the feature splicing process is represented as follows: , where represents the spectral characteristics after splicing, is the spectral feature after near-infrared spectrum fusion, is the spectral feature after Raman spectrum fusion; Represents a splicing operation; In the step of using the contrastive learning algorithm to optimize feature representation, data enhancement is first performed, and a positive and negative sample construction strategy based on contrastive learning is adopted to expand data according to the data distribution in different spectral modes; then the contrastive loss is calculated, where the total loss of the model is expressed as: ； Among them, L is the total loss value of the model, N is the total number of samples, is the feature vector of the i-th sample, is the feature vector of the sample that is positively correlated with the i-th sample; is the feature vector of the jth sample, is the temperature parameter, which is used to control the smoothness of contrast loss; Indicates The feature vector of each sample is obtained; in the model training stage, the back propagation algorithm is used to calculate the gradient of the loss function model parameters, and the model parameters are updated through the gradient; in the model optimization stage, the model weights are adjusted based on the loss function minimization strategy.

5. The rapid nondestructive testing system based on the fusion of near infrared spectroscopy and Raman spectroscopy according to claim 4 is characterized in that: The Raman spectrum preprocessing step based on near infrared spectrum includes background correction, noise removal and feature alignment; The steps of near infrared spectrum preprocessing based on Raman spectrum include baseline correction, feature normalization and feature alignment.

6. The rapid nondestructive testing system based on the fusion of near infrared spectroscopy and Raman spectroscopy according to claim 5 is characterized in that: In the step of combining the global attention module and the cross attention module to optimize the bimodal information interaction, the global attention is expressed as: ; ; where Q, K, and V are the query, key, and value of the input feature. is the transpose of the key; d is the feature dimension, is the attention weight matrix, is the weighted global feature; The cross attention is expressed as: ； ； ； ； in, , , Represent the query, key and value of near infrared spectrum respectively, , , Represent the query, key and value of Raman spectrum respectively, represents the attention weight matrix of the near infrared spectrum, is the attention weight matrix of Raman spectroscopy, is the transpose of the near-infrared spectral bond vector, is the transpose of the Raman spectrum bond vector, is the spectral feature after near-infrared spectrum fusion, is the spectral feature after Raman spectrum fusion, and d represents the feature dimension.

7. The rapid nondestructive testing system based on the fusion of near infrared spectroscopy and Raman spectroscopy according to claim 6 is characterized in that: In the model adaptation module, the similarity between the feature vector of the new data and the feature vector stored in the model is calculated, and data comparison is performed based on the similarity; According to the comparison results, the parameters of the model are dynamically adjusted; among them, the learning rate is adjusted based on the adaptive learning rate adjustment algorithm and according to the gradient of the objective function, which is expressed as: ;in, is the current learning rate, is the attenuation factor, is the minimum value of the learning rate, is the sum of squares of gradients, is a constant; Represents the previous learning rate; The accuracy performance index of the model is calculated on the validation set to verify the performance of the updated model, which is expressed as: ； Among them, Accuracy represents the accuracy index, represents the number of true positive examples, represents the number of true negative examples, represents the total sample size; When the model validation result is positive, the updated model parameters are saved.

8. The rapid nondestructive testing system based on the fusion of near infrared spectroscopy and Raman spectroscopy according to claim 7 is characterized in that: In the inference prediction module, the pre-trained model is used to predict the real-time spectral data, including: Obtain real-time spectral data files and pre-process the data; Use convolutional neural network for model inference, expressed as: ,in, and The distributions are the weights and biases of the model, is the normalized input data; W T represents the transpose of the weight; Y represents the output result of the reasoning; The prediction results are evaluated by accuracy and F1 value.

Images