Absorption spectrum inversion method based on neural network and correlation method

Abstract

This invention discloses an absorption spectrum inversion method based on neural networks and correlation methods, mainly addressing the problems of low accuracy, poor real-time performance, and strict requirements for absorption spectrum data standards in existing absorption spectrum temperature inversion methods. The implementation scheme is as follows: acquire raw absorption spectrum data and preprocess it; construct a neural network model including a fully convolutional network (FCN) and a long short-term memory network (LSTM); train the neural network model using the preprocessed absorption spectrum dataset through backpropagation to obtain a trained neural network model; input the absorption spectrum data to be measured into the neural network model, and output the temperature range corresponding to the absorption spectrum; post-process the temperature range output by the neural network based on the correlation method to obtain the temperature value. This invention effectively improves the accuracy and real-time performance of absorption spectrum temperature inversion, reduces the requirements for absorption spectrum data standards, and can be used for gas composition analysis, environmental monitoring, drug analysis, and food safety testing in the fields of chemistry, physics, and biology.

Description

Technical Field

[0001] This invention belongs to the field of absorption optics technology, specifically relating to an absorption spectral inversion method that can be used for gas composition analysis, environmental monitoring, drug analysis, biomolecule detection, and food safety testing in the fields of chemistry, physics, and biology. Background Technology

[0002] Absorption spectroscopy is a method used to study the interaction between matter and light. By analyzing the degree to which a substance absorbs light of different wavelengths, information about its composition, temperature, concentration, and reaction kinetics can be obtained. It has wide applications in chemistry, physics, and biology.

[0003] Existing absorption spectroscopy inversion methods include spectrothermal methods, blackbody radiation methods, and color temperature measurement methods, all of which have several drawbacks. First, external environmental factors such as illumination and background radiation can interfere with the measurement results, especially in spectrothermal methods where interference is more pronounced. Second, some methods are limited by instrument resolution and sensitivity, potentially losing accuracy when measuring extremely high or low temperatures. Furthermore, blackbody radiation methods require consideration of the emissivity of the object's surface, a value difficult to measure accurately. These methods typically rely on physical assumptions, such as the blackbody radiation law, which may not hold under certain conditions, leading to measurement errors. Regular instrument calibration is also necessary; otherwise, uncertainty in temperature measurements may result. When selecting an appropriate method, a comprehensive consideration of application requirements, environmental conditions, and the characteristics of the method is essential to achieve optimal temperature measurement results.

[0004] Patent document CN20220391067.7 discloses a "Method and System for Extracting Tissue Optical Parameters Based on Long Short-Term Memory Networks," which introduces long short-term memory networks into a tissue spatially resolved diffuse reflectance spectrum generation model. This model fully exploits the correlations among spectral data points, increasing predictive power while reducing model parameters. When training the tissue spatially resolved diffuse reflectance spectrum generation model, the fiber optic probe structure is disregarded, and all spatially resolved diffuse reflectance spectra emitted from the tissue surface are included in the pre-dataset, resulting in a higher data dimensionality. While this method effectively improves the accuracy of the mapping model between tissue optical parameters and diffuse reflectance spectra, the insufficient ability of long short-term memory networks to extract local features leads to low accuracy in optical parameter extraction.

[0005] Patent document CN202010585647.0 discloses a "Deep Learning Model Applicable to Small Sample Hyperspectral Image Classification," which is based on an encoder-decoder architecture. It performs two random samplings on the original dataset to generate a set of random training sample sequences. A global convolutional long short-term memory module is used to extract spectral dependence features over long and short ranges, as well as spatial relationships of pixels within local ranges. Global spectral and spatial attention mechanisms are used to extract detailed spectral dependence features and spatial details, respectively, enabling the extraction of the most discriminative features even when training samples are limited, thus ensuring classification accuracy. However, this model has a relatively complex structure, high computational resource requirements, and is sensitive to parameter tuning, making it difficult to meet the needs of current industrial applications.

[0006] In summary, existing absorption spectral temperature inversion methods generally suffer from drawbacks such as low inversion accuracy, high computational complexity, poor real-time performance, and strict requirements for absorption spectral data standards. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing absorption spectral inversion methods in terms of computational complexity, inversion accuracy, and real-time performance. This invention proposes an absorption spectral inversion method based on neural networks and correlation methods. By combining the deep feature extraction capabilities of fully convolutional networks (FCN) and long short-term memory networks (LSTM) with the precise matching characteristics of correlation methods, this invention effectively improves the training efficiency and inversion accuracy of neural network models while significantly reducing computational complexity.

[0008] To achieve the above objectives, the technical solution of the present invention includes the following steps:

[0009] 1. An absorption spectral inversion method based on neural networks and correlation methods, characterized by comprising the following:

[0010] (1) Obtain the raw absorption spectrum data and preprocess it;

[0011] (2) Construct a neural network model including a fully convolutional network FCN and a long short-term memory network LSTM;

[0012] (3) The neural network model was trained using the preprocessed absorption spectrum dataset through backpropagation to obtain the trained neural network model.

[0013] (4) Input the absorption spectrum data to be measured into the neural network model and output the temperature range corresponding to the absorption spectrum;

[0014] (5) The temperature range output by the neural network is post-processed based on the correlation method to obtain the temperature value.

[0015] Furthermore, the construction of the neural network model, which includes a fully convolutional network (FCN) and a long short-term memory network (LSTM), is implemented as follows:

[0016] A fully convolutional network is constructed, consisting of three cascaded convolutional units. Each convolutional unit includes a batch regularization layer, a ReLU activation function, and a global pooling operation. The first convolutional unit has an 8x8 kernel size and 128 kernels; the second convolutional unit has a 5x5 kernel size and 256 kernels; and the third convolutional unit has a 3x3 kernel size and 128 kernels. Each convolutional kernel is initialized using Kamming to improve convergence speed.

[0017] A long short-term memory network was constructed, which consists of three cascaded long short-term memory network layers. Each long short-term memory network layer contains memory neurons. The first long short-term memory network layer contains 32 memory neurons; the second long short-term memory network layer contains 64 memory neurons; and the third long short-term memory network layer contains 128 memory neurons.

[0018] A feature fusion module is constructed, which includes convolutional layers and feature aggregation layers. The convolutional layers are used to fuse the output features of the fully convolutional network module and the long short-term memory network module, and the kernel size is 3*3. The feature aggregation layer is used to average the pixels of all channels of the output features of the convolutional layer at each spatial location to achieve feature aggregation.

[0019] The fully convolutional network FCN and the long short-term memory network LSTM are combined in parallel to form a dual-branch processing architecture for absorption spectral data. The features extracted from the two paths are integrated through a feature fusion module. The fusion process includes joint feature modeling using 3×3 convolutional layers and global weighting of features using feature aggregation layers. Finally, the integrated feature representation is mapped to the output space through fully connected layers to form a complete neural network model.

[0020] Compared with the prior art, the present invention has the following advantages:

[0021] Firstly, this invention achieves efficient inference of the temperature range corresponding to the absorption spectrum curve through the constructed FCN-LSTM neural network, which is significantly better than the complex process of traditional methods that require comparing temperature curves one by one. Moreover, the neural network model can eliminate the dependence on experimental conditions, effectively solving the problem of temperature measurement difficulties caused by the high similarity of absorption spectrum curves. That is, the temperature range can be quickly inferred with only one spectrum extraction.

[0022] Secondly, this invention organically integrates the features extracted by the fully convolutional network FCN and the long short-term memory network LSTM, making full use of the advantages of both. Specifically, it leverages the advantage of LSTM in precisely controlling the flow of information through input gates, forget gates, and output gates to efficiently extract long sequence features; and leverages the advantage of FCN in effectively extracting local features from sequence data and preserving temporal structure to better capture contextual information, thereby achieving high-precision and high-efficiency temperature inversion.

[0023] Third, this invention utilizes the powerful feature extraction capabilities of the fully convolutional network (FCN) and does not impose any restrictions on the length of the spectral curve sequence. It can effectively achieve temperature inversion regardless of the sequence length, overcoming the limitations of traditional methods that have strict requirements for absorption spectral data standards, making it applicable to a wider range of application scenarios. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the implementation of this invention;

[0025] Figure 2 This is a diagram of the neural network structure constructed in this invention;

[0026] Figure 3 This is an absorption spectrum data diagram obtained in this invention;

[0027] Figure 4 This is a graph showing the absorption spectrum data and the processed data in this invention;

[0028] Figure 5 This is a graph showing the results of temperature inversion of the selected absorption spectrum data in this invention. Detailed Implementation

[0029] The embodiments and effects of the present invention will be further described in detail below with reference to the accompanying drawings.

[0030] Reference Figure 1 The implementation steps of this embodiment are as follows:

[0031] Step 1: Obtain the raw absorption spectrum data and preprocess it.

[0032] In this embodiment, the input raw absorption spectrum data is as follows: Figure 3 As shown, Z-Score standardization was performed to eliminate the influence of units and improve data consistency. The standardized data is as follows: Figure 4 As shown;

[0033] Subsequently, Principal Component Analysis (PCA) was used to reduce the dimensionality of the standardized data: the covariance matrix was calculated and principal components were extracted by eigenvalue decomposition. The components were sorted by eigenvalue size, and the top k principal components with the largest contribution rate were selected to reduce the number of features in the data while preserving the main information, thus completing the dimensionality reduction process and obtaining the preprocessed absorption spectrum dataset.

[0034] Step 2: Construct a neural network model that includes a fully convolutional network FCN and a long short-term memory network LSTM.

[0035] Reference Figure 2 The implementation of this step is as follows:

[0036] 2.1) The neural network model constructed in this embodiment includes a fully convolutional network, a long short-term memory network, and a feature fusion module. Wherein:

[0037] The fully convolutional network consists of three cascaded convolutional units. Each convolutional unit includes a batch regularization layer, a ReLU activation function, and a global pooling operation. The first convolutional unit has an 8x8 kernel size and 128 kernels; the second convolutional unit has a 5x5 kernel size and 256 kernels; and the third convolutional unit has a 3x3 kernel size and 128 kernels. Each convolutional kernel is initialized using Kamming to improve convergence speed.

[0038] The long short-term memory network portion comprises three cascaded long short-term memory network layers, each containing memory neural units. The first long short-term memory network layer contains 32 memory neural units; the second long short-term memory network layer contains 64 memory neural units; and the third long short-term memory network layer contains 128 memory neural units.

[0039] The feature fusion module consists of a convolutional layer and a feature aggregation layer. The convolutional layer is used to fuse features extracted by the fully convolutional network and the long short-term memory network, and its convolutional kernel size is 3×3. The feature aggregation layer achieves efficient feature integration by averaging the pixel values ​​of the channel dimension at each spatial location.

[0040] 2.2) The fully convolutional network FCN and the long short-term memory network LSTM are combined in parallel to form a dual-branch processing architecture for absorption spectral data. This dual-branch architecture is used to extract features from the absorption spectral data. The extracted features are integrated through a feature fusion module, which performs joint feature modeling through a 3×3 convolutional layer and global weighting of features through a feature aggregation layer. The integrated feature representation is then mapped to the temperature range corresponding to the absorption spectral curve through an external fully connected layer to form a complete neural network model.

[0041] Step 3: Define the loss function for the neural network model.

[0042] Because absorption spectral curve data exhibits a "long-tail" effect, meaning the number of sample classes is imbalanced, this embodiment uses the hinged cross-entropy loss function as the loss function (Loss) for the neural network to improve model training accuracy. Its expression is as follows:

[0043]

[0044] Where y represents the prediction result of the neural network model, and margin threshold represents the boundary threshold. i Indicates the true label, This represents the probability predicted by the neural network model for the i-th sample.

[0045] Step 4: Train the neural network model using the adaptive stochastic gradient descent algorithm to obtain the trained neural network model.

[0046] 4.1) Input the preprocessed absorption spectrum dataset into the constructed neural network model to obtain the temperature range corresponding to the absorption spectrum data;

[0047] 4.2) Using the loss function, calculate the regression loss between the output temperature range and the corresponding original temperature range;

[0048] 4.3) The neural network model based on the fully convolutional network FCN and the long short-term memory network LSTM is trained in multiple rounds using an adaptive stochastic gradient descent algorithm until the regression loss converges, thus obtaining the trained neural network model.

[0049] Step 5: Use the trained neural network model to obtain the temperature range corresponding to the test set.

[0050] 5.1) Input the test data into the trained neural network model, extract local features of the spectral data through multi-layer convolution operation of the fully convolutional network FCN, extract global sequence features of the spectral data through the long short-term memory network LSTM, and prevent overfitting through the Dropout mechanism.

[0051] 5.2) The fusion module fuses the features extracted by the FCN branch of the fully convolutional network and the LSTM branch at multiple scales, and then further processes the fused features through an external fully connected layer to output the temperature range corresponding to the spectrum to be measured.

[0052] Step 6: Post-process the temperature range output by the neural network based on the correlation method to obtain a definite temperature value.

[0053] The correlation method is an algorithm used to calculate the similarity between two absorption spectrum curves.

[0054] In this embodiment, the specific implementation of post-processing the absorption spectrum data to be measured using the correlation method is as follows:

[0055] 6.1) Calculate the correlation coefficient r between the data to be tested and each curve in the reference curve set:

[0056]

[0057] Among them, the correlation coefficient r is an important indicator for measuring the degree of matching between the test data and the reference curve, x i Let y represent the i-th predicted value output by the neural network. i This represents the i-th value of the true target parameter. and are the mean of the predicted value and the actual value, respectively, and N is the total number of data samples;

[0058] 6.2) The temperature of the reference curve corresponding to the maximum value of r is selected as the temperature value output by the final neural network. The result is as follows: Figure 5 As shown.

[0059] The above content is a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Obviously, for those skilled in the art, several simple deductions or substitutions can be made without departing from the concept of the present invention, but these should all be considered within the scope of protection of the present invention.

Claims

1. An absorption spectral inversion method based on neural networks and correlation methods, characterized in that, Including the following: (1) Obtain the raw absorption spectrum data and preprocess it; (2) Construct a neural network model including a fully convolutional network FCN and a long short-term memory network LSTM. The implementation steps include the following: (2a) Construct a fully convolutional network containing three concatenated convolutional units, each of which contains batches. The system employs quantitative regularization layers, ReLU activation functions, and global pooling operations. The first convolutional unit has an 8x8 kernel size and 128 kernels; the second convolutional unit has a 5x5 kernel size and 256 kernels; and the third convolutional unit has a 3x3 kernel size and 128 kernels. Each convolutional kernel is initialized using Kamming to improve convergence speed. (2b) Construct a long short-term memory network, which consists of three cascaded long short-term memory network layers. Each long short-term memory network layer contains memory neural units. The number of memory neural units in the first long short-term memory network layer is 32; the number of memory neural units in the second long short-term memory network layer is 64; and the number of memory neural units in the third long short-term memory network layer is 128. (2c) Construct a feature fusion module, which includes a convolutional layer and a feature aggregation layer. The convolutional layer is used to fuse the output features of the fully convolutional network module and the long short-term memory network module. The size of the convolutional kernel is 3*3. The feature aggregation layer is used to average the pixels of all channels of the output features of the convolutional layer at each spatial location to achieve feature aggregation. (2d) The fully convolutional network FCN and the long short-term memory network LSTM are combined in parallel to form a dual-branch processing architecture for absorption spectral data. This dual-branch architecture is used to extract features from the absorption spectral data. The extracted features are integrated through a feature fusion module, which performs joint feature modeling through a 3×3 convolutional layer and global weighting of features through a feature aggregation layer. The integrated feature representation is then mapped to the temperature range corresponding to the absorption spectral curve through an external fully connected layer to form a complete neural network model. (3) The neural network model is trained using the preprocessed absorption spectrum dataset through backpropagation to obtain the trained neural network model. The steps include the following: (3a) Input the preprocessed absorption spectrum dataset into the constructed neural network model to obtain the temperature range corresponding to the absorption spectrum data; (3b) Combining the characteristics of cross-entropy loss and hinge loss, define the loss function of the neural network model. : ； in, This indicates the model's prediction results. Boundary threshold, Indicates the true label, The model represents the first The predicted probability of each sample; (3c) Using the loss function, calculate the regression loss between the output temperature range and the corresponding original temperature range; (3d) The neural network model based on the fully convolutional network FCN and the long short-term memory network LSTM is trained in multiple rounds using an adaptive stochastic gradient descent algorithm until the regression loss converges, thus obtaining the trained neural network model. (4) Input the absorption spectrum data to be measured into the neural network model and output the temperature range corresponding to the absorption spectrum; (5) The temperature range output by the neural network is post-processed based on the correlation method to obtain the temperature value.

2. The method according to claim 1, characterized in that, Step (1) involves preprocessing the raw absorption spectrum data, and the steps include the following: (1a) Perform Z-Score normalization on the raw absorption spectral data; (1b) Principal component analysis (PCA) was used to reduce the dimensionality of the standardized data in order to reduce the number of features and retain the main information.

3. The method according to claim 1, characterized in that, Step (4) Input the absorption spectrum data to be measured into the neural network model and output the temperature range corresponding to the absorption spectrum. The steps include the following: (4a) The absorption spectrum data to be measured is copied and input into the fully convolutional network FCN and the long-term and short-term networks respectively. In the LSTM memory network; (4b) Fully convolutional network (FCN) extracts local features and multi-scale information of spectral data through multi-layer convolution operations; (4c) The Long Short-Term Memory (LSTM) network first undergoes dimensionality adjustment to adapt to sequence modeling requirements, and then utilizes... The LSTM layer models the time dependence of spectral sequence data and uses the Dropout mechanism to prevent overfitting; (4d) Multi-scale fusion of features extracted from the fully convolutional network FCN branch and the long short-term memory LSTM branch. The fused features are then further processed through a fully connected layer to output the temperature range corresponding to the measured spectrum.

4. The method according to claim 1, characterized in that, In step (5), the temperature range output by the neural network is post-processed based on the correlation method. The implementation steps include the following: (5a) Obtain the corresponding absorption spectrum data based on the temperature range predicted by the neural network as a set of reference curves; (5b) Calculate the correlation coefficient between the absorption spectrum data to be measured and each curve in the set of reference curves. ; (5c) Sort all correlation coefficients and select the reference curve temperature corresponding to the maximum value as the final value. Temperature value output via network.

5. The method according to claim 4, characterized in that, Calculate the correlation coefficient in step (5b). The formula is as follows: ； in, This represents the i-th predicted value output by the neural network. This represents the i-th value of the true target parameter. and These are the mean of the predicted value and the actual value, respectively. This represents the total number of data samples.

Images