A hybrid machine learning-driven approach for nonlinear generation of Raman spectra

Abstract

The present invention relates to the field of Raman spectroscopy and discloses a hybrid machine learning-driven nonlinear generation method for Raman spectra, which is used to meet the requirements of modern Raman spectroscopy analysis technology for mixture spectral data with wide coverage, complete types, and reliable quality. It includes generating a preliminary mixture spectrum by weighted linear combination of the spectra of each pure component based on the concentration ratio; enhancing data diversity; generating a predicted spectrum; optimizing the generator parameters through multi-level feature comparison and adversarial training of the discriminator; matching and deleting false peaks to reconstruct physically reasonable characteristic peaks; dynamically adjusting the characteristic peak intensity to output high-fidelity spectral data. By integrating physical constraints and deep learning, the present invention significantly improves the peak position accuracy, intensity consistency, and noise robustness of the generated spectrum, supports the quantitative analysis of complex mixtures over a wide concentration range, and provides an efficient solution for Raman spectroscopy database construction and chemical detection.

Description

Technical Field

[0001] The present invention relates to the field of Raman spectroscopy, and in particular to a hybrid machine learning-driven Raman spectroscopy nonlinear generation method. Background Art

[0002] Traditional Raman spectroscopy acquisition systems rely on highly specialized operating modes with extremely stringent requirements for experimental environment and sample quality to obtain reliable, high-quality spectral data. Limited by the difficulty in obtaining available chemicals, especially hazardous chemicals, traditional Raman spectroscopy acquisition methods face significant bottlenecks in acquiring spectral data for mixtures. This not only restricts in-depth analysis of the molecular structures of hazardous chemicals but also severely impacts their application in quantitative concentration analysis. More importantly, the vast amount of existing mixture spectral data has yet to be fully explored and effectively utilized.

[0003] According to the excitation principle of Raman scattering light, under specific experimental conditions (such as constant excitation light intensity and fixed measurement temperature), the Raman scattering intensity of a substance exhibits a strict linear positive correlation with its molecular concentration. Based on this principle, researchers proposed a solution to construct a Raman spectral data generation model using a generative adversarial network (GAN). By building a neural network model to learn real spectral characteristics, it can generate high-quality, high-fidelity simulated Raman spectral data.

[0004] However, while GANs can theoretically effectively expand Raman spectral databases and have made some progress in cross-instrument data generation between scientific-grade Raman and handheld Raman spectrometers, as well as in optimizing the generation of pure substance spectral data, existing GAN models rely solely on game theory to guide spectral generation and lack explicit modeling of physical laws, such as the excitation principle of Raman scattering. This lack of physical constraints makes it difficult for the models to effectively distinguish between signal and noise, and the generated spectral data lacks physical plausibility. When processing high-dimensional and complex data such as Raman spectra, GAN models are prone to mode collapse and training instability during training, making it difficult to converge to the realistic data distribution. Therefore, it is difficult to directly and reasonably generate Raman spectral data for mixtures using GANs alone. In fact, molecular interactions between components in a mixture can induce nonlinear superposition effects in the spectrum. These effects not only cause characteristic peaks to shift, broaden, or change in intensity, but can also generate new vibrational modes. This complex nonlinear transformation makes it impossible to accurately model the mixture spectrum by simply linearly combining the input pure substance Raman spectra according to concentration ratios. Linear operations cannot fully capture the spectral changes caused by intermolecular interactions, resulting in significant deviations in the physical properties and details of the generated spectral data from the actual situation. The simple weighted superposition of the spectra of each component not only ignores the nonlinear effects caused by intermolecular interactions, but also fails to effectively deal with noise and background interference. In addition, due to issues such as harsh experimental conditions, limited access to chemicals, and the compatibility of mixtures, the scarcity of data samples makes it difficult to fully cover the concentration range and component combinations of various mixtures, resulting in the inability to accurately learn the complex spectral feature changes using only the GAN model. The lack of physical law modeling, complex nonlinear effects, harsh experimental conditions, and scarce data further limit the GAN's ability to learn complex spectral features, making it difficult to generate high-quality and physically reasonable Raman spectral data for mixtures. This limitation directly leads to the insufficient generalization ability of the generative model: when faced with new component combinations or concentration ratios, the model finds it difficult to accurately predict and generate the corresponding mixture spectral features.

[0005] Therefore, we propose a hybrid machine learning-driven Raman spectrum nonlinear generation method to address the above problems. Summary of the Invention

[0006] The present invention provides a hybrid machine learning-driven Raman spectroscopy nonlinear generation method to meet the requirements of modern Raman spectroscopy analysis technology for mixture spectral data with wide coverage, complete types and reliable quality.

[0007] The first aspect of the present invention provides a hybrid machine learning-driven nonlinear generation method for Raman spectra, which includes: performing weighted calculation on the Raman spectra of each pure component according to their concentration ratio, extracting characteristic peak information and generating a preliminary mixture spectrum; performing controllable noise injection, random spectral masking, data mirror inversion or spectral fragment shifting operations based on the preliminary mixture spectrum to generate diversified enhanced data; inputting the enhanced data into a generator, capturing the nonlinear interaction characteristics of the mixture spectrum through a deep convolutional neural network and a long short-term memory network architecture to generate a predicted spectrum; performing multi-level feature comparison between the predicted spectrum and the real spectrum through a discriminator, outputting the discrimination result to drive the generator to optimize the parameters to obtain a generated spectrum; performing wavelet transform baseline correction on the generated spectrum, and deleting false peaks and reconstructing characteristic peaks based on the chemical bond vibration frequency matching standard spectrum library to obtain a denoised generated spectrum; obtaining a concentration error based on the denoised generated spectrum, performing gradient correction on the characteristic peak intensity in combination with a simulated annealing algorithm, and outputting physically reasonable mixture Raman spectrum data.

[0008] Optionally, in a first implementation method of the first aspect of the present invention, it includes: constructing a structured data set containing spectral data and concentration parameters of each component based on the input pure component name and its corresponding concentration ratio; using a peak-finding algorithm to extract characteristic peak parameters of the Raman spectrum of each pure component, the characteristic peak parameters including peak position, peak intensity and half-peak width; performing weighted calculation on the characteristic peak intensity based on the concentration ratio to generate weighted characteristic peak intensity data of each component; combining the weighted characteristic peak intensity data in wavenumber order to generate a preliminary mixture spectrum containing the characteristic peaks of all components.

[0009] Optionally, in a second implementation method of the first aspect of the present invention, it includes: performing a controllable noise injection operation on the preliminary mixture spectrum to generate a noise-enhanced spectrum; performing a random spectral masking operation on the preliminary mixture spectrum to generate a locally masked enhanced spectrum; performing a data mirror inversion operation on the preliminary mixture spectrum to generate a reversed enhanced spectrum; performing a spectral segment shifting operation on the preliminary mixture spectrum to generate a shifted enhanced spectrum; and selecting at least one from the noise-enhanced spectrum, the locally masked enhanced spectrum, the reversed enhanced spectrum or the shifted enhanced spectrum as diversified enhancement data.

[0010] Specifically, a controllable noise injection operation is performed on the preliminary mixture spectrum to generate a noise-enhanced spectrum, including: generating random noise within a preset noise intensity range, and superimposing the random noise onto the intensity data of the preliminary mixture spectrum.

[0011] Optionally, in a third implementation method of the first aspect of the present invention, it includes: performing multi-scale feature extraction and time series modeling on the enhanced data through the encoding layer to generate fusion features; performing spectral reconstruction on the fusion features through the decoding layer to generate multi-channel prediction data; performing channel integration and nonlinear mapping on the multi-channel prediction data through the fully connected layer to generate a single-channel prediction spectrum.

[0012] Specifically, the enhanced data is subjected to multi-scale feature extraction and temporal modeling through the coding layer to generate fusion features, including: extracting local multi-scale features of the spectrum using a one-dimensional convolutional layer; modeling the temporal dependency of spectral wavenumbers through a long short-term memory network to capture global context features; and fusing local multi-scale features with global context features to generate fusion features.

[0013] Optionally, in a fourth implementation method of the first aspect of the present invention, it includes: performing local and global feature analysis on the predicted spectrum and the true spectrum through a multi-level feature extraction network to generate a discriminant feature vector; calculating the true and false probability based on the discriminant feature vector to generate a discriminant loss function; transferring the gradient of the discriminant loss function to the generator through the back propagation algorithm to drive the optimization of the generator parameters; and using the predicted spectrum output by the optimized generator as the final generated spectrum.

[0014] Specifically, a multi-level feature extraction network is used to perform local and global feature analysis on the predicted spectrum and the true spectrum to generate a discriminant feature vector, including: using a multi-layer one-dimensional convolutional network to extract the local characteristic peak shape and intensity distribution of the spectrum; extracting the global statistical features of the spectrum through a fully connected layer, including peak position offset, half-peak width mean and signal-to-noise ratio; and splicing the local features and global features into a discriminant feature vector.

[0015] Optionally, in a fifth implementation of the first aspect of the present invention, it includes: performing wavelet transform baseline correction on the generated spectrum to generate a baseline-corrected spectrum; performing chemical bond vibration frequency matching on the baseline-corrected spectrum to generate false peak marker data; performing peak deletion and reconstruction on the baseline-corrected spectrum based on the false peak marker data to generate a denoised generated spectrum; and outputting a denoised generated spectrum that completely retains the physical rationality characteristics.

[0016] Specifically, wavelet transform baseline correction is performed on the generated spectrum to generate a baseline-corrected spectrum, including: extracting a low-frequency baseline component of the generated spectrum by wavelet decomposition; and subtracting the low-frequency baseline component from the original generated spectrum to eliminate baseline drift and background noise.

[0017] Optionally, in a sixth implementation method of the first aspect of the present invention, it includes: performing quantitative analysis on the denoised generated spectrum to generate component concentration errors; constructing an objective function of the simulated annealing algorithm based on the concentration error vector to generate initial correction parameters; performing gradient correction on the characteristic peak intensity through iterative perturbation and optimization to generate optimized spectral data; and outputting the final optimized spectral data as physically reasonable Raman spectral data of the mixture.

[0018] Specifically, the denoised spectrum is quantitatively analyzed to generate component concentration errors, including: extracting the intensity data of each characteristic peak in the denoised spectrum through a quantitative analysis module; calculating the predicted concentration value of each component based on a preset concentration-intensity mapping function; and comparing the predicted concentration value with the initial target concentration to generate a concentration error vector.

[0019] Specifically, the characteristic peak intensity is gradient-corrected through iterative perturbation and optimization to generate optimized spectral data, including: in each iteration, randomly selecting the characteristic peak interval and applying intensity perturbation to generate the perturbed spectrum; recalculating the concentration error of the perturbed spectrum and updating the objective function value; accepting or rejecting the perturbation according to the Metropolis criterion, and giving priority to retaining the correction amount that reduces the error; reducing the temperature parameter and narrowing the perturbation range until the maximum number of iterations is met or the objective function value is lower than the preset error threshold.

[0020] Beneficial effects of the present invention:

[0021] It solves the nonlinear superposition effect of mixture spectra, breaks through the limitations of linear combination modeling, and introduces quantitative analysis to ensure the physical constraints and chemical laws of the generated data. Ultimately, high-quality mixture Raman spectral data covering a wide concentration range, diverse component combinations and low noise interference are generated. Specifically, it can accurately simulate the variation law of the mixture spectrum under different concentration gradients, covering the full range from extremely low concentrations to saturation concentrations. At the same time, this method supports a variety of component combinations and can handle the interactions between components with different chemical properties. In addition, the generated spectral data has low noise characteristics, which can effectively distinguish between real signals and background interference, ensuring that the signal-to-noise ratio of the data meets the actual analysis requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is the overall architecture diagram of the present invention;

[0023] Figure 2 Schematic diagram of the generator-discriminator of the present invention;

[0024] Figure 3 Schematic diagram of the characteristic peak denoising module of the present invention;

[0025] Figure 4 Schematic diagram of the peak alignment correction module of the present invention;

[0026] Figure 5 This is a spectrum data diagram of a real mixture of the present invention;

[0027] Figure 6 a data plot generated for the nonlinear model of the present invention;

[0028] Figure 7 a plot of real data generated for the nonlinear model of the present invention;

[0029] Figure 8 a plot of data generated for the linear model of the present invention;

[0030] Figure 9 A plot of real data generated for the linear model of the present invention;

[0031] Figure 10 A cosine similarity graph between the generated spectrum and the true spectrum of the present invention;

[0032] Figure 11 A quantitative comparison diagram of the generated spectrum and the real spectrum of the present invention;

[0033] Figure 12 This is a comparison chart of the characteristic peak intensity accuracy of the present invention. DETAILED DESCRIPTION

[0034] Embodiments of the present invention provide a hybrid machine learning-driven method for nonlinear Raman spectra generation, designed to meet the needs of modern Raman spectroscopy analysis technology for broad, comprehensive, and reliable mixture spectral data. In the present specification and claims, and in the accompanying drawings, the terms "first," "second," "third," "fourth," and so forth (if any) are used to distinguish similar items and are not necessarily intended to describe a particular order or precedence. It should be understood that such terms are interchangeable where appropriate, such that the embodiments described herein can be implemented in an order other than that illustrated or described herein. Furthermore, the terms "including," "comprising," "having," and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus.

[0035] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 An embodiment of the hybrid machine learning driven Raman spectrum nonlinear generation method in the embodiment of the present invention includes:

[0036] 101. Perform weighted calculation on the Raman spectra of each pure component according to its concentration ratio, extract characteristic peak information and generate a preliminary mixture spectrum;

[0037] It is understood that the execution subject of the present invention can be a hybrid machine learning driven Raman spectrum nonlinear generation system, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking the server as the execution subject as an example.

[0038] Specifically, according to the input pure component names and their corresponding concentration ratios, a structured data set containing the spectral data and concentration parameters of each component is constructed;

[0039] The peak-finding algorithm is used to extract the characteristic peak parameters of the Raman spectrum of each pure component. The characteristic peak parameters include peak position, peak intensity and half-peak width.

[0040] The characteristic peak intensity is weighted based on the concentration ratio to generate weighted characteristic peak intensity data for each component;

[0041] The weighted characteristic peak intensity data are combined in wavenumber order to generate a preliminary mixture spectrum containing the characteristic peaks of all components.

[0042] 102. Perform controllable noise injection, random spectrum masking, data mirror inversion or spectrum segment shifting operations based on the preliminary mixture spectrum to generate diversified enhanced data;

[0043] Specifically, a controllable noise injection operation is performed on the preliminary mixture spectrum to generate a noise-enhanced spectrum;

[0044] Specifically, the method includes: generating random noise within a preset noise intensity range (signal-to-noise ratio in the range of 3-10), and superimposing the random noise onto the intensity data of the preliminary mixture spectrum;

[0045] Performing random spectral masking operation on the preliminary mixture spectrum to generate a local masking enhanced spectrum;

[0046] Specifically, it includes: randomly selecting a continuous wave number interval and setting the spectral intensity data within the interval to zero;

[0047] Performing a data mirror inversion operation on the preliminary mixture spectrum to generate an inverse enhanced spectrum;

[0048] Specifically, the method includes: reversing the wave number sequence of the preliminary mixture spectrum to generate mirror-symmetrical spectrum data;

[0049] performing a spectral fragment shift operation on the preliminary mixture spectrum to generate a shift-enhanced spectrum;

[0050] Specifically, the wave number sequence of the preliminary mixture spectrum is shifted to the left or right by a preset wave number offset, and the empty value area beyond the original wave number range is filled with zero value;

[0051] At least one of a noise-containing enhancement spectrum, a local masking enhancement spectrum, a reverse enhancement spectrum, or a shifted enhancement spectrum is selected as the diversified enhancement data and inputted into the generator.

[0052] 103. The enhanced data is input into the generator, and the nonlinear interaction characteristics of the mixture spectrum are captured through the deep convolutional neural network and long short-term memory network (LSTM) architecture to generate the predicted spectrum;

[0053] Specifically, the enhanced data is subjected to multi-scale feature extraction and time series modeling through the encoding layer to generate fusion features;

[0054] Specifically include:

[0055] A one-dimensional convolutional layer is used to extract local multi-scale features of the spectrum;

[0056] The temporal dependency of spectral wavenumbers is modeled through a long short-term memory network (LSTM) to capture global context features.

[0057] Fuse local multi-scale features with global context features to generate fused features;

[0058] The fusion features are spectrally reconstructed through the decoding layer to generate multi-channel prediction data;

[0059] Specifically include:

[0060] The transposed convolution operation is used to upsample the fused features layer by layer to restore the spectral dimension;

[0061] The features of the encoding layer are fused with the intermediate features of the decoding layer through the residual connection layer to enhance the feature transfer capability;

[0062] Output multi-channel prediction data with the same wave number length as the input enhanced data;

[0063] Through the fully connected layer, the multi-channel prediction data is integrated and nonlinearly mapped to generate a single-channel prediction spectrum;

[0064] Specifically include:

[0065] Input multi-channel prediction data into the fully connected network to learn the nonlinear relationship between channels;

[0066] Output a single-channel predicted spectrum that is aligned with the true spectrum wavenumber as the output of the generator.

[0067] 104. The discriminator performs a multi-level feature comparison between the predicted spectrum and the true spectrum, outputs the discrimination result to drive the generator to optimize the parameters and obtain the generated spectrum;

[0068] Specifically, a multi-level feature extraction network is used to perform local and global feature analysis on the predicted spectrum and the true spectrum to generate a discriminant feature vector;

[0069] Specifically include:

[0070] A multi-layer one-dimensional convolutional network is used to extract the local characteristic peak shape and intensity distribution of the spectrum;

[0071] The global statistical features of the spectrum are extracted through the fully connected layer, including peak offset, half-peak width mean and signal-to-noise ratio;

[0072] Concatenate local features and global features into a discriminant feature vector;

[0073] Calculate the true and false probability based on the discriminant feature vector and generate the discriminant loss function;

[0074] Specifically include:

[0075] The discriminant feature vector is input into the fully connected classifier, and the probability value of the predicted spectrum being the real data is output;

[0076] Calculate the cross entropy loss function based on the difference between the probability value and the true label (the true spectrum is 1 and the predicted spectrum is 0);

[0077] The gradient of the discriminant loss function is transferred to the generator through the back-propagation algorithm to drive the optimization of the generator parameters;

[0078] Specifically include:

[0079] Fix the discriminator network parameters and use the gradient to update the weights of the generator's convolutional layer, long short-term memory network layer, and fully connected layer;

[0080] Iteratively optimize the generator so that the predicted spectrum it outputs is close to the real spectrum in terms of local characteristic peak distribution and global statistical characteristics;

[0081] The predicted spectrum output by the optimized generator is used as the final generated spectrum.

[0082] 105. Perform wavelet transform baseline correction on the generated spectrum, and match the standard spectrum library based on the chemical bond vibration frequency, delete the false peaks and reconstruct the characteristic peaks to obtain the denoised generated spectrum;

[0083] Specifically, performing wavelet transform baseline correction on the generated spectrum to generate a baseline-corrected spectrum;

[0084] Specifically include:

[0085] The low-frequency baseline component of the generated spectrum was extracted by wavelet decomposition;

[0086] Subtract the low-frequency baseline component from the original generated spectrum to eliminate baseline drift and background noise;

[0087] Baseline-corrected spectra were matched for chemical bond vibration frequencies to generate false peak marker data;

[0088] Specifically include:

[0089] Compare the characteristic peak positions of the baseline-corrected spectrum with the chemical bond vibration frequencies in the standard spectrum library;

[0090] The peak positions that do not match the threshold range of the standard spectral library are marked as false peaks;

[0091] Peak deletion and reconstruction of the baseline-corrected spectrum are performed based on the false peak marker data to generate a denoised generated spectrum;

[0092] Specifically include:

[0093] Delete the intensity data marked as false peaks and retain the characteristic peaks that conform to the laws of chemical bond vibration;

[0094] The Voigt function was used to fit the retained characteristic peaks and reconstruct the peak position, peak intensity and half-peak width parameters;

[0095] Output is a denoised spectrum that fully preserves the physical plausibility characteristics.

[0096] 106. The concentration error is obtained based on the denoising spectrum, and the characteristic peak intensity is gradient corrected by combining the simulated annealing algorithm to output physically reasonable Raman spectrum data of the mixture.

[0097] Specifically, the denoised generated spectra are quantitatively analyzed to generate component concentration errors;

[0098] Specifically include:

[0099] The intensity data of each characteristic peak in the denoised spectrum is extracted through the quantitative analysis module;

[0100] Calculate the predicted concentration value of each component based on the preset concentration-intensity mapping function;

[0101] Compare the predicted concentration value with the initial target concentration to generate a concentration error vector;

[0102] The objective function of the simulated annealing algorithm is constructed based on the concentration error vector to generate the initial correction parameters;

[0103] Specifically include:

[0104] Set the initial temperature, cooling rate and maximum number of iterations;

[0105] The objective function is defined as the root mean square error of the concentration error vector;

[0106] Initialize the characteristic peak intensity correction amount to zero vector;

[0107] Perform gradient correction on characteristic peak intensity through iterative perturbation and optimization to generate optimized spectral data;

[0108] Specifically include:

[0109] In each iteration, the characteristic peak interval is randomly selected and intensity perturbation is applied to generate the perturbed spectrum;

[0110] Recalculate the concentration error of the perturbed spectrum and update the objective function value;

[0111] Accept or reject the perturbation according to the Metropolis criterion, giving priority to retaining the correction that reduces the error;

[0112] Lower the temperature parameter and reduce the perturbation range until the maximum number of iterations is met or the objective function value is lower than the preset error threshold;

[0113] The final optimized spectral data are output as physically reasonable Raman spectral data of the mixture.

[0114] The following is a more specific example of measuring the spectrum of a chemical mixture. , its mathematical expression can be defined as ,in, is background noise. The existing spectral preprocessing algorithm can effectively eliminate noise interference such as baseline and obtain high-quality Raman data of the mixture Therefore, the ideal measured spectrum of the mixture can be expressed as:

[0115]

[0116] in, For the Spectral contribution of the components in the mixture, combination coefficient and ingredient concentration Ideally, there should be a linear proportional relationship between them. However, in practice, due to the intermolecular forces between the components in the mixture, the spectrum of a certain component often undergoes nonlinear changes, such as peak broadening or wavenumber drift. If the above formula is used to predict the spectrum of the mixture, this nonlinear effect will cause significant fitting errors in the results. Therefore, the spectrum of the mixture Should be expressed as follows:

[0117]

[0118] in for The spectral contribution of the first component in the mixture is There are significant differences in peak position, peak shape and peak intensity. The actual component spectrum affected by the interaction It cannot be directly measured experimentally.

[0119] Therefore, to solve the problem of complex nonlinear changes in mixture spectra, the present invention proposes a hybrid machine learning driven Raman spectrum nonlinear generation method. The overall architecture is as follows: Figure 1 As shown in Figure 2, the method consists of a linear combination module, a data enhancement module, a generator, a discriminator, a feature peak denoising module, and a peak alignment correction module.

[0120] The linear combination module designed in this invention acquires information from the initial input data, determines the number of mixture components in the initial input data by the number of spectra, and obtains the name of each component and the corresponding concentration ratio through user input, and constructs this information into a structured Python dictionary:

[0121] {Ingredients 1: , ingredient 2: ,…,Element : }

[0122] {Ingredients 1: , ingredient 2: ,…,Element : }

[0123] (where m represents the mth component in the mixture) is the density or concentration ratio of the mth component in the final mixture, which is a weight coefficient. The peak search algorithm is then used to accurately extract the spectrum of each component. The characteristic peak position, peak intensity, full peak half width and other key information are obtained by weighting the characteristic peaks using the concentration ratio and intelligently combining the weighted characteristic peaks in wavenumber order to generate a preliminary mixed spectrum containing the characteristic peaks of all components, so as to realize the generation of multi-component mixture spectra in arbitrary proportions, thereby providing high-quality initial data for subsequent nonlinear optimization.

[0124] The data augmentation module integrates four data augmentation techniques: Gaussian noise injection, random spectral masking, data mirror inversion, and spectral segment shifting. To illustrate the enhancement effect of each technique, we use the randomly generated simplified data [0.2, 0.4, 0.6, 0.8, 1.0, 0.7, 0.5] as an example:

[0125] Gaussian noise injection is random noise with a range of 0-0.1 for each data: [0.22, 0.38, 0.63, 0.79, 0.97, 0.72, 0.51]

[0126] The random spectral mask covers the data in a certain range to 0: [0.2, 0, 0, 0.6, 1.0, 0, 0]

[0127] Data mirror inversion is to arrange all data in reverse order: [0.5, 0.7, 1.0, 0.8, 0.6, 0.4, 0.2]

[0128] Spectral segment shifting uses random shift amounts to shift the data to the left or right in random directions (e.g. +2 means shifting 2 bits to the right): [0.7, 0.5, 0.2, 0.4, 0.6, 0.8, 1.0]

[0129] The data augmentation module uses a randomized augmentation strategy to process the output spectra of the linear combination module using one of four techniques. Furthermore, augmentation parameters (such as noise intensity, masking ratio, and shift amount) are randomly generated within a preset range. This "partially controllable" strategy ensures data diversity while maintaining essentially unchanged spectrochemical characteristics.

[0130] like Figure 2 As shown in the figure, the generator under the GAN framework uses a deep neural network to learn the features of the spectrum processed by the data enhancement module, generates a predicted spectrum, and inputs it together with the real spectrum into the discriminator for authenticity identification. The generator continuously optimizes the network parameters based on the feedback of the discriminator. This generation-discrimination adversarial training process continues to iterate until the system reaches a Nash equilibrium.

[0131] Specifically, the generator adds the gradient feedback provided by the discriminator to the generator's loss function to achieve rapid parameter optimization. The detailed generator-discriminator workflow under the GAN framework is as follows:

[0132] Step 1: The generator uses the preprocessed spectrum output by the data augmentation module as the initial input of the model. The input data is randomly enhanced after being fused with the preliminary feature extraction in the previous module, thereby obtaining high-quality input data that retains the key features of the original spectrum and has sufficient diversity.

[0133] The second step: The encoding layer integrates the deep convolutional network and the long short-term memory network architecture, regards the spectral wavenumber frequency as a time series, uses the long short-term memory network to model the temporal dependency between features, captures the global context information of the spectral features, and then downsamples the input spectrum layer by layer through a series of one-dimensional convolution and maximum pooling operations to capture the multi-scale features of the spectrum. An activation function is connected after each convolution-pooling operation to introduce nonlinear transformation capabilities.

[0134] Step 3: The residual connection layer continuously reduces the data dimension by stacking multiple one-dimensional convolutional layers, capturing deeper spectral feature representations. To prevent the gradient vanishing problem caused by excessive network depth, the feature output of the encoding layer is fused with the convolution result of the current layer.

[0135] Step 4: The decoding layer adopts a symmetrical upsampling architecture, gradually restores the spectral dimension through transposed convolution operations, and outputs multi-channel prediction data with the same length as the input spectrum.

[0136] Step 5: The fully connected layer integrates the multi-channel prediction data generated by the decoding layer, uses the fully connected network to learn the nonlinear mapping relationship between channels, and outputs a single-channel mixture prediction spectrum with the same length as the input spectrum.

[0137] Step 6: The discriminator uses a binary classification mechanism, labeling the predicted data 1 for true spectral data and the generated data 0 for generated spectral data. The predicted data and the true data are fed into the discriminator's feature extraction network. After multiple layers of convolution and nonlinear activation, the discriminator outputs the corresponding probability of discrimination. Based on the difference between the predicted results and the true labels, the discriminator calculates the loss and updates its parameters to improve its discrimination capabilities.

[0138] Step 7: As an important part of the generator loss function, the discriminant error provides the generator with optimization gradients through backpropagation, guiding it to adjust the network parameters to generate more realistic predicted spectra.

[0139] This process is repeated in subsequent iterations until the GAN framework reaches Nash equilibrium.

[0140] like Figure 3 As shown in the figure, the characteristic peak denoising module designed in the present invention first performs baseline correction on the generated spectrum through wavelet transform to remove low-frequency background noise; then, based on the physical constraints of chemical bond vibration frequencies, the peak position of the generated spectrum is matched with the standard component spectrum library to delete false peaks that do not conform to the chemical bond vibration characteristics; finally, the Voigt function is used to iteratively fit and reconstruct the retained characteristic peaks to restore key spectral features such as peak position, peak intensity and peak shape.

[0141] like Figure 4 As shown, the peak alignment correction module first receives spectral data from the characteristic peak denoising module and inputs it into the quantitative analysis module to calculate the concentration ratios of each component. Because the initial concentration ratios of the mixture components are pre-set baseline values, the quality of the generated spectrum can be accurately assessed by comparing the quantitative analysis results with the preset values. If the concentration ratio error is within the allowable range, the spectrum is considered ideal and directly output. If it exceeds the threshold, peak alignment correction is initiated.

[0142] Specifically, the peak alignment correction module is systematically optimized based on the simulated annealing algorithm. First, the algorithm parameters are initialized, including the initial temperature, cooling rate, and concentration difference objective function; then, during the iteration process, the spectral characteristic peak interval is randomly selected to apply controllable perturbations, the concentration error is recalculated through quantitative analysis, and the Metropolis criterion is used to dynamically accept new solutions - solutions with reduced errors are adopted first, while inferior solutions are accepted with probability to escape the local optimum; as the temperature gradually decreases, the algorithm convergence increases, and the probability of accepting inferior solutions decreases accordingly. When the maximum number of iterations is met or the objective function value reaches the preset error threshold (the error is less than 10 -5 ), the algorithm terminates and outputs the optimal correction spectrum.

[0143] The above describes the hybrid machine learning driven Raman spectrum nonlinear generation method in the embodiment of the present invention. The following describes the hybrid machine learning driven Raman spectrum nonlinear generation system in the embodiment of the present invention. An embodiment of the hybrid machine learning driven Raman spectrum nonlinear generation system in the embodiment of the present invention is composed of multiple key modules, including a linear combination module, a data enhancement module, a generator, a discriminator, a characteristic peak denoising module, and a peak alignment correction module. Among them, the linear combination module performs weighted calculations on the input spectra of each pure component according to their concentration ratio. The data enhancement module is used to expand the training data set, the generator is responsible for generating high-quality spectral data, and the discriminator is used to distinguish between generated data and real data. The characteristic peak denoising module eliminates noise interference in the spectrum, and the peak alignment correction module ensures that the peak position of the generated spectrum is consistent with the real spectrum.

[0144] Specifically, the linear combination module performs weighted calculation on the input spectra of each pure component according to their concentration ratio based on the excitation principle, and then extracts the key spectral features of each component through the characteristic peak recognition algorithm, and performs linear superposition to generate preliminary mixture spectral data.

[0145] Furthermore, the data enhancement module uses enhancement techniques such as controllable noise injection, random spectral masking, and data mirror inversion on the preliminary combined data to improve the diversity of training samples, enhance the generator's robustness to noise and its ability to complete missing data.

[0146] Furthermore, the generator uses a deep convolutional neural network architecture to adaptively learn the enhanced diverse training data, capturing the complex nonlinear interactions between the components in the mixture and outputting more realistic mixture spectral data.

[0147] Furthermore, the discriminator performs multi-level feature extraction and comparative analysis on the input real spectrum and the generated spectrum to identify the authenticity of the data and continuously drive the generator to optimize its network parameters, gradually improving the quality of the generated spectrum.

[0148] Furthermore, the characteristic peak denoising module integrates chemical prior knowledge, intelligently matches the characteristic peak position distribution of each component to the generated data, accurately identifies and deletes false peaks that do not conform to the vibration characteristics of chemical bonds, and simultaneously completes noise reduction processing across the entire spectrum.

[0149] Furthermore, the peak alignment correction module constructs a concentration feedback correction mechanism, inputs the denoised generated spectrum into the quantitative analysis module, calculates the concentration value of each component and compares it with the target concentration to generate a gradient guidance signal. This gradient signal is used to intelligently adjust the characteristic peak intensity, focusing on correcting the peak intensity areas with large deviations to generate a more realistic Raman spectrum of the mixture.

[0150] The present invention overcomes the limitations of traditional instrument acquisition and the technical constraints of existing GANs in generating mixture spectra. By integrating deep neural networks with physical and chemical constraints, the method achieves Raman spectral data generation for mixtures. First, a deep learning-based feature extraction generator and authenticity discriminator are designed. Compared to traditional linear dimensionality reduction methods such as principal component analysis (PCA), these detectors can more effectively capture the nonlinear characteristics of Raman spectra. Second, a characteristic peak denoising module is designed to avoid the generation of redundant noise peaks outside the characteristic peaks, ensuring that the Raman peaks in the generated spectra conform to the positional distribution of chemical bonds. Finally, the characteristic peak denoising module utilizes prior knowledge of chemical bond vibrational frequencies and combines it with a spectral feature recognition algorithm to ensure that the peak positions in the generated spectra strictly conform to the vibrational distribution of chemical bonds, eliminating the generation of spurious peaks and improving the rationality of the generated spectra. Finally, a peak alignment correction module is designed. By introducing a concentration-intensity mapping function, the intensity of the characteristic peaks in the generated spectra is intelligently corrected, enabling direct use in qualitative and quantitative analysis of mixtures, significantly enhancing the practical value of the generated spectra.

[0151] The proposed nonlinear Raman spectral generation method is compared with a common linear generation method. The experimental setup is as follows: Spectral data for each pure component and its corresponding concentration ratio are input into the IDE. In this example, a common ethanol-methanol mixture with a concentration ratio of 1:1 (organic-inorganic mixtures and complex multi-component systems are also acceptable) is used as the test sample. The system invokes the proposed generation model to output a high-fidelity nonlinear mixture spectrum. For comparison, based on the same input data, the spectra of each pure component are directly weighted and summed according to their concentration ratios to generate an ideal linear superposition spectrum.

[0152] The spectral data of real mixtures are as follows Figure 5 As shown, the comparison between the data generated by the nonlinear model and the real data is shown in Figure 6 and Figure 7 As shown, the comparison between the data generated by the linear model and the real data is as follows Figure 8 and Figure 9As shown, it can be seen that the peak values ​​of the spectral data generated by the linear model are quite different, and there are more erroneous peaks, while the peak values ​​of the spectral data generated by the nonlinear model are relatively small, there are fewer erroneous peaks, and most of the characteristic peaks are close to the real spectrum, showing a more ideal nonlinearity.

[0153] By calculating the cosine similarity between the generated spectrum and the true spectrum, we can get Figure 10 The comparison results are shown in Figure 2. Data analysis shows that the similarity between the spectral data output by the nonlinear generation model and the true spectrum reaches 96.77%, which is significantly higher than the 90.01% of the linear superposition model.

[0154] Through the quantitative comparative analysis of the generated spectrum and the real spectrum, it is found that in terms of the accuracy of the characteristic peak position, Figure 11 As shown in Figure 2, the nonlinear generation model and the linear superposition model perform similarly, with the average position error of both models being less than 1.00%. However, the nonlinear model exhibits a significant advantage in terms of characteristic peak intensity accuracy, as shown in Figure 2. Figure 12 As shown in the figure, its maximum intensity error is 4.00% (excluding lost peaks), the maximum error is 12.00% when considering the loss of low-intensity peaks, and the average error is about 4.00% (a total of one low-intensity characteristic peak is lost). In comparison, the maximum intensity error of the linear model reaches 34.00%, and the average error is 12.16% (a total of one low-intensity characteristic peak is lost).

[0155] The present invention also provides a hybrid machine learning-driven Raman spectrum nonlinear generation device, which includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of the hybrid machine learning-driven Raman spectrum nonlinear generation method in the above-mentioned embodiments.

[0156] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to execute the steps of a hybrid machine learning-driven Raman spectrum nonlinear generation method.

[0157] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0158] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0159] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hybrid machine learning-driven Raman spectroscopy nonlinear generation method, characterized in that: Hybrid machine learning-driven Raman spectroscopy nonlinear generation method includes: The Raman spectra of each pure component are weighted according to their concentration ratio, characteristic peak information is extracted and a preliminary mixture spectrum is generated; Perform controllable noise injection, random spectrum masking, data mirror inversion or spectrum fragment shifting operations based on the preliminary mixture spectrum to generate diversified enhanced data; The enhanced data is input into the generator, which captures the nonlinear interaction characteristics of the mixture spectrum through a deep convolutional neural network and a long short-term memory network architecture to generate a predicted spectrum; The discriminator performs a multi-level feature comparison between the predicted spectrum and the true spectrum, and outputs the discrimination result to drive the generator to optimize the parameters and obtain the generated spectrum; The generated spectrum is subjected to wavelet transform baseline correction and matched with the standard spectrum library based on the chemical bond vibration frequency, the false peaks are deleted and the characteristic peaks are reconstructed to obtain the denoised generated spectrum; The concentration error is obtained based on the denoised spectrum, and the characteristic peak intensity is gradient corrected using the simulated annealing algorithm to output physically reasonable Raman spectrum data of the mixture.

2. The hybrid machine learning driven Raman spectrum nonlinear generation method according to claim 1, characterized in that include: According to the input pure component name and its corresponding concentration ratio, a structured data set containing the spectral data and concentration parameters of each component is constructed; The peak-finding algorithm is used to extract the characteristic peak parameters of the Raman spectrum of each pure component. The characteristic peak parameters include peak position, peak intensity and half-peak width. The characteristic peak intensity is weighted based on the concentration ratio to generate weighted characteristic peak intensity data for each component; The weighted characteristic peak intensity data are combined in wavenumber order to generate a preliminary mixture spectrum containing the characteristic peaks of all components.

3. The hybrid machine learning driven Raman spectrum nonlinear generation method according to claim 1, characterized in that include: Performing a controllable noise injection operation on the preliminary mixture spectrum to generate a noise-enhanced spectrum; Performing random spectral masking operation on the preliminary mixture spectrum to generate a local masking enhanced spectrum; Performing a data mirror inversion operation on the preliminary mixture spectrum to generate an inverse enhanced spectrum; performing a spectral fragment shift operation on the preliminary mixture spectrum to generate a shift-enhanced spectrum; At least one of a noise-containing enhancement spectrum, a local masking enhancement spectrum, a reverse enhancement spectrum, or a shifted enhancement spectrum is selected as the diversified enhancement data.

4. The hybrid machine learning driven Raman spectrum nonlinear generation method according to claim 3, characterized in that The controllable noise injection operation is performed on the preliminary mixture spectrum to generate a noise-enhanced spectrum, comprising: Random noise is generated within a preset noise intensity range and superimposed onto the intensity data of the preliminary mixture spectrum.

5. The hybrid machine learning driven Raman spectrum nonlinear generation method according to claim 1, characterized in that include: Perform multi-scale feature extraction and time series modeling on the enhanced data through the encoding layer to generate fusion features; The fusion features are spectrally reconstructed through the decoding layer to generate multi-channel prediction data; The multi-channel prediction data is integrated and nonlinearly mapped through the fully connected layer to generate a single-channel prediction spectrum.

6. The hybrid machine learning driven Raman spectrum nonlinear generation method according to claim 5, characterized in that The multi-scale feature extraction and time series modeling of the enhanced data are performed through the coding layer to generate fusion features, including: A one-dimensional convolutional layer is used to extract local multi-scale features of the spectrum; The temporal dependency of spectral wavenumbers is modeled by long short-term memory networks to capture global context features. The local multi-scale features are fused with the global context features to generate fused features.

7. The hybrid machine learning driven Raman spectrum nonlinear generation method according to claim 1, characterized in that include: The predicted spectrum and the real spectrum are analyzed locally and globally through a multi-level feature extraction network to generate a discriminant feature vector. Calculate the true and false probability based on the discriminant feature vector and generate the discriminant loss function; The gradient of the discriminant loss function is transferred to the generator through the back-propagation algorithm to drive the optimization of the generator parameters; The predicted spectrum output by the optimized generator is used as the final generated spectrum.

8. The hybrid machine learning driven Raman spectrum nonlinear generation method according to claim 7, characterized in that: The method of performing local and global feature analysis on the predicted spectrum and the true spectrum through a multi-level feature extraction network to generate a discriminant feature vector includes: A multi-layer one-dimensional convolutional network is used to extract the local characteristic peak shape and intensity distribution of the spectrum; The global statistical features of the spectrum are extracted through the fully connected layer, including peak offset, half-peak width mean and signal-to-noise ratio; The local features and global features are concatenated into a discriminant feature vector.

9. The hybrid machine learning driven Raman spectrum nonlinear generation method according to claim 1, characterized in that: include: Performing wavelet transform baseline correction on the generated spectrum to generate a baseline-corrected spectrum; Baseline-corrected spectra were matched for chemical bond vibration frequencies to generate false peak marker data; Peak deletion and reconstruction of the baseline-corrected spectrum are performed based on the false peak marker data to generate a denoised generated spectrum; Output is a denoised spectrum that fully preserves the physical plausibility characteristics.

10. The hybrid machine learning driven Raman spectrum nonlinear generation method according to claim 9, characterized in that: The performing wavelet transform baseline correction on the generated spectrum to generate a baseline-corrected spectrum includes: The low-frequency baseline component of the generated spectrum was extracted by wavelet decomposition; The low-frequency baseline component was subtracted from the raw generated spectrum to eliminate baseline drift and background noise.

11. The hybrid machine learning driven Raman spectrum nonlinear generation method according to claim 1, characterized in that: include: Quantitative analysis of the denoised spectra was performed to generate component concentration errors; The objective function of the simulated annealing algorithm is constructed based on the concentration error vector to generate the initial correction parameters; Perform gradient correction on characteristic peak intensity through iterative perturbation and optimization to generate optimized spectral data; The final optimized spectral data are output as physically reasonable Raman spectral data of the mixture.

12. The hybrid machine learning driven Raman spectrum nonlinear generation method according to claim 11, characterized in that: The quantitative analysis of the de-noised spectrum to generate component concentration errors includes: The intensity data of each characteristic peak in the denoised spectrum is extracted through the quantitative analysis module; Calculate the predicted concentration value of each component based on the preset concentration-intensity mapping function; The concentration predictions are compared with the initial target concentrations to generate a concentration error vector.

13. The hybrid machine learning driven Raman spectrum nonlinear generation method according to claim 11, characterized in that: The step of performing gradient correction on the characteristic peak intensity through iterative perturbation and optimization to generate optimized spectral data includes: In each iteration, the characteristic peak interval is randomly selected and intensity perturbation is applied to generate the perturbed spectrum; Recalculate the concentration error of the perturbed spectrum and update the objective function value; Accept or reject the perturbation according to the Metropolis criterion, giving priority to retaining the correction that reduces the error; Lower the temperature parameter and reduce the disturbance range until the maximum number of iterations is met or the objective function value is lower than the preset error threshold.

Images