Neural network training method and spectrum reconstruction method based on neural network

Abstract

The invention provides a neural network training method and a neural network-based spectrum reconstruction method, and the method comprises the steps: obtaining a training sample which comprises a plurality of theoretical spectrums and the light response intensity corresponding to each theoretical spectrum; the training sample is input into an initial neural network, a prediction spectrum is output, and the initial neural network comprises a residual error network layer, a recurrent neural network layer, a self-attention mechanism layer and a full connection layer. And determining a loss function value based on the predicted spectrum and a theoretical spectrum corresponding to the light response intensity. And training the initial neural network based on the loss function value until a training cut-off condition is met to obtain a trained neural network, so that spectrum reconstruction can be carried out by using the trained neural network subsequently. The neural network is trained by using the theoretical spectrum and the actual light response intensity corresponding to the theoretical spectrum, so that spectrum reconstruction is performed by using the neural network, and the spectrum reconstruction efficiency and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the technical field of spectral analysis, and specifically relates to a method for training a neural network and a spectral reconstruction method based on the neural network. Background Art

[0002] As a key instrument for obtaining the composition and physical properties of substances by analyzing spectral information, spectrometers have been widely used in fields such as scientific research, industrial production, environmental monitoring, and medical diagnosis. Most of the existing spectrometers belong to large-scale precision instruments, and their large volume due to the presence of spectral components limits the application scenarios of spectrometers. Some detectors with a photographing function can measure the actual optical signal, but cannot directly obtain spectral information. To solve the spectral reconstruction problem, a variety of spectral reconstruction algorithms have emerged to restore near-real spectral information.

[0003] However, some existing spectral reconstruction algorithms rely on an iterative optimization process (such as L1 norm optimization), with a complex calculation process and low processing efficiency. In the presence of external noise interference and other situations, the accuracy of the traditional spectral reconstruction algorithm for reconstructing spectra decreases significantly. Summary of the Invention

[0004] In view of this, the present application is committed to providing a method for training a neural network and a spectral reconstruction method based on the neural network to improve the efficiency and accuracy of spectral reconstruction.

[0005] In a first aspect, the present application provides a method for training a neural network, the method comprising:

[0006] Obtaining training samples, the training samples including a plurality of theoretical spectra and the light response intensity corresponding to each theoretical spectrum, the light response intensity being determined based on the theoretical spectrum and a calibration function;

[0007] Inputting the training samples into an initial neural network to output a predicted spectrum corresponding to the light response intensity, the initial neural network including a residual network layer, a recurrent neural network layer, a self-attention mechanism layer, and a fully connected layer;

[0008] Determining a loss function value based on the predicted spectrum and the theoretical spectrum corresponding to the light response intensity;

[0009] Training the initial neural network based on the loss function value until a training termination condition is satisfied to obtain a trained neural network for spectral reconstruction using the neural network.

[0010] In a possible implementation, the determining a loss function value based on the predicted spectrum and the theoretical spectrum corresponding to the light response intensity includes:

[0011] Determine a wavelength error based on the difference between the wavelengths in the predicted spectrum and the corresponding wavelengths in the theoretical spectrum;

[0012] Determine the weight coefficient corresponding to each wavelength in the theoretical spectrum;

[0013] Determine the loss function value based on the wavelength error and the weight coefficient.

[0014] In a possible implementation manner, the determining the weight coefficient corresponding to each wavelength in the theoretical spectrum includes:

[0015] Judge whether the light intensity corresponding to each wavelength in the theoretical spectrum is a peak;

[0016] When the light intensity corresponding to the wavelength is a peak, determine that the weight coefficient corresponding to the wavelength is the first weight; when the light intensity corresponding to the wavelength is not a peak, determine that the weight coefficient corresponding to the wavelength is the second weight, and the first weight is greater than the second weight.

[0017] In a possible implementation manner, the determining the loss function value based on the wavelength error and the weight coefficient includes:

[0018] Calculate the product of the wavelength error and the weight coefficient of the wavelength corresponding to the wavelength error;

[0019] Calculate the average value of the products corresponding to each wavelength to obtain the loss function value.

[0020] In a possible implementation manner, the recurrent neural network layer includes: a gated recurrent unit GRU, and the inputting the training sample into the initial neural network and outputting the predicted spectrum corresponding to the light response intensity includes:

[0021] Input the light response intensity into the residual network layer to obtain a first extracted feature;

[0022] Input the first extracted feature into the GRU to obtain a second extracted feature;

[0023] Input the second extracted feature into the self-attention mechanism layer to obtain a third extracted feature;

[0024] Input the third extracted feature into the fully connected layer to output the predicted spectrum.

[0025] In a possible implementation manner, the determination process of the calibration function includes:

[0026] Calibrate the LED standard light source using a spectrometer to obtain a spectrum;

[0027] Use an image detector to photograph the LED standard light source to obtain the actual light response intensity;

[0028] Based on the spectrum and the actual light response intensity, determine the calibration function.

[0029] In a possible implementation, the obtaining of the training samples, where the training samples include a plurality of theoretical spectra and the light response intensity corresponding to each theoretical spectrum, includes:

[0030] Use a Gaussian line shape function to generate the theoretical spectra;

[0031] Based on the theoretical spectra and the calibration function, determine the light response intensity.

[0032] In a second aspect, the present application provides a spectral reconstruction method based on a neural network, the method including:

[0033] Obtain the actual light response intensity of the object to be measured;

[0034] Input the actual light response intensity of the object to be measured into the neural network to obtain the reconstructed spectrum of the object to be measured, where the neural network is obtained by the method for training a neural network according to any one of the implementations in the above first aspect.

[0035] In a possible implementation, the neural network includes a residual network layer, a GRU, a self-attention mechanism layer, and a fully connected layer. The inputting the actual light response intensity of the object to be measured into the neural network to obtain the reconstructed spectrum of the object to be measured includes:

[0036] Input the actual light response intensity into the residual network layer to obtain a first spectral feature;

[0037] Input the first spectral feature into the GRU to obtain a second spectral feature;

[0038] Input the second spectral feature into the self-attention mechanism layer to obtain a third spectral feature;

[0039] Input the third spectral feature into the fully connected layer to output the reconstructed spectrum.

[0040] In a third aspect, the present application provides a device for training a neural network, the device including:

[0041] A sample acquisition unit for obtaining training samples, where the training samples include a plurality of theoretical spectra and the light response intensity corresponding to each theoretical spectrum, and the light response intensity is determined based on the theoretical spectra and the calibration function;

[0042] A prediction unit, configured to input the training samples into an initial neural network and output a predicted spectrum corresponding to the light response intensity, where the initial neural network includes a residual network layer, a recurrent neural network layer, a self-attention mechanism layer, and a fully connected layer;

[0043] A determination unit, configured to determine a loss function value based on the predicted spectrum and the theoretical spectrum corresponding to the light response intensity;

[0044] A training unit, configured to train the initial neural network based on the loss function value until a training termination condition is satisfied, to obtain a trained neural network, so as to perform spectrum reconstruction using the neural network.

[0045] In a fourth aspect, the present application provides a spectrum reconstruction device based on a neural network, where the device includes:

[0046] An acquisition unit, configured to acquire the actual light response intensity of a to-be-detected object;

[0047] A reconstruction unit, configured to input the actual light response intensity of the to-be-detected object into the neural network to obtain a reconstructed spectrum of the to-be-detected object, where the neural network is obtained by using the method for training a neural network according to any one of the implementation manners of the first aspect above.

[0048] In a fifth aspect, the present application provides an electronic device, where the device includes: a memory and a processor;

[0049] The memory is used to store relevant program codes;

[0050] The processor is used to call the program codes to execute the method for training a neural network according to any one of the implementation manners of the first aspect above or the spectrum reconstruction method based on a neural network according to any one of the implementation manners of the second aspect above.

[0051] In a sixth aspect, the present application provides a computer-readable storage medium, where the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method for training a neural network according to any one of the implementation manners of the first aspect above or the spectrum reconstruction method based on a neural network according to any one of the implementation manners of the second aspect above.

[0052] In a seventh aspect, the present application provides a computer program product, where the computer program product includes computer programs / instructions, and when the computer programs / instructions are executed by a processor, the method for training a neural network according to any one of the implementation manners of the first aspect above or the spectrum reconstruction method based on a neural network according to any one of the implementation manners of the second aspect above is implemented.

[0053] In the above implementation manner of the present application, in order to train a neural network for spectral reconstruction, it is first necessary to obtain training samples. Among them, the training samples include a plurality of theoretical spectra and the light response intensity corresponding to each theoretical spectrum, and the light response intensity is determined based on the theoretical spectrum and the calibration function. The training samples are input into the initial neural network, and the predicted spectrum corresponding to the light response intensity is output. Among them, the initial neural network includes a residual network layer, a recurrent neural network layer, a self-attention mechanism layer, and a fully connected layer. Based on the predicted spectrum and the theoretical spectrum corresponding to the light response intensity, the loss function value is determined. The initial neural network is trained based on the loss function value until the training cutoff condition is met, and a trained neural network is obtained for subsequent spectral reconstruction using the trained neural network. Through the method provided by the present application, a neural network is trained using the theoretical spectrum and the actual light response intensity corresponding to the theoretical spectrum, so as to use the neural network for spectral reconstruction, improving the efficiency and accuracy of the reconstructed spectrum. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments provided in the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0055] Figure 1 It is a flowchart of a method for training a neural network provided by an embodiment of the present application.

[0056] Figure 2 It is a schematic structural diagram of an initial neural network provided by an embodiment of the present application.

[0057] Figure 3 It is a flowchart of a spectral reconstruction method based on a neural network provided by an embodiment of the present application.

[0058] Figure 4a It is a schematic diagram of the result of spectral reconstruction based on the generalized inverse algorithm provided by an embodiment of the present application.

[0059] Figure 4b It is a schematic diagram of the result of spectral reconstruction based on a neural network provided by an embodiment of the present application.

[0060] Figure 5 It is a schematic diagram of a device for training a neural network provided by an embodiment of the present application.

[0061] Figure 6 It is a schematic diagram of a spectral reconstruction device based on a neural network provided by an embodiment of the present application.

[0062] Figure 7Schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0063] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. The described embodiments are only exemplary embodiments of the present application, not all implementation manners. Those skilled in the art can obtain other embodiments without creative work in combination with the embodiments of the present application, and these embodiments are also within the protection scope of the present application.

[0064] As a key instrument for obtaining the composition and physical properties of substances by analyzing spectral information, spectrometers have been widely used in fields such as scientific research, industrial production, environmental monitoring, and medical diagnosis. Most of the existing spectrometers belong to large-scale precision instruments. Due to the existence of spectral splitting elements, their volumes are large, which limits the application scenarios of spectrometers. Some detectors with a photographing function can measure the actual optical signal, but cannot directly obtain spectral information. To solve the spectral reconstruction problem, a variety of spectral reconstruction algorithms have emerged to restore spectral information close to the real one.

[0065] However, some existing spectral reconstruction algorithms rely on an iterative optimization process (such as L1 norm optimization), with a complex calculation process and low processing efficiency. Under noise interference or external environmental changes, the accuracy of spectral reconstruction by traditional spectral reconstruction algorithms drops significantly.

[0066] Based on this, an embodiment of the present application provides a method for training a neural network to improve the efficiency and accuracy of spectral reconstruction. Specifically, when implementing, to train the neural network for spectral reconstruction, training samples need to be obtained first. Among them, the training samples include multiple theoretical spectra and the light response intensity corresponding to each theoretical spectrum, and the light response intensity is determined based on the theoretical spectrum and the calibration function. The training samples are input into the initial neural network, and a predicted spectrum corresponding to the light response intensity is output. Among them, the initial neural network includes a residual network layer, a recurrent neural network layer, a self-attention mechanism layer, and a fully connected layer. Based on the predicted spectrum and the theoretical spectrum corresponding to the light response intensity, the loss function value is determined. The initial neural network is trained based on the loss function value until the training cut-off condition is met, and a trained neural network is obtained for subsequent spectral reconstruction using the trained neural network. Through the method provided by the present application, the neural network is trained using the theoretical spectrum and the actual light response intensity corresponding to the theoretical spectrum, so as to use the neural network for spectral reconstruction and improve the efficiency and accuracy of spectral reconstruction.

[0067] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the following will be specifically introduced in conjunction with the accompanying drawings in the embodiments.

[0068] SeeFigure 1 As shown in the figure, it is a flowchart of a method for training a neural network provided by an embodiment of the present application.

[0069] This method can be executed by a processing device. Optionally, the processing device can be a terminal or a server. Among them, the terminal includes but is not limited to a desktop computer, a laptop computer, a tablet computer, and a smart phone. The server can be a single server or a server cluster formed by multiple servers.

[0070] This method may include the following steps:

[0071] S101: Obtain training samples, where the training samples include multiple theoretical spectra and the light response intensity corresponding to each theoretical spectrum.

[0072] Among them, the theoretical spectrum can be understood as the spectrum of light emitted by a light source without undergoing processes such as dispersion and absorption. When using an image detector to capture the light emitted by a light source, the intensity corresponding to light of different wavelengths detected in the image detector, that is, the light response loudness. The initial neural network can be trained with the light response intensity as the sample input and the theoretical spectrum as the sample output (i.e., the true value).

[0073] In a possible implementation manner, during the process of determining the training samples, in order to increase the diversity of the samples, improve the accuracy of training the initial neural network, and simplify the process of obtaining the theoretical spectra of different light sources, multiple different theoretical spectra can be generated using a Gaussian line shape function. After obtaining multiple theoretical spectra, based on the theoretical spectra and the calibration function, the light response intensity corresponding to the theoretical spectra can be determined. Among them, the calibration function can represent the correspondence between the theoretical spectrum and the light response intensity, and can be obtained in advance through experimental measurement.

[0074] The process of determining the calibration function will be introduced below in combination with an application scenario.

[0075] In specific implementation, a spectrometer can be used to calibrate the LED standard light source to obtain the spectrum of the LED standard light source. For example, a constant voltage or constant current power supply can be used to supply power to the LED standard light source to ensure a stable light-emitting state of the LED. The optical fiber probe of the spectrometer is closely attached to the surface of the LED lamp bead, and stray light affecting the LED is minimized as much as possible, thereby improving the purity and accuracy of the measured optical signal. The spectrometer is used to calibrate the LED standard light source to obtain the spectrum corresponding to the LED standard light source. In an actual test scenario, an image detector can be used to photograph the LED standard light source to obtain the actual light response intensity. When the image detector captures the light emitted by the LED standard light source, the photosensitive elements in the image detector can detect the intensities of light of different wavelengths, that is, the actual light response intensity is obtained. Optionally, the image detector may include: a charge-coupled device (CCD), a complementary metal oxide semiconductor (CMOS), and the like.

[0076] After determining the spectrum and the actual light response intensity of the LED standard light source, a calibration function can be determined based on the spectrum and the actual light response intensity to represent the relationship between the spectrum and the actual light response intensity. For example, the light response intensity and the spectrum can be expressed in the form of a matrix, consisting of components corresponding to multiple wavelengths. Then, the calibration function can also be expressed in the form of a matrix, that is, the observation matrix. The relationship between the spectrum, the actual light response intensity, and the observation matrix can be expressed as: light response intensity = spectrum * observation matrix.

[0077] It should be noted that the method for determining the calibration function provided in the above embodiments is only an exemplary illustration and is not limited to the above form. Other methods that can achieve similar effects also fall within the protection scope of this application.

[0078] S102: Input the training samples into the initial neural network to output the predicted spectrum corresponding to the light response intensity.

[0079] The initial neural network includes a residual network layer, a recurrent neural network layer, a self-attention mechanism layer, and a fully connected layer. The residual network layer includes a residual network, which can be composed of multiple residual blocks. Each residual block contains a shortcut path and a main path. After the light response intensity is input into the residual network, the shortcut path can directly pass the input of the residual block to the output, while the main path processes the input of the residual block through a series of convolutional layers to extract features from the input light response intensity. Finally, the outputs of the two paths are superimposed to form the output of the residual block. Using the residual network to extract the features of the light response intensity can not only enhance the expression ability of the network but also alleviate the problem of gradient disappearance in the training of the neural network through the short-circuit connection.

[0080] Due to the significant smoothness and correlation of spectral data in the wavelength dimension, this characteristic makes the signals of adjacent wavelengths show a certain degree of continuous change. Therefore, spectral data can be processed as sequential data. The recurrent neural network layer includes a recurrent neural network. Through its recurrent structure, the recurrent neural network can use the hidden state of the previous time step to calculate the hidden state of the current time step at each time step, thereby capturing the temporal dependencies in the sequential data. And the output of the previous step can be used as part of the input of the next step. Therefore, it can learn the context information of the sequential data, extract more accurate feature information, and thus help the neural network make more accurate prediction results.

[0081] In addition, in order to improve the attention ability of the neural network, a self-attention mechanism layer is added. The self-attention mechanism can dynamically focus on the information at different positions in the input data, thereby capturing the internal dependencies in the data. By dynamically identifying the important features in the input data, the prediction accuracy of the neural network is improved. Finally, the predicted result is output through the fully connected layer of the initial neural network, that is, the predicted spectrum corresponding to the optical response intensity is output.

[0082] In one possible implementation, the recurrent neural network can be a gated recurrent unit (GRU). By introducing a gating mechanism, GRU can dynamically adjust the degree of information retention, ensure that the local characteristics of short-term wavelengths are accurately characterized, and at the same time retain the global dependencies between long-distance wavelengths. And GRU can better control the gradient propagation, alleviate the problems of gradient vanishing or gradient explosion, and improve the stability of the training process.

[0083] In specific implementation, the optical response intensity is input into the residual network layer to obtain the first extracted feature. The first extracted feature is input into the GRU to obtain the second extracted feature. The second extracted feature is input into the self-attention mechanism layer to obtain the third extracted feature. The third extracted feature is input into the fully connected layer to output the predicted spectrum.

[0084] In one possible implementation, in order to reduce the difficulty of data processing during the training process, a pooling layer can also be added after the residual network layer. The pooling layer is used to perform dimensionality reduction on the features extracted by the residual network layer, simplifying the subsequent processing process.

[0085] See Figure 2As shown in the figure, it is a schematic structural diagram of an initial neural network provided by an embodiment of the present application. The residual network layer of the initial neural network includes two residual blocks, namely residual block 1 and residual block 2, which are used to extract features. Residual block 2 is connected to a pooling layer for feature dimensionality reduction. The recurrent neural network layer includes two GRUs, namely GRU1 and GRU2, which can focus on the temporal dependencies in spectral data and extract features more accurately. Then it is connected to a self-attention mechanism layer to dynamically identify important features in the spectrum based on the self-attention mechanism. Finally, the prediction result, that is, the predicted spectrum corresponding to the light response intensity, is output by a fully connected layer.

[0086] S103: Determine the loss function value based on the predicted spectrum and the theoretical spectrum corresponding to the light response intensity.

[0087] After the initial neural network outputs the predicted spectrum, since the theoretical spectrum corresponding to the light response intensity in the training samples is the sample true value, the loss function value can be determined based on the predicted spectrum and the theoretical spectrum corresponding to the light response intensity. Among them, the loss function value can represent the error between the predicted spectrum and the theoretical spectrum. When the loss function value is larger, that is, the error between the predicted spectrum and the theoretical spectrum is larger, the accuracy of the initial neural network is lower, and then the initial neural network needs to be trained based on the loss function value.

[0088] In a possible implementation manner, the loss function value can be determined in the following way. Since the spectrum includes multiple wavelengths, the wavelength error can be determined based on the difference between the wavelengths in the predicted spectrum and the corresponding wavelengths in the theoretical spectrum. That is, for each wavelength in the predicted spectrum, the corresponding wavelength in the theoretical spectrum can be found, so that the error between the two wavelengths can be determined. Among them, the wavelength error can be expressed as the error between the light intensities corresponding to the two wavelengths. Optionally, the square of the difference between the wavelength in the predicted spectrum and the corresponding wavelength in the theoretical spectrum can be calculated as the wavelength error between the two wavelengths.

[0089] For each wavelength in the theoretical spectrum, the weight coefficient corresponding to each wavelength in the theoretical spectrum can be determined. Thus, the loss function value can be determined based on the wavelength error and the weight coefficient of the wavelength corresponding to the wavelength error.

[0090] In specific implementation, the weight coefficient corresponding to each wavelength can be determined in the following manner, and it is judged whether the light intensity corresponding to each wavelength in the theoretical spectrum is a peak. When the light intensity corresponding to this wavelength is a peak, it is determined that the weight coefficient corresponding to this wavelength is the first weight. When the light intensity corresponding to this wavelength is not a peak, it is determined that the weight coefficient corresponding to this wavelength is the second weight, and the first weight is greater than the second weight. That is, it can be determined that the weight coefficient corresponding to the peak in each wavelength is relatively larger, so that the initial neural network can pay more attention to the characteristics of the peak in the wavelength and improve the prediction accuracy of the initial neural network. It should be noted that the embodiments of the present application do not limit the specific values of the first weight and the second weight. For example, the first weight can be set to 2 and the second weight can be set to 1.

[0091] In a possible implementation manner, the light intensity corresponding to the wavelength in the theoretical spectrum can be determined whether it is a peak in the following manner. For any wavelength in the theoretical spectrum, the second-order difference corresponding to this wavelength can be calculated. For example, taking target i to represent any wavelength in the theoretical spectrum, the second-order difference corresponding to this wavelength can be expressed as (target i+2 ―target i+1 )―(target i+1 ―target i ). When the second-order difference is less than 0, it indicates that the light intensity corresponding to this wavelength is a peak; otherwise, it indicates that the light intensity corresponding to this wavelength is not a peak.

[0092] After determining the weight coefficient corresponding to each wavelength in the spectrum, the product of each wavelength error and the weight coefficient of the wavelength corresponding to this wavelength error can be calculated. Calculate the average value of the products corresponding to each wavelength to obtain the loss function value. Specifically, for the first wavelength in the theoretical spectrum, determine the second wavelength corresponding to this first wavelength in the predicted spectrum, where the first wavelength can represent any wavelength in the theoretical spectrum. Then calculate the wavelength error between the first wavelength and the second wavelength, and calculate the product of the wavelength error and the weight coefficient corresponding to the first wavelength, that is, each wavelength corresponds to a product. Calculate the average value of the products corresponding to each wavelength as the loss function value.

[0093] Specifically, taking target i to represent the wavelength of the theoretical spectrum, and taking output i to represent the wavelength of the predicted spectrum, then the wavelength error L i can be expressed as L i =(output i ―target i ) 2 . Taking w i to represent the weight coefficient corresponding to the wavelength, then the loss function value L can be expressed as Among them, n represents the number of wavelengths included in the theoretical spectrum.

[0094] S104: Train the initial neural network based on the loss function value until the training termination condition is satisfied, and obtain the trained neural network.

[0095] After determining the loss function value of the predicted spectrum and the theoretical spectrum, train the initial neural network based on this loss function value. Specifically, when this loss function value is greater than or equal to the preset threshold, it indicates that the error between the predicted spectrum and the theoretical spectrum is large, and the accuracy of the initial neural network does not meet the requirements. Then, it is necessary to adjust the parameters of the initial neural network. Then, re-execute the process of inputting the training samples into the neural network with adjusted parameters and the subsequent training process until the training termination condition is satisfied, and stop the training process, thereby obtaining the trained neural network.

[0096] Optionally, the training termination condition may include that the loss function value is less than the preset threshold, or the number of iterative training reaches the preset number. That is, after adjusting the parameters, re-determine the loss function value. When this loss function value is less than the preset threshold, the training process can be stopped. Or, during the iterative training process, the loss function value is always greater than or equal to the preset threshold, but the number of iterative training reaches the preset number. At this time, the training process can also be stopped to obtain the trained neural network. After training the neural network, the neural network can be used for spectral reconstruction subsequently.

[0097] By using the method provided in the embodiments of the present application, the neural network is trained by using the theoretical spectrum and the actual light response intensity corresponding to the theoretical spectrum, and a recurrent neural network layer is added to learn the time dependence relationship in the spectrum, further improving the efficiency and accuracy of the neural network for spectral reconstruction.

[0098] Based on this, the embodiments of the present application also provide a spectral reconstruction method based on a neural network. Refer to Figure 3 As shown, it is a flowchart of a spectral reconstruction method based on a neural network provided by the embodiments of the present application.

[0099] Optionally, this method can be executed by a spectral reconstruction device. Among them, the spectral reconstruction device and the image processing device can be the same device or different devices. When the spectral reconstruction device and the processing device belong to different devices, the spectral reconstruction device can obtain the trained detection neural network from the processing device and store it in the spectral reconstruction device, and then the neural network can be directly called from the storage area subsequently.

[0100] This method may include the following steps:

[0101] S301: Obtain the actual light response intensity of the object to be measured.

[0102] For example, the actual light response intensity can be obtained by the detector photographing the object to be measured.

[0103] S302: Input the actual light response intensity of the object to be measured into the neural network to obtain the reconstructed spectrum of the object to be measured.

[0104] Wherein, the neural network is obtained based on the method for training the neural network described in the above method embodiment.

[0105] Through the method provided by the embodiments of the present application, using the pre-trained neural network for spectral reconstruction, a recurrent neural network layer is also added to learn the temporal dependence relationship in the spectrum, which can improve the efficiency and accuracy of the neural network for reconstructing the spectrum.

[0106] In a possible implementation manner, the neural network includes a residual network layer, a GRU, a self-attention mechanism layer, and a fully connected layer. Specifically, input the actual light response intensity into the residual network layer to obtain the first spectral feature. Input the first spectral feature into the GRU to obtain the second spectral feature. Input the second spectral feature into the self-attention mechanism layer to obtain the third spectral feature. Input the third spectral feature into the fully connected layer to output the reconstructed spectrum.

[0107] In a possible implementation manner, in order to reduce the difficulty of data processing, a pooling layer can also be added after the residual network layer, and the pooling layer is used to perform dimensionality reduction processing on the features extracted by the residual network layer to simplify the subsequent processing process.

[0108] Specifically, the neural network can include a residual network layer, a pooling layer, a GRU, a self-attention mechanism layer, and a fully connected layer. Input the actual light response intensity into the residual network layer to obtain the first spectral feature. Input the first spectral feature into the pooling layer to obtain the fourth spectral feature. Input the fourth spectral feature into the GRU to obtain the fifth spectral feature. Input the fifth spectral feature into the self-attention mechanism layer to obtain the sixth spectral feature. Input the sixth spectral feature into the fully connected layer to output the reconstructed spectrum.

[0109] Next, the results of reconstructing the spectrum using the neural network provided by the embodiments of the present application will be introduced in combination with a specific application scenario. In this application scenario, a set of samples including test light response intensity and test spectrum can be used to verify the accuracy of the neural network. The test light response intensity is used as the input, and the test spectrum is used as the true value for comparison. And spectral reconstruction using the generalized inverse algorithm is provided as a comparison. The method for spectral reconstruction using the generalized inverse algorithm can be referred to the existing technology and will not be elaborated in this application. In Figure 4a and Figure 4b the light intensity is represented in a normalized format. The dashed line represents the predicted spectrum, and the solid line represents the test spectrum.

[0110] See Figure 4a shown in the figure, which is a schematic diagram of the result of spectral reconstruction based on the generalized inverse algorithm provided by an embodiment of the present application, including the comparison result between the predicted spectrum obtained by the generalized inverse algorithm and the test spectrum. See Figure 4b shown in the figure, which is a schematic diagram of the result of spectral reconstruction based on a neural network provided by an embodiment of the present application, including the comparison result between the predicted spectrum obtained by the neural network provided by the embodiment of the present application and the test spectrum. It can be seen from the above figures that the accuracy of spectral reconstruction based on the neural network provided by the embodiment of the present application is relatively high.

[0111] Based on the above method embodiment, an embodiment of the present application further provides a device for training a neural network. See Figure 5 shown in the figure, which is a schematic diagram of a device for training a neural network provided by an embodiment of the present application.

[0112] The device 500 includes:

[0113] A sample acquisition unit 501, configured to obtain training samples, where the training samples include a plurality of theoretical spectra and the light response intensity corresponding to each theoretical spectrum, and the light response intensity is determined based on the theoretical spectrum and a calibration function;

[0114] A prediction unit 502, configured to input the training samples into an initial neural network and output a predicted spectrum corresponding to the light response intensity, where the initial neural network includes a residual network layer, a recurrent neural network layer, a self-attention mechanism layer, and a fully connected layer;

[0115] A determination unit 503, configured to determine a loss function value based on the predicted spectrum and the theoretical spectrum corresponding to the light response intensity;

[0116] A training unit 504, configured to train the initial neural network based on the loss function value until a training termination condition is met, and obtain a trained neural network for spectral reconstruction using the neural network.

[0117] In a possible implementation manner, the determination unit 503 is specifically configured to determine a wavelength error based on the difference between the wavelength in the predicted spectrum and the corresponding wavelength in the theoretical spectrum; determine a weight coefficient corresponding to each wavelength in the theoretical spectrum; and determine the loss function value based on the wavelength error and the weight coefficient.

[0118] In a possible implementation manner, the determining unit 503 is specifically configured to determine whether the light intensity corresponding to each wavelength in the theoretical spectrum is a peak value; when the light intensity corresponding to the wavelength is a peak value, determine that the weight coefficient corresponding to the wavelength is a first weight; when the light intensity corresponding to the wavelength is not a peak value, determine that the weight coefficient corresponding to the wavelength is a second weight, where the first weight is greater than the second weight.

[0119] In a possible implementation manner, the determining unit 503 is specifically configured to calculate the product of the wavelength error and the weight coefficient corresponding to the wavelength of the wavelength error; calculate the average value of the products corresponding to each wavelength to obtain the loss function value.

[0120] In a possible implementation manner, the recurrent neural network layer includes: a gated recurrent unit GRU. The predicting unit 502 is specifically configured to input the light response intensity into the residual network layer to obtain a first extracted feature; input the first extracted feature into the GRU to obtain a second extracted feature; input the second extracted feature into the self-attention mechanism layer to obtain a third extracted feature; input the third extracted feature into the fully connected layer, and output the predicted spectrum.

[0121] In a possible implementation manner, the determining process of the calibration function includes: calibrating an LED standard light source by using a spectrometer to obtain a spectrum; photographing the LED standard light source by using an image detector to obtain an actual light response intensity; determining the calibration function based on the spectrum and the actual light response intensity.

[0122] In a possible implementation manner, the sample acquisition unit 401 is specifically configured to generate the theoretical spectrum by using a Gaussian line shape function; determine the light response intensity based on the theoretical spectrum and the calibration function.

[0123] In addition, an embodiment of the present application further provides a spectral reconstruction device based on a neural network. Refer to Figure 6 as shown in Figure 6 which is a schematic diagram of a spectral reconstruction device based on a neural network provided by an embodiment of the present application.

[0124] The device 600 includes:

[0125] an acquisition unit 601, configured to acquire the actual light response intensity of the object to be measured;

[0126] a reconstruction unit 602, configured to input the actual light response intensity of the object to be measured into a neural network to obtain a reconstructed spectrum of the object to be measured, where the neural network is obtained by using the method for training a neural network described in the foregoing method embodiment.

[0127] In a possible implementation, the neural network includes a residual network layer, a GRU, a self-attention mechanism layer, and a fully connected layer. The reconstruction unit 602 is specifically configured to input the actual light response intensity into the residual network layer to obtain a first spectral feature; input the first spectral feature into the GRU to obtain a second spectral feature; input the second spectral feature into the self-attention mechanism layer to obtain a third spectral feature; and input the third spectral feature into the fully connected layer to output the reconstructed spectrum.

[0128] Based on the above method embodiments and apparatus embodiments, an embodiment of the present application further provides an electronic device. This will be introduced below with reference to the accompanying drawings.

[0129] See Figure 7 , Figure 7 which is a schematic diagram of an electronic device provided by an embodiment of the present application.

[0130] The device 700 includes: a memory 701 and a processor 702;

[0131] The memory 701 is used to store relevant program codes;

[0132] The processor 702 is used to call the program codes to execute the method for training a neural network or the spectral reconstruction method based on a neural network described in the above method embodiments.

[0133] In addition, an embodiment of the present application further provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute the method for training a neural network or the spectral reconstruction method based on a neural network described in the above method embodiments.

[0134] An embodiment of the present application further provides a computer program product, which includes a computer program / instructions, and when the computer program / instructions are executed by a processor, the method for training a neural network or the spectral reconstruction method based on a neural network described in the above method embodiments is implemented.

[0135] It should be noted that the above-mentioned computer-readable medium of the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0136] The computer program product can be written in any combination of one or more programming languages to write program code for performing the operations of the embodiments of the present application. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0137] It should be noted that the various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. In particular, for system or apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiments. The device embodiments described above are merely illustrative. The units or modules described as separate components may or may not be physically separated. The components shown as units or modules may or may not be physical modules, that is, they can be located in one place or distributed to multiple network units. Some or all of the units or modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0138] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functions, and operations that may be implemented by methods, apparatuses, devices, etc. according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0139] It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist simultaneously. Here, A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (item) of the following" or its similar expressions refer to any combination of these items, including any combination of single item (s) or plural item (s). For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0140] It should also be noted that in the present application, relative terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device that includes the said element.

[0141] The steps of the methods or algorithms described in connection with the embodiments disclosed in this application can be implemented directly in hardware, in software modules executed by a processor, or in a combination of both. The software modules can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well known in the art.

[0142] The foregoing description of the disclosed embodiments enables those skilled in the art to practice or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Thus, this application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for training a neural network, characterized in that: The method comprises: Acquire a training sample, wherein the training sample includes a plurality of theoretical spectra and a light response intensity corresponding to each theoretical spectrum, wherein the light response intensity is determined based on the theoretical spectra and a calibration function; Inputting the training sample into an initial neural network and outputting a predicted spectrum corresponding to the light response intensity, wherein the initial neural network includes a residual network layer, a recurrent neural network layer, a self-attention mechanism layer, and a fully connected layer; Determining a loss function value based on the predicted spectrum and a theoretical spectrum corresponding to the light response intensity; The initial neural network is trained based on the loss function value until a training cutoff condition is met, thereby obtaining a trained neural network so as to use the neural network to perform spectral reconstruction.

2. The method according to claim 1, characterized in that The determining of the loss function value based on the predicted spectrum and the theoretical spectrum corresponding to the light response intensity includes: Determining a wavelength error based on a difference between a wavelength in the predicted spectrum and a corresponding wavelength in the theoretical spectrum; Determining a weight coefficient corresponding to each wavelength in the theoretical spectrum; The loss function value is determined based on the wavelength error and the weight coefficient.

3. The method according to claim 2, characterized in that The determining of the weight coefficient corresponding to each wavelength in the theoretical spectrum includes: Determine whether the light intensity corresponding to each wavelength in the theoretical spectrum is a peak value; When the light intensity corresponding to the wavelength is a peak value, the weight coefficient corresponding to the wavelength is determined to be a first weight; when the light intensity corresponding to the wavelength is not a peak value, the weight coefficient corresponding to the wavelength is determined to be a second weight, and the first weight is greater than the second weight.

4. The method according to claim 2, characterized in that: The determining the loss function value based on the wavelength error and the weight coefficient comprises: Calculating the product of the wavelength error and the weight coefficient of the wavelength corresponding to the wavelength error; The average value of the products corresponding to each wavelength is calculated to obtain the loss function value.

5. The method according to claim 1, characterized in that The recurrent neural network layer includes: a gated recurrent unit GRU, and the inputting of the training sample into the initial neural network and the outputting of the predicted spectrum corresponding to the light response intensity include: Inputting the light response intensity into the residual network layer to obtain a first extracted feature; Inputting the first extracted features into the GRU to obtain second extracted features; Inputting the second extracted feature into the self-attention mechanism layer to obtain a third extracted feature; The third extracted feature is input into the fully connected layer, and the predicted spectrum is output.

6. The method according to claim 1, characterized in that The determination process of the calibration function includes: Use a spectrometer to calibrate the LED standard light source and obtain the spectrum; Using an image detector to photograph the LED standard light source to obtain actual light response intensity; Based on the spectrum and the actual photoresponse intensity, the calibration function is determined.

7. The method according to any one of claims 1 to 6, characterized in that: The obtaining of training samples, wherein the training samples include a plurality of theoretical spectra and a light response intensity corresponding to each theoretical spectrum, comprises: Generating the theoretical spectrum using a Gaussian line function; Based on the theoretical spectrum and the calibration function, the photoresponse intensity is determined.

8. A spectrum reconstruction method based on neural network, characterized in that: The method comprises: Obtaining the actual light response intensity of the object to be measured; The actual light response intensity of the object to be measured is input into a neural network to obtain a reconstructed spectrum of the object to be measured, wherein the neural network is obtained based on the method for training a neural network according to any one of claims 1 to 7.

9. The method according to claim 8, characterized in that The neural network includes a residual network layer, a GRU, a self-attention mechanism layer and a fully connected layer. The actual light response intensity of the object to be measured is input into the neural network to obtain the reconstructed spectrum of the object to be measured, including: Inputting the actual light response intensity into the residual network layer to obtain a first spectral feature; Inputting the first spectral feature into the GRU to obtain a second spectral feature; Inputting the second spectral feature into the self-attention mechanism layer to obtain a third spectral feature; The third spectral feature is input into the fully connected layer, and the reconstructed spectrum is output.

10. An electronic device, characterized in that: The device comprises: a memory and a processor; The memory is used to store relevant program codes; The processor is used to call the program code to execute the method for training a neural network as described in any one of claims 1 to 7 or the spectral reconstruction method based on a neural network as described in claim 8 or 9.

Images