Spectrum reconstruction method based on self-attention mechanism

Abstract

The invention discloses a spectrum reconstruction method based on a self-attention mechanism, and the method comprises the steps: 1, adding different degrees of Gaussian noise to spectrum data, and obtaining a preprocessed spectrum vector; 2, screening the filtering function library to obtain a low-correlation filter; 3, establishing a spectrum reconstruction network based on a self-attention mechanism and processing a spectrum vector to obtain a reconstructed spectrum vector; and 4, constructing a loss function, and training the spectrum reconstruction network to obtain an optimal spectrum reconstruction network. According to the method, important information in input data can be more accurately captured by utilizing a self-attention mechanism, so that the accuracy and reliability of spectrum reconstruction can be improved.

Description

Technical Field

[0001] The present invention belongs to the cross - technical field of spectral reconstruction and computer vision, and specifically relates to a spectral reconstruction method based on self - attention mechanism. Background Art

[0002] Hyperspectral imaging technology provides rich data support for fields such as environmental monitoring, material identification, and medical diagnosis by capturing the reflection or radiation information of objects in continuous spectral bands. However, traditional hyperspectral reconstruction methods face the following key problems:

[0003] Dependence on a large amount of labeled data: Existing deep - learning - based methods rely on a large amount of labeled spectral data, and the annotation cost of hyperspectral data is high, which limits the practical application of the model;

[0004] Strong noise sensitivity: Spectral signals are easily interfered by environmental noise (such as Gaussian noise, Poisson noise) during the acquisition process. Existing methods have insufficient robustness to noise, resulting in distorted reconstruction results;

[0005] Insufficient local feature extraction: Traditional convolutional neural networks (CNNs) are difficult to capture long - range dependencies in spectral data, and the fusion of local spectral features and global context information is insufficient, affecting the reconstruction accuracy;

[0006] Fixed filter design: The filter selection of existing filter arrays is mostly static design and cannot be dynamically adjusted according to the input spectral characteristics, resulting in low coding efficiency.

[0007] To address the above problems, recent research has tried to introduce self - supervised learning or data augmentation techniques, but there are still defects such as weak model generalization ability and insufficient multi - scale feature fusion. Summary of the Invention

[0008] The present invention is to solve the above - mentioned deficiencies of the existing technology, and proposes a spectral reconstruction method based on self - attention mechanism, aiming to balance noise robustness, data efficiency, and feature expression ability, so as to improve the accuracy and reliability of spectral reconstruction, and solve the problems such as limited accuracy, poor adaptability, low computational efficiency, and lack of intelligent processing ability in the existing technology.

[0009] To achieve the above - mentioned invention purpose, the present invention adopts the following technical solutions:

[0010] The characteristics of a spectral reconstruction method based on self - attention mechanism of the present invention include the following steps:

[0011] Step 1. Obtain the original spectral data set , where, represents the t - th spectral vector, D is the number of spectral wavelength points; T represents the total number of spectral vectors;

[0012] Pair Gaussian noise is added sequentially and Poisson noise After that, the t-th spectral vector with noise is obtained , where represents a constant, and ;

[0013] Pair After normalization, the t-th preprocessed spectral vector is obtained ;

[0014] Step 2. Create a filter set , and calculate The cross-correlation coefficient between any two filters in, and the filters whose cross-correlation coefficients meet the threshold are used as low-correlation filters, so as to screen out all correlation filters in and form a filtered filter set , where represents the m-th low-correlation filter, and M represents the number of filtered filters;

[0015] Step 2. Build a spectral reconstruction network based on the self-attention mechanism, which includes in sequence: an input normalization module, N cascaded global self-attention modules and N cascaded local self-attention modules, a fusion module, and a normalization module;

[0016] Step 2.1. The input normalization module uses Pair for processing to obtain the m-th encoded vector , thus obtaining the t-th spectral encoding sequence ; Then pair After standardization, the m-th standardized encoded vector is obtained ; Thus obtaining the t-th standardized spectral encoding sequence ;

[0017] Step 2.2. Each cascaded global self-attention module is provided with a multi-head self-attention mechanism layer, and pair for processing to obtain the N-th cascaded global feature , where represents The corresponding N-th cascaded global feature;

[0018] Step 2.3. Each cascaded local attention module is also provided with a multi-head self-attention mechanism layer, and pair for processing to obtain the N-th cascaded local feature ;

[0019] Step 2.4. After the fusion module performs weighted fusion on and the t-th fused feature is obtained, where is a parameter to be learned;

[0020] Step 2.5. The normalization module processes the fused feature to obtain the t-th normalized attention feature , and then the t-th hidden attention feature is obtained using Equation (1):

[0021] (1)

[0022] In Equation (1), , are the two weights of the fully connected layer, , are the two bias terms; is the layer normalization function, is the smooth activation function;

[0023] The hidden attention feature is mapped to the spectral space, and then the t-th reconstructed spectral vector is obtained using Equation (2): (2)

[0024] In Equation (2), represents the projection matrix, is the bias term; is the linear activation function;

[0025] Step 3. The total loss of the spectral reconstruction network is constructed using Equation (3) :

[0026] (3)

[0027] In Equation (1), represents the mean squared error loss, represents the spectral smoothness constraint loss, is a parameter to be learned;

[0028] Step 4. AdamW is used to train the spectral reconstruction network and calculate the total loss to update the network parameters until the total loss converges or reaches the maximum number of iterations, and then the optimal spectral reconstruction model is obtained for reconstructing the input spectral vector.

[0029] The characteristics of a spectral reconstruction method based on self-attention mechanism described in the present invention also lie in that step 2.2 includes the following steps:

[0030] Step 2.2.1. When n = 1, the nth cascaded global self-attention module processes to obtain the mth global feature of the nth cascade ;

[0031] The linear transformation layer of the jth head self-attention layer in the nth cascaded global self-attention module processes to obtain the jth query vector of the nth cascade, the jth key vector of the nth cascade, and the jth value vector of the nth cascade, so as to obtain the jth self-attention feature of the nth cascade, and further obtain the jth global feature of the nth cascade; finally, the jth global weighted feature of the nth cascade is obtained, where , , respectively represent the weight matrices corresponding to the three linear transformations of the jth head self-attention layer in the nth cascaded global self-attention module; softmax represents the activation function, and d is the dimension of the single-head self-attention layer; represents the attention weight parameter to be learned of the weight of the jth head in the nth cascaded global self-attention module; J represents the total number of heads of the multi-head self-attention mechanism layer;

[0032] Step 2.2.2. When n = 2, 3,..., N, the mth global feature of the (n - 1)th cascade is input into the nth cascaded global self-attention module for processing to obtain the mth global feature of the nth cascade; thus, the Nth cascaded global self-attention module outputs the Nth cascaded global feature .

[0033] Furthermore, step 2.3 includes the following steps:

[0034] Step 2.3.1. When n = 1, the nth cascaded local attention module divides into W local windows and then processes each local window according to the process of step 2.2.1 to obtain the local features of the nth cascade of each local window;

[0035] The local features of the nth cascade within the wth window are weighted and summed to generate the local context feature of the nth cascade of the wth local window; wherein, represents the n-th cascaded local feature of the w-th local window, represents the n-th weight of the w-th local window, ;

[0036] After concatenating the n-th cascaded local context features of all local windows in the original order, the n-th cascaded local feature is obtained ;

[0037] Step 2.3.2 When n = 2, 3, …, N, the (n - 1)-th cascaded local feature is input into the n-th cascaded local attention module for processing to obtain the n-th cascaded local feature ; thus, the N-th cascaded local feature is output by the N-th cascaded local attention module.

[0038] Furthermore, in Step 3, and are obtained by using Equations (4) and (5):

[0039] (4)

[0040] (5)

[0041] In Equation (5), is the (t + 1)-th preprocessed spectral vector.

[0042] An electronic device according to the present invention includes a memory and a processor, characterized in that the memory is used to store a program for supporting the processor to execute the spectral reconstruction method, and the processor is configured to execute the program stored in the memory.

[0043] A computer-readable storage medium according to the present invention, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, it executes the steps of the spectral reconstruction method.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] 1. The present invention significantly improves the noise robustness and solves the problem that the existing methods are sensitive to Gaussian noise and Poisson noise by actively injecting multiple types of noise into the training data, such as Gaussian noise and Poisson noise , and combining with the AdamW Opt technology to suppress overfitting. The model can still stably extract spectral features in a noisy environment, reducing the reconstruction error in the measured data. Especially in the low signal-to-noise ratio scenario, the peak signal-to-noise ratio is improved;

[0046] 2. The present invention breaks through the limitation of local feature extraction of traditional CNNs and realizes the deep fusion of global-local features; adopts a cascaded global self-attention module and a local window self-attention module, and dynamically fuses the two types of features through a learnable weight β; in the reconstruction of complex spectral signals, it improves the feature coverage rate and the retention degree of edge details;

[0047] 3. The present invention optimizes the filter design efficiency, solves the problem of insufficient coding ability of static filters, and screens a set of low-correlation filters based on the cross-correlation coefficient threshold , dynamically constructs a coding matrix that matches the input spectral characteristics, improves the filter utilization rate, increases the coding efficiency, reduces the hardware resource consumption, and at the same time speeds up the reconstruction speed;

[0048] 4. The dynamic learning rate strategy and joint loss function of the present invention accelerate convergence and improve accuracy. It adopts the AdamW optimizer combined with dynamic learning rate decay, and designs a joint optimization loss function of mean square error and spectral smoothness constraint, which shortens the model convergence period and significantly enhances the training stability; reduces the measured spectral smoothness error and reduces the spectral curve burr phenomenon;

[0049] 5. The residual connection and layer normalization of the present invention enhance the training stability of the deep network. Introducing residual connection and layer normalization in the self-attention module effectively alleviates the problem of gradient disappearance, improves the deep feature expression ability, and improves the consistency of the reconstruction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is the flowchart of filter screening of the present invention;

[0051] Figure 2 is the interactive flowchart of the global-local attention module of the present invention;

[0052] Figure 3 is the principle framework diagram of the spectral reconstruction algorithm based on the self-attention mechanism of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] In this embodiment, a spectral reconstruction method based on the self-attention mechanism is as Figure 1 shown, and the filter screening process includes simulating and generating an initial filter library, calculating the cross-correlation coefficient matrix, and dynamically screening low-correlation filters. The specific implementation is as follows:

[0054] Step 1. Obtain the original spectral dataset , where Denote the \(t\)-th spectral vector, \(D\) is the number of spectral wavelength points; \(T\) represents the total number of spectral vectors. Using a publicly available hyperspectral dataset (such as the ICVL spectral library), each spectral vector covers a wavelength range of 200 - 800 nm (\(D = 256\)) with a total of 201 sampling points, and 5000 basic spectral data are generated.

[0055] For add Gaussian noise and Poisson noise in sequence, and the \(t\)-th noisy spectral vector is obtained, where represents a constant, and ; for after normalization, the \(t\)-th preprocessed spectral vector is obtained. Replace the original dataset with the normalized noisy spectral data, expand the basic spectral data to 25,000 by adding noise and divide it into a training set and a validation set in the ratio of 8:2.

[0056] Step 2. Create a filter set Calculate the cross-correlation coefficient , of any two filters , where \(x\) and \(y\) represent arbitrary numbers; screen out the low-correlation filters that satisfy ≤ to form , where \(M\leq N\). And dynamically adjust the threshold to adapt to different noise levels: . Where is the reference noise intensity, is the basic threshold. Take the filters with cross-correlation coefficients satisfying the threshold as low-correlation filters, and thus screen out all the correlation filters in to form the filtered filter set , where represents the \(m\)-th low-correlation filter, and \(M\) represents the number of filtered filters. The cross-correlation coefficient = 0.3, and finally 100 low-correlation filters are retained.

[0057] The electrical signal intensity after the spectral sequence is converted by the encoding matrix filter array is:

[0058]

[0059] where is the wavelength; discretizing the integral equation in the above formula, a matrix equation as follows can be obtained:

[0060]

[0061] In the above formula, A is the number of filters used, and B is the number of bands of the spectral signal to be reconstructed. In principle, if there are a sufficient number of equations, the spectrum of the object can be directly reconstructed through matrix inversion. However, in practice, A is usually less than the number of spectral sampling points B to reduce the size of the spectrometer, so the unique solution of this matrix cannot be obtained. Even if A can be increased to make the matrix meet the inversion condition, there will still be errors between the measured values and the calibrated values of the transmission spectral curve, and the characteristics will not be obvious enough due to the existence of noise in the acquisition device. Therefore, the matrix equation cannot be directly solved, so a spectral reconstruction method using the self-attention mechanism is used to solve this system of equations.

[0062] Step 2. The overall framework of the algorithm is as Figure 3 shown. A spectral reconstruction network based on the self-attention mechanism is established, which successively includes: an input normalization module as Figure 2 shown, N cascaded global self-attention modules and N cascaded local self-attention modules, a fusion module, and a normalization module;

[0063] Step 2.1. The input normalization module uses to process to obtain the m-th encoded vector , so as to obtain the t-th spectral encoding sequence ; then after normalizing , the m-th normalized encoded vector is obtained; thus, the t-th normalized spectral encoding sequence is obtained.

[0064] Step 2.2. Each cascaded global self-attention module is provided with a multi-head self-attention mechanism layer, and processes to obtain the N-th cascaded global feature , where represents the N-th cascaded global feature corresponding to .

[0065] Step 2.2.1. When n = 1, the n-th cascaded global self-attention module processes to obtain the m-th global feature of the n-th cascade ;

[0066] The linear transformation layer of the j-th head self-attention layer in the n-th cascaded global self-attention module processes Process to obtain the j-th query vector of the n-th cascade , the j-th key vector of the n-th cascade , and the j-th value vector of the n-th cascade , so as to obtain the j-th self-attention feature of the n-th cascade , and further obtain the j-th global feature of the n-th cascade ; finally obtain the m-th global weighted feature of the n-th cascade , where , , respectively represent the weight matrices corresponding to the three linear transformations of the j-th head self-attention layer in the global self-attention module of the n-th cascade; softmax represents the activation function, and d is the dimension of the single-head self-attention layer; represents the attention weight parameter to be learned for the weight of the j-th head in the global self-attention module of the n-th cascade; J represents the total number of heads in the multi-head self-attention mechanism layer.

[0067] Step 2.2.2. When n = 2, 3,..., N, input the m-th global feature of the (n - 1)-th cascade into the global self-attention module of the n-th cascade for processing to obtain the m-th global feature of the n-th cascade; thus, output the global feature of the N-th cascade from the global self-attention module of the N-th cascade;

[0068] Step 2.3. Each local attention module of each cascade is also provided with a multi-head self-attention mechanism layer, and process to obtain the local feature of the N-th cascade.

[0069] Step 2.3.1. When n = 1, the local attention module of the n-th cascade divides into W local windows, and then processes each local window according to the process of Step 2.2.1 to obtain the local feature of the n-th cascade of each local window;

[0070] Perform weighted summation on the local features of the n-th cascade within the w-th window to generate the local context feature of the n-th cascade of the w-th local window; where represents the local feature of the n-th cascade of the w-th local window, represents the weight of the w-th local window of the n-th cascade, ;

[0071] After splicing the local context features of the n-th cascade of all local windows in the original order, obtain the local feature of the n-th cascade 。

[0072] Step 2.3.2 When n = 2, 3, …, N, input the (n - 1)-th cascaded local feature into the n-th cascaded local attention module for processing to obtain the n-th cascaded local feature ; thus, the N-th cascaded local feature is output by the N-th cascaded local attention module 。

[0073] Step 2.4. After the fusion module performs weighted fusion on and , the t-th fused feature is obtained, where the parameter to be learned is 0.6. The number and dimension of the hidden layers are flexibly set according to the task requirements. For example, it can be a series of layers with different numbers of units from D1 to D2 (e.g., D1 = 160, D2 = 240). The input dimension of 100 to the first hidden layer of 160 is for capturing wide-band features; the second hidden layer of 200 is for dividing windows and processing local details; the third hidden layer of 240 is for further refining local features.

[0074] Step 2.5. The normalization module processes the fused feature to obtain the t-th normalized attention feature , and thus the t-th hidden attention feature is obtained using Equation (1):

[0075] (1)

[0076] In Equation (1), , are the 2 weights of the fully connected layer, , are the 2 bias terms; is the layer normalization function, is the smooth activation function; two fully connected layers in the feed-forward network, with the middle dimension expanded by 4 times, and two LayerNorms are applied to the attention output and the feed-forward network output respectively to stabilize the training process;

[0077] Map the hidden attention feature to the spectral space, and thus the reconstructed t-th spectral vector is obtained using Equation (2): (2)

[0078] In Equation (2), represents the projection matrix, is the bias term; is the rectified linear function;

[0079] Step 3. Construct the total loss of the spectral reconstruction network using Equation (3). :

[0080] (3)

[0081] In Equation (3), represents the mean squared error loss, represents the spectral smoothness constraint loss. In this embodiment, the parameter to be learned is 0.5; and there is:

[0082] (4)

[0083] (5)

[0084] In Equation (5), is the (t + 1)-th preprocessed spectral vector.

[0085] Step 4. Use AdamW Opt to update the model parameters. The learning rate scheduler dynamically adjusts the learning rate according to the performance on the validation set. If the validation loss does not decrease for 5 consecutive training epochs, the learning rate is decayed to 0.01, and the initial learning rate is 0.001. The training termination condition is that the validation loss does not decrease for 5 consecutive times or reaches the maximum number of training epochs of 200 rounds. The validation metrics are output every 10 rounds and the results are visualized.

[0086] During the training process, the model performance is evaluated on the validation set regularly, and the R² and MSE metrics are calculated. By comparing the R² and MSE metrics on the validation set, the best-performing model parameters are selected for saving.

[0087] In this embodiment, an electronic device includes a memory and a processor. The memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0088] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is run by a processor, it executes the steps of the above method.

Claims

1. A spectrum reconstruction method based on self-attention mechanism, characterized in that: The steps include: Step 1. Obtain the original spectral dataset ,in, represents the tth spectral vector, D is the number of spectral wavelength points; T represents the total number of spectral vectors; right Add Gaussian noise in sequence and Poisson noise After that, we get the tth noisy spectrum vector ,in, represents a constant, and ; right After normalization, the tth preprocessed spectrum vector is obtained ; Step 2. Create a filter set , and calculate The mutual correlation coefficient of any two filters in the , and the filter whose mutual correlation coefficient meets the threshold is taken as the low correlation filter, so that Filter out all the correlation filters and form a filtered filter set ,in, represents the mth low correlation filter, and M represents the number of filters after screening; Step 2. Establish a spectral reconstruction network based on the self-attention mechanism, which includes: an input normalization module, N cascaded global self-attention modules and N cascaded local self-attention modules, a fusion module and a normalization module; Step 2.

1. The input normalization module uses right Process it to get the mth encoding vector , thus obtaining the tth spectral coding sequence ; then After standardization, the mth standardized encoding vector is obtained ; Thus, the tth standardized spectral coding sequence is obtained ; Step 2.

2. Each cascaded global self-attention module is equipped with a multi-head self-attention mechanism layer, and Processing is performed to obtain the global features of the Nth cascade ,in, express The corresponding global features of the Nth cascade; Step 2.

3. Each cascaded local attention module is also equipped with a multi-head self-attention mechanism layer, and Processing is performed to obtain the local features of the Nth cascade ; Step 2.

4. Fusion module pairs and After weighted fusion, the tth fusion feature is obtained ,in, is the parameter to be learned; Step 2.

5. Normalization module for fusion features Processing is performed to obtain the tth normalized attention feature , and then use formula (1) to get the tth hidden attention feature : (1) In formula (1), , are the 2 weights of the fully connected layer, , are 2 bias terms; is the layer normalization function, is a smooth activation function; Hide the attention feature Mapped to the spectral space, the reconstructed t-th spectral vector is obtained using formula (2): : (2) In formula (2), represents the projection matrix, is the bias term; is a linear activation function; Step 3. Use formula (3) to construct the total loss of the spectral reconstruction network : (3) In formula (1), represents the mean square error loss, represents the spectral smoothness constraint loss, is the parameter to be learned; Step 4. Use AdamW to train the spectral reconstruction network and calculate the total loss To update the network parameters until the total loss Until convergence or the maximum number of iterations is reached, the optimal spectral reconstruction model is obtained to reconstruct the input spectral vector.

2. The spectral reconstruction method based on the self-attention mechanism according to claim 1, characterized in that: Step 2.2 includes the following steps: Step 2.2.

1. When n=1, the global self-attention module of the nth cascade is Processing is performed to obtain the mth global feature of the nth cascade ; The linear change layer pair of the j-th self-attention layer in the n-th cascaded global self-attention module Processing is performed to obtain the jth query vector of the nth cascade , the jth key vector of the nth concatenation , the jth value vector of the nth cascade , thus obtaining the jth self-attention feature of the nth cascade , and then get the jth global feature of the nth cascade ; Finally, the mth global weighted feature of the nth cascade is obtained ,in, , , They represent the weight matrices corresponding to the three linear transformations of the j-th self-attention layer in the n-th cascaded global self-attention module; softmax represents the activation function, and d is the dimension of the single-head self-attention layer; represents the attention weight parameter to be learned for the weight of the jth head in the nth cascaded global self-attention module; J represents the total number of heads in the multi-head self-attention mechanism layer; Step 2.2.

2. When n=2,3,…,N, add the mth global feature of the n-1th cascade Input into the global self-attention module of the nth cascade for processing to obtain the mth global feature of the nth cascade ; Thus, the global self-attention module of the Nth cascade outputs the global features of the Nth cascade .

3. The spectral reconstruction method based on the self-attention mechanism according to claim 2, characterized in that: Step 2.3 includes the following steps: Step 2.3.

1. When n=1, the local attention module of the nth cascade will After being divided into W local windows, each local window is processed according to the process in step 2.2.1 to obtain the nth cascaded local features of each local window; Perform weighted summation on the local features of the nth cascade in the wth window to generate the local context features of the nth cascade of the wth local window. ;in, represents the local features of the nth cascade of the wth local window, represents the nth weight of the wth local window, ; After concatenating the n-th cascade local context features of all local windows in the original order, the n-th cascade local features are obtained. ; Step 2.3.2 When n=2,3,…,N, the local features of the n-1th cascade Input into the local attention module of the nth cascade for processing to obtain the local features of the nth cascade ; Thus, the local attention module of the Nth cascade outputs the local features of the Nth cascade .

4. The spectrum reconstruction method based on the self-attention mechanism according to claim 3, characterized in that: In step 3, equations (4) and (5) are used to obtain and : (4) (5) In formula (5), is the t+1th preprocessed spectrum vector.

5. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports a processor to execute the spectral reconstruction method according to any one of claims 1 to 4, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the spectrum reconstruction method according to any one of claims 1 to 4 are executed.

Images