Method for realizing function near-infrared deartifact based on long short-term memory autoencoder

Abstract

The application relates to the field of biomedical data processing, in particular to a method for realizing function near-infrared deartifact based on a long short-term memory self-encoder, which processes original near-infrared signals in three stages: 1) the original near-infrared signals are subjected to morphological feature extraction through a downsampling module; 2) a long short-term memory module captures the time correlation between single samples to enhance the features; and 3) an upsampling module recovers and reconstructs the morphological feature information of the near-infrared signals from a latent space. Finally, the reconstructed pure near-infrared signals are output in an output layer. The application overcomes the shortcomings that the motion artifact removal method of traditional signal processing is too dependent on the optimal selection of model parameters of the brain region and professional knowledge, realizes full-automatic removal of the motion artifact, and the method achieves good denoising results on different data sets.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of biomedical data processing, and particularly relates to a method for realizing functional near-infrared de-artifact based on a long short-term memory autoencoder. BACKGROUND

[0002] Functional near-infrared spectroscopy (fNIRS) is a non-invasive, non-ionizing method for monitoring and imaging cerebral hemodynamic function. It uses infrared light in the near-infrared range (e.g., 760 and 850 nm) to measure changes in hemodynamic responses related to neuronal activity in the brain from the light emitted from the source to the brain tissue, and converts the changes in the concentration of oxygenated hemoglobin (HBO) and deoxygenated hemoglobin (HBr) using the modified Beer-Lambert law, which is interpreted as the size of the activation of the superficial layer of the cortex. The low cost and low invasiveness of near-infrared make it increasingly applied to clinical and experimental studies of different populations. However, when the subjects move their heads, the shift or decoupling between the source or detector (optode) and the scalp can cause sudden changes in light intensity, resulting in motion artifacts. Although a good coupling method between the optical fiber and the scalp can strongly attenuate these effects, it is difficult to completely avoid these artifacts. In populations with more and extreme head movements, such as young children and infants, motion artifacts are more severe, and in studies of atypical patterns of neural activity in populations with elevated levels of head movement, such as autism spectrum disorders, attention deficit / hyperactivity disorder, and epilepsy.

[0003] Several motion correction algorithms have been developed to prevent data loss. PCA assumes that the activation signal is orthogonal to the spatial interference pattern in the baseline data, and separates the activity-induced response from physiological interference in diffuse optical imaging data by spatial feature filtering. Since the hemodynamic response can occur simultaneously in multiple channels, conventional PCA can accidentally delete the required signal from the data. tPCA is an extension of PCA to prevent overcorrection, which is only applied to data periods with motion artifacts, but this method is more complex to use and requires users to manually set multiple parameters. The sample spline method uses a moving standard deviation and spline interpolation method to model noisy data, which can semi-automatically detect and reduce motion artifacts in the data. This method relies on manually setting the best parameters, and the variance of the motion artifact must be greater than that of the physiological signal component. In wavelet filtering, the measured signal is first decomposed on a wavelet basis, and after zeroing the wavelet coefficients that are considered to represent artifacts, the signal is reconstructed from the corrected coefficient set. Wavelet filtering is very effective in removing spike signals, but it is less effective in removing baseline shifts, and the effectiveness of wavelet filtering in removing artifacts is strongly affected by the signal-to-noise ratio of the original data.

[0004] The above several algorithms have good effects in removing motion artifacts of near-infrared signals, but the performance of the several algorithms is excessively dependent on model parameters and basic selection, and the traditional signal processing method establishes a linear relationship between the noisy signal and the pure signal through a linear relationship, and it is difficult to establish a mapping relationship through this method for the non-linear and non-stationary parts in the deeper layer of the signal. SUMMARY

[0005] The purpose of the present application is to overcome the shortcomings of the prior art, and to provide a method for realizing functional near-infrared artifact removal based on a long short-term memory autoencoder, which overcomes the shortcomings of the traditional motion artifact removal method of signal processing, which is excessively dependent on the best selection of model parameters of professional knowledge and brain regions, and realizes the full-automatic removal of motion artifacts.

[0006] In order to achieve the above purpose, the technical scheme specifically adopted by the present application is as follows:

[0007] The method for realizing functional near-infrared artifact removal based on a long short-term memory autoencoder comprises the following steps:

[0008] Step 1, collecting the action data of multiple subjects, including the data of multiple subjects reading aloud, nodding up and down, nodding to the side, twisting the upper body to the right, twisting the upper body to the left, quickly shaking the head left and right, and raising the eyebrows to generate motion artifacts;

[0009] Step 2, cutting the data set, and dividing the data set into a training set, a validation set and a test set in a ratio of 8:1:1;

[0010] Step 3, constructing a long short-term memory autoencoder model, wherein the denoising module of the autoencoder model is composed of a down-sampling module, a long short-term memory module and an up-sampling module, and the calculation process of the long short-term memory module is composed of the following steps:

[0011] f t =σ(W f ·[h t-1 ,θ t ]+b f )

[0012] i t =σ(W i ·[h t-1 ,θ t ]+b i )

[0013]

[0014] In the formula, f t ,i t , denotes the forget gate, the input gate and the new cell state created using tanh, W denotes the weight matrix, and b f i c denote the bias parameters of the forget gate, the input gate and the cell state respectively; sigma denotes the sigmoid activation function, and the output value is between 0 and 1; tanh denotes the hyperbolic tangent activation function, and the output value is between -1 and 1; h t-1 denotes the hidden state of the previous time step, C t-1 denotes the cell state of the previous time step, C t denotes the cell state of the current time step;

[0015] The output o of the long short-term memory module morphology enhancement t is expressed as:

[0016] o t = sigma (W o · [h t-1 , theta t ] + b o

[0017] h t = o t ** tanh (C t )

[0018] In the formula, o t denotes the information to be output, C t denotes the updated cell state, h t denotes the output result of the current sequence model;

[0019] Step 4, set the learning process of the long short-term memory autoencoder model:

[0020] Step 4.1, set the loss function of the model training as the mean square error function, and calculate the point-by-point error of the sample pair based on the mean square error function; wherein the mean square error function is:

[0021]

[0022] In the formula, x i denotes the pure blood dynamics response, x ri denotes the reconstructed near-infrared signal of the model, and n denotes the sampling point size of the sample;

[0023] Step 4.2, set the batch size of the model training to 512, the training round to 100, and the learning rate to 0.0001;

[0024] ​​Step 5: Use the leave-one-out method to perform validation. Train the model by simulating the data of all subjects except one, and test the model on the removed subject. Repeat the above steps until all subjects are used for testing.

[0025] Step 6: Synthesize a hemodynamic response in the original near-infrared signal by introducing a signal change of 1% of the 690 nm signal relative to the baseline and a signal change of 2% of the 830 nm signal relative to the baseline, and add the synthesized hemodynamic response to the real data containing motion artifacts collected in Step 1;

[0026] Step 7: Import the real data with the added synthetic hemodynamic response into Homer2 software, convert the original real data into optical density changes, perform bandpass processing to reduce low-frequency drift and high-frequency noise, and finally perform block averaging using the block averaging function to evaluate the hemodynamic response;

[0027] Step 8: Introduce mean square error, signal-to-noise ratio, root mean square ratio, and Pearson correlation coefficient to calculate the recovery result of the hemodynamic response in step 7 to measure the ability of the long short-term memory autoencoder model to remove motion artifacts.

[0028] Furthermore, in step 3, when constructing the downsampling module, a one-dimensional convolution layer is applied to the local feature mining of the original near-infrared signal, and a maximum pooling layer is added in each block to gradually downsample the near-infrared signal in the time domain to obtain deeper features; the feature vector filtered by the downsampling layer can be expressed as θ = (θ1, θ2, ..., θ N ), the long short-term memory module is used to measure the temporal correlation between individual samples and apply the results to the morphological features for feature enhancement. It enhances the feature representation by capturing the temporal dynamic characteristics of the near-infrared signal. The process can be described as:

[0029] h t ,c t =LSTM(h t-1 , c t-1 ,θ t )1 <t<N

[0030] Where h, c are the hidden layer state and unit state of the long short-term memory module; N is the total number of samples, θ t Represents the feature vector of the t-th sample.

[0031] Furthermore, in step 3, when constructing the upsampling module, the multi-dimensional features after morphological feature extraction by the downsampling module and feature enhancement by the long short-term memory module are up-sampled, and finally the clean hemodynamic response after reconstructing the denoising network is output at the terminal output layer.

[0032] Furthermore, in step 3, the downsampled features are concatenated with the upsampled features of each layer, which can prevent the loss of information of the near-infrared signal during the upsampling process and retain more detailed features.

[0033] The present invention has the following characteristics and beneficial effects: Unlike traditional signal processing motion artifact removal methods that rely too much on professional knowledge and the optimal selection of model parameters of brain regions, the present invention uses an autoencoder model based on long short-term memory to remove motion artifacts from near-infrared signals. This method can extract comprehensive features and information from nonlinear and non-stationary original near-infrared signals, thereby resolving the uncertain relationship between the original noisy signal and the pure signal, and the optimal denoising parameters are learned during the denoising process, thereby achieving fully automatic removal of motion artifacts. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0035] Figure 1 Flowchart of a method for removing artifacts from an autoencoder based on long short-term memory according to an embodiment of the present invention;

[0036] Figure 2 This is a structural diagram of the long short-term memory autoencoder model in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention is described in detail below in conjunction with specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0038] In order to make up for the deficiency of the motion artifact removal method of traditional signal processing which over-reliance on professional knowledge and optimal selection of model parameters of brain region, the application proposes a method for realizing functional near-infrared artifact removal based on long short-term memory self-encoder, which reduces motion artifacts by training a neural network to reconstruct the hemodynamic response coupled with neuronal activity, which first uses a deep neural network to denoise the near-infrared signal, and in the training stage, each layer in the model is regarded as a filter to learn the core features output from the last layer, and the filtered near-infrared waveform features are input to the next layer. The features extracted by the network can distinguish between real near-infrared (non-artifact) and artifact. Then after multi-layer filtering, these features are used to reconstruct the artifact-free near-infrared signal at the output layer. The point-by-point error calculation of the artifact-containing near-infrared signal output by the output layer and the clean near-infrared signal by the loss function makes the reconstructed near-infrared signal as consistent as possible with the pure near-infrared signal, and the model reaches the optimal through multiple training. Finally, in the test stage, the trained model directly operates on the original near-infrared signal and produces the predicted artifact-free near-infrared signal, as shown in Figure 1 The application further discloses a method for realizing functional near-infrared artifact removal based on long short-term memory self-encoder, and the specific implementation process comprises the following steps:

[0039] Step 1, collecting action data of multiple subjects: collecting the data of multiple subjects performing actions such as loud reading, nodding up and down, nodding to the side, twisting the upper body to the right, twisting the upper body to the left, quickly shaking the head left and right, and raising the eyebrows to generate motion artifacts.

[0040] Step 2, cutting the data set, and dividing the data set into a training set, a validation set and a test set in a ratio of 8:1:1.

[0041] Step 3, constructing a long short-term memory-based self-encoder model, wherein the denoising module of the self-encoder model is composed of a down-sampling module, a long short-term memory module and an up-sampling module, and the construction comprises the following steps:

[0042] Step 3.1, constructing a down-sampling module, wherein the application applies a one-dimensional convolutional layer to local feature mining of the original near-infrared signal. A maximum pooling layer is added in each block to gradually down-sample the near-infrared signal in the time domain to obtain deeper features, and the feature vector filtered by the down-sampling layer can be expressed as θ=(θ1,θ2,...,θ N ).

[0043] Step 3.2, constructing a long short-term memory module, wherein the long short-term memory module is used to measure the time correlation between single samples and apply the result to morphological features for feature enhancement. The long short-term memory module enhances the feature representation process by capturing the time dynamic characteristics of the near-infrared signal, which can be described as:

[0044] h t , c t =LSTM(ht-1 , c t-1 , θ t )1<t<N (1)

[0045] where h, c are the hidden state and cell state of the LSTM module, N is the total number of samples, θ t represents the feature vector of the t-th sample;

[0046] The calculation process of the LSTM module consists of the following steps:

[0047] f t = σ(W f · [h t-1 , θ t ]+b f ) (2)

[0048] i t = σ(W i · [h t-1 , θ t ]+b i ) (3)

[0049]

[0050] where f t , i t , respectively represent the forget gate, input gate and represent the new cell state created using tanh. W represents the weight matrix, b represents the bias parameter, b f , b i , b c are the bias parameters of the forget gate, input gate and cell state respectively. σ represents the sigmoid activation function, the output value is between 0 and 1. tanh represents the hyperbolic tangent activation function, the output value is between -1 and 1. h t-1 represents the hidden state of the previous time step, C t-1 represents the cell state of the previous time step, C t represents the cell state of the current time step.

[0051] The output o t after the shape enhancement of the LSTM module is represented as:

[0052] o t = σ(W o · [h t-1 , θ t ]+b o ) (6)

[0053] h t = o t *tanh(Ct ) (7)

[0054] In the formula, o t represents the output result of the feature vector enhancement of the current module for the tth sample, h t represents the hidden state of the current time step.

[0055] Step 3.3, construct an up-sampling module, up-sample the multi-dimensional features after the morphological feature extraction by the down-sampling module and the feature enhancement by the long short-term memory module, and finally output the final clean hemodynamic response reconstructed by the denoising network in the end output layer.

[0056] Step 3.4, splice the down-sampled features with the up-sampled features of each layer, which can prevent the loss of near-infrared signal information in the up-sampling process and retain more detailed features.

[0057] Step 4, set the learning process of the long short-term memory autoencoder model, specifically:

[0058] Step 4.1, set the loss function of model training as the mean square error function:

[0059]

[0060] In the formula, x i represents the pure hemodynamic response, x ri represents the reconstructed near-infrared signal, and n represents the sample point size of the sample. The mean square error function calculates the point error for each sample pair.

[0061] Step 4.2, set the batch size of model training to 512, the number of training rounds to 100, and the learning rate to 0.0001.

[0062] Step 5, use the leave-one-out method for verification. Train the model by simulating the removal of all subjects except one subject, test the model by removing the subject, and repeat the above steps until all subjects are used for testing.

[0063] As Figure 2As shown, the long short-term memory autoencoder model constructed by the present application works, including the following steps: 1) the original near-infrared signal is subjected to morphological feature extraction through the downsampling module; 2) the original near-infrared signal subjected to morphological feature extraction through the downsampling module is subjected to morphological feature enhancement by measuring the time correlation between single samples through the long short-term memory module; 3) the original near-infrared signal subjected to morphological feature enhancement through the long short-term memory module is subjected to recovery and reconstruction through the upsampling module, and the outputs of each layer in the downsampling and upsampling modules are spliced in the upsampling process to prevent information loss of the near-infrared signal in the upsampling process, so as to retain more detailed features; 4) finally, the morphological features after upsampling are aggregated in the output layer to generate the reconstructed pure near-infrared signal.

[0064] Step 6, synthesize the hemodynamic response in the original near-infrared signal by introducing a signal change of 1% of the 690nm signal relative to the baseline and 2% of the 830nm signal relative to the baseline, and add the synthesized hemodynamic response to the real data collected in step 1 containing motion artifacts.

[0065] Step 7, import the real data added with the synthesized hemodynamic response into Homer2 software, convert the original real data into optical density changes, reduce low-frequency drift and high-frequency noise through band-pass processing, and finally evaluate the hemodynamic response through block averaging function for block averaging.

[0066] Step 8, synthetic simulation data:

[0067] The simulated near-infrared signal can be described as y(t):

[0068] y(t) = z(t) + r(t) + ε(t) (9)

[0069] In the formula, z(t) simulates the hemodynamic response, r(t) simulates the near-infrared motion artifact, and ε(t) simulates the resting-state near-infrared signal.

[0070] The simulated hemodynamic response z(t) is generated by convolving the task stimulation function u(t) with the standard blood flow hemodynamic response function h(t), which can be described as:

[0071] z(t) = u(t) * h(t) (10)

[0072] The standard hemodynamic response h(t) is simulated by a gamma function, wherein the response function amplitude is 54 μ M·mm. In the present application, the amplitude is set to 40 ～ 80 μ Random values are selected between the distributions of M·mm to facilitate that our model can learn between samples with different parameters.

[0073] The task stimulus function u(t) is represented as:

[0074]

[0075] where B represents the boxcar function, b is the duration of the stimulus, t1 represents the rest duration, t2 represents the time shift factor of the stimulus, i(b+t1) represents the stimulus phase, t s represents the time point of the spike occurrence.

[0076] The hemodynamic response stimulus interval is 70 s, and multiple stimuli can be provided during the 6-minute recording period. The resting-state data generated from the real resting data using the AR model, the simulated motion artifact ε(t) includes two parts of baseline shift and spike θ(t), which can be described as:

[0077]

[0078] where A represents the amplitude of the spike artifact, t s represents the time point of the spike occurrence, b represents the scale parameter, and e represents the Laplace distribution function. (12) can be rewritten as

[0079] The baseline shift artifact is modeled as a random positive or negative change in the fNIRS simulation signal, ranging between -20 μM·mm and 20 μM·mm.

[0080] Step 9, the recovery results of the hemodynamic response of the real data set in step 7 and the simulated data set in step 7 are calculated by using the mean square error, signal-to-noise ratio, percentage root ratio, and Pearson correlation coefficient to measure the ability of the long short-term memory autoencoder model to remove motion artifacts. Among them:

[0081] Mean square error:

[0082]

[0083] where HRF represents the standard hemodynamic response, and HRF' represents the hemodynamic response containing motion artifacts. Here, MSE represents the error between the true hemodynamic response and the denoised hemodynamic response, and the smaller the MSE, the better the denoising result.

[0084] Signal-to-noise ratio:

[0085]

[0086] where ΔSNR represents the index of the signal-to-noise ratio, and the larger the ΔSNR, the better the denoising result.

[0087] Percentage root ratio:

[0088]

[0089] The percentage root mean square error indicates the matching degree of two signals, the smaller the PDR result is, the higher the signal matching degree is, and the lower the motion artifact content in the signal is.

[0090] Pearson correlation coefficient:

[0091]

[0092] In the formula, R 2 represents the similarity between the true signal and the processed signal, R 2 The larger the R

[0093] The results are shown in Tables 1 and 2, and the proposed long short-term memory autoencoder realizes the function of near-infrared artifact correction method is better than the traditional method in the simulation data set and real data set on each index, showing reasonable motion artifact removal ability.

[0094] Table 1: The values of mean square error (MSE), Pearson correlation coefficient (R 2 ), signal-to-noise ratio (ΔSNR) and percentage root mean square error (PDR) of different motion artifact correction algorithms on the simulation data set

[0095]

[0096] Table 2: The values of mean square error (MSE), Pearson correlation coefficient (R 2 ), signal-to-noise ratio (ΔSNR) and percentage root mean square error (PDR) of different motion artifact correction algorithms on the real data set

[0097]

[0098] The above shows and describes the basic principles, main features and advantages of the present application. It should be understood by those skilled in the art that the present application is not limited by the above examples, and the above examples and descriptions in the specification are only preferred examples of the present application and are not intended to limit the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for implementing functional near-infrared artifact removal based on a long short-term memory autoencoder, characterized by: The steps include: Step 1: Collecting the execution action data of multiple subjects; Step 2: Split the dataset into training set, validation set, and test set in a ratio of 8:1:1; Step 3: Construct an autoencoder model based on long short-term memory. The denoising module of the autoencoder model consists of a downsampling module, a long short-term memory module, and an upsampling module. The calculation process of the long short-term memory module consists of the following steps: f t =σ(W f ·[h t-1 ,i t ]+b f ) I t =σ(W i ·[h t-1 ,i t ]+b i ) Where, f t ,i t , represents the forget gate, the input gate and represents the new cell state created using tanh, W represents the weight matrix, b f 、b i 、b c They represent the bias parameters of the forget gate, input gate, and cell state respectively; σ represents the sigmoid activation function, and the output value is between 0 and 1; tanh represents the hyperbolic tangent activation function, and the output value is between -1 and 1; h t-1 represents the hidden state of the previous time step, C t-1 Represents the cell state at the previous time step, C t Represents the cell state at the current time step; Output o after morphological enhancement by long short-term memory module t Expressed as: the t =σ(W o ·[h t-1 ,i t ]+b o h t =o t *fishy(C) t ) In the formula, o t Represents the information to be output, C t Represents the updated cell state, h t Represents the output of the current sequence model; The feature vector filtered by the downsampling module can be expressed as θ=(θ1, θ2, ..., θ N ), the long short-term memory module is used to measure the temporal correlation between individual samples and apply the results to the morphological features for feature enhancement. It enhances the feature representation by capturing the temporal dynamic characteristics of the near-infrared signal. The process can be described as: h t ,c t =LSTM(h t-1 ,c t-1 ,θ t ) 1<t<N Where h, c are the hidden layer state and unit state of the long short-term memory module; Step 4: Set up the learning process of the long short-term memory autoencoder model; Step 5: Use the leave-one-out method to perform validation. Train the model by simulating the data of all subjects except one, and test the model on the removed subject. Repeat the above steps until all subjects are used for testing. Step 6: synthesize a hemodynamic response in the original near-infrared signal by introducing signal variation, and add the synthesized hemodynamic response to the real data containing motion artifacts collected in step 1; Step 7: Import the real data with the added synthetic hemodynamic response into Homer2 software, convert the original real data into optical density changes, perform bandpass processing to reduce low-frequency drift and high-frequency noise, and finally perform block averaging using the block averaging function to evaluate the hemodynamic response; Step 8: Introduce mean square error, signal-to-noise ratio, root mean square ratio, and Pearson correlation coefficient to calculate the recovery result of the hemodynamic response in step 7 to measure the ability of the long short-term memory autoencoder model to remove motion artifacts.

2. The method for implementing functional near-infrared artifact removal based on a long short-term memory autoencoder according to claim 1, characterized in that: The execution action data in step 1 includes data of reading aloud, nodding the head up and down, nodding the head to the side, twisting the upper body to the right, twisting the upper body to the left, shaking the head quickly left and right, and raising eyebrows to generate motion artifacts.

3. The method for implementing functional near-infrared artifact removal based on a long short-term memory autoencoder according to claim 1, characterized in that: In step 3, when constructing the downsampling module, a one-dimensional convolution layer is applied to the local feature mining of the original near-infrared signal, and a maximum pooling layer is added in each block to gradually downsample the near-infrared signal in the time domain to obtain deeper features.

4. The method for implementing functional near-infrared artifact removal based on a long short-term memory autoencoder according to claim 1, characterized in that: In step 3, when constructing the upsampling module, the multi-dimensional features after morphological feature extraction by the downsampling module and feature enhancement by the long short-term memory module are up-sampled, and finally the clean hemodynamic response after reconstructing the denoising network is output at the terminal output layer.

5. The method for implementing functional near-infrared artifact removal based on a long short-term memory autoencoder according to claim 1, characterized in that: In step 3, the downsampled features are concatenated with the upsampled features of each layer, which can prevent the information loss of the near-infrared signal during the upsampling process and retain more detailed features.

6. The method for implementing functional near-infrared artifact removal based on a long short-term memory autoencoder according to claim 1, characterized in that: The step 4 comprises the following steps: Step 4.

1. Set the loss function of the model training to the mean square error function, and calculate the point-by-point error of the sample pairs based on the mean square error function; Step 4.2: Set the batch size of the model training to 512, the number of training rounds to 100, and the learning rate to 0.0001.

7. The method for implementing functional near-infrared artifact removal based on a long short-term memory autoencoder according to claim 6, characterized in that: The mean square error function in step 4.1 is: Where x i represents the pure hemodynamic response, x ri represents the near-infrared signal after model reconstruction, and n represents the sampling point size of the sample.

8. The method for implementing functional near-infrared artifact removal based on a long short-term memory autoencoder according to claim 7, characterized in that: In step 6, a hemodynamic response is synthesized in the original near-infrared signal by introducing a signal change of 1% of the 690nm signal relative to the baseline and a signal change of 2% of the 830nm signal relative to the baseline, and the synthesized hemodynamic response is added to the real data containing motion artifacts collected in step 1.

Images