A brain task load identification method, application and device

Abstract

The present invention relates to the field of medical detection technology, and discloses a method, application and device for identifying brain task load, the method comprising: using a functional near-infrared light source and a detection probe to perform multi-channel detection on the brain of a user in an active state, and obtaining light intensity signals of M channels; preprocessing the light intensity signals of the M channels, and converting the light intensity signals into hemoglobin concentration signals; extracting topological information between the functional near-infrared light source, the detection probe and the channels; constructing a grid image matrix of the hemoglobin concentration signal according to the topological information; inputting the grid image matrix into a task load identification model to perform task load identification, and obtaining a brain task load result. The present invention can learn the time information and spatial information of fNIRS data, ensure the high precision of the classification of the task load identification model, and thus improve the accuracy of brain task load identification.

Description

Technical Field

[0001] The present invention relates to the field of medical detection technology, and more specifically, to a method, application and device for identifying brain task load. Background Art

[0002] Too high or too low a task load in the brain can affect a person's cognitive ability, leading to individuals making wrong or irrational decisions. fNIRS (Functional near-infrared spectroscopy) is an imaging technology that collects endogenous signals generated during brain activity to form brain function images. It has been applied in the field of brain task load detection. It uses the good scattering properties of the main components of blood for 600-900nm near-infrared light to obtain the changes in oxygenated hemoglobin and deoxygenated hemoglobin during brain activity, and further analyzes and processes the changes to obtain the brain's task load.

[0003] The existing task load recognition method based on deep learning and fNIRS data extracts features including simple maximum, minimum, mean and variance, and then uses the features as the input of the neural network. After the neural network is learned and trained, it is classified to obtain the recognition result.

[0004] However, the above method ignores the spatial and temporal information of fNIRS signals, and the classification results output by the neural network have low accuracy, which will cause the defect of inaccurate task load identification. Summary of the invention

[0005] The present invention provides a method, application and device for identifying brain task load in order to improve the accuracy of identifying brain task load.

[0006] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0007] In a first aspect, the present invention provides a method for identifying brain task load, comprising:

[0008] A functional near-infrared light source and a detection probe are used to perform multi-channel detection on the user's brain when the user is in an active state, and light intensity signals of M channels are obtained, where M is a positive integer;

[0009] Preprocessing the light intensity signals of the M channels to convert the light intensity signals into hemoglobin concentration signals;

[0010] Extract topological information between functional near-infrared light sources, detection probes, and channels;

[0011] constructing a grid image matrix of the hemoglobin concentration signal according to the topological information;

[0012] The grid image matrix is ​​input into a task load recognition model to perform task load recognition to obtain a brain task load result.

[0013] In a second aspect, the present invention further proposes an application of the brain task load identification method as described in the first aspect in rehabilitation training, comprising:

[0014] A functional near-infrared light source and a detection probe are used to perform multi-channel detection on the brain of a user who is undergoing rehabilitation training, and light intensity signals of M channels are obtained, where M is a positive integer.

[0015] The light intensity signals of the M channels are preprocessed to convert the light intensity signals into hemoglobin concentration signals.

[0016] Extract topological information between functional near-infrared light sources, detection probes, and channels.

[0017] A grid image matrix of the hemoglobin concentration signal is constructed according to the topological information.

[0018] The grid image matrix is ​​input into a task load recognition model to perform task load recognition to obtain a brain task load result.

[0019] In a third aspect, the present invention further proposes a computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the operations performed by the brain task load identification method as described in the first aspect are implemented.

[0020] Compared with the prior art, the beneficial effects of the technical solution of the present invention are as follows: the present invention utilizes a functional near-infrared light source and a detection probe to perform multi-channel hemoglobin concentration signals on the brain of a user when the user is in an active state, and converts the multi-channel hemoglobin concentration signals into a grid image matrix according to the topological information between the functional near-infrared light source, the detection probe and the channels, which can link the hemoglobin concentration signals of all channels, so that the subsequent task load recognition model can learn the time information and spatial information of the hemoglobin concentration signal, ensuring the high accuracy of the classification of the task load recognition model, thereby improving the accuracy of brain task load recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flowchart of the brain task load identification method according to an embodiment of the present application.

[0022] Figure 2 This is a topological diagram consisting of a functional near-infrared light source, a detection probe and a channel according to an embodiment of the present application.

[0023] Figure 3A second grid image matrix mapped from the topological map of the embodiment of the present application.

[0024] Figure 4 This is a grid image matrix diagram of an embodiment of the present application.

[0025] Figure 5 This is a network structure diagram of the task load identification model of an embodiment of the present application. DETAILED DESCRIPTION

[0026] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0027] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0028] fNIRS technology uses the good scattering properties of the main components of blood for 600-900nm near-infrared light, thereby obtaining the changes in oxygenated hemoglobin and deoxygenated hemoglobin during brain activity. There are three core concepts of fNIRS: light source (S), detector (D), and channel (C). The light source emits near-infrared light, which is refracted by the skull and received by the detector. At this time, the fNIRS signal of the brain area between the light source and the detector can be obtained, which is called the channel.

[0029] Embodiment 1

[0030] See also Figure 1 , this embodiment proposes a method for identifying brain task load, comprising the following steps:

[0031] A functional near-infrared light source and a detection probe are used to perform multi-channel detection on the user's brain when the user is in an active state, and light intensity signals of M channels are obtained, where M is a positive integer.

[0032] The light intensity signals of the M channels are preprocessed to convert the light intensity signals into hemoglobin concentration signals.

[0033] Extract topological information between functional near-infrared light sources, detection probes, and channels.

[0034] A grid image matrix of the hemoglobin concentration signal is constructed according to the topological information.

[0035] The grid image matrix is ​​input into a task load recognition model to perform task load recognition to obtain a brain task load result.

[0036] The brain task load identification method proposed in this embodiment uses a functional near-infrared light source and a detection probe to collect multi-channel hemoglobin concentration signals from the user's brain when the user is in an active state, and converts the multi-channel hemoglobin concentration signals into a grid image matrix according to the topological information between the functional near-infrared light source, the detection probe and the channels. The hemoglobin concentration signals of all channels can be linked together, so that the subsequent task load identification model can learn the time information and spatial information of the hemoglobin concentration signal, ensuring the high accuracy of the classification of the task load identification model, thereby improving the accuracy of brain task load identification.

[0037] Embodiment 2

[0038] This embodiment improves the brain task load identification method proposed in the first embodiment. The method includes:

[0039] A functional near-infrared light source and a detection probe are used to perform multi-channel detection on the user's brain when the user is in an active state, and light intensity signals of M channels are obtained, where M is a positive integer.

[0040] In this embodiment, 23 functional near-infrared light sources and 15 detection probes are used to perform multi-channel detection of the brain, and light intensity signals of M channels can be obtained, M=47.

[0041] In the specific implementation process, a "ring toss" game was written, and the difficulty of the game was changed by setting the size of the ring. The user needs to control the ring with a game controller to catch the ball. The system collected fNIRS data of the prefrontal lobe and motor cortex of the subject's brain while playing the game.

[0042] The light intensity signals of the 47 channels are preprocessed to convert the light intensity signals into hemoglobin concentration signals. The hemoglobin concentration signals include oxygenated hemoglobin concentration signals and deoxygenated hemoglobin concentration signals.

[0043] In the specific implementation process, the light intensity signal is converted into the oxygenated hemoglobin concentration signal and the deoxygenated hemoglobin concentration signal using the improved Beer-Lambert law. The specific expressions are as follows:

[0044]

[0045] Among them, t i Indicates the current moment, △C HbO ( i ) represents the change in oxygenated hemoglobin concentration at the current moment, △C HbR ( i ) represents the change in the concentration of deoxyhemoglobin at the current moment, α Hbo ( 1 ) is the wavelength λ 1The extinction coefficient of oxygenated hemoglobin, α HbR ( 1 ) is the wavelength λ 1 The extinction coefficient of deoxyhemoglobin concentration at HbO ( 2 ) is the wavelength λ 2 The extinction coefficient of oxygenated hemoglobin, α HbR ( 2 ) is the wavelength λ 2 The extinction coefficient of deoxyhemoglobin concentration at i ; 1 ) is the wavelength λ at the current moment 1 The change in optical density at the point, △OD(t i ; 2 ) is the wavelength λ at the current moment 2 where l is the length of the path covered by the light and d(λ) is the differential path length factor at wavelength λ.

[0046] The oxygenated hemoglobin concentration signal and the deoxygenated hemoglobin concentration signal are filtered by a fourth-order zero-phase ButterWorth filter with a cutoff frequency of 0.3 Hz to remove high-frequency pseudo signals caused by respiration, blood pressure and heartbeat.

[0047] In addition, baseline correction: the change in oxygenated hemoglobin concentration and deoxygenated hemoglobin concentration during the task relative to the resting state (baseline) needs to be calculated for baseline correction.

[0048] After filtering the oxygenated hemoglobin concentration signal and the deoxygenated hemoglobin concentration signal through a filter, the method further includes:

[0049] All channels are normalized, and the expression is as follows:

[0050]

[0051] Extract topological information between functional near-infrared light sources, detection probes, and channels.

[0052] In this embodiment, the topological information is a topological diagram consisting of at least one functional near-infrared light source, at least one detection probe and M channels, M≥1; the at least one detection probe receives at least one adjacent functional near-infrared light source to form at least one channel. Figure 2 As shown, in the topological diagram of this embodiment, 23 functional near-infrared light sources (S 1 ,S 2 ,......,S 23 ) and 15 detection probes to perform multi-channel detection of the brain (D1 ,D 2 ,......,D 23 ), 47 channels of light intensity signals (C 1 ,C 2 ,......,C 23 ).

[0053] A grid image matrix of the hemoglobin concentration signal is constructed according to the topological information.

[0054] like Figure 2 and 3 As shown, in a specific implementation process, the grid image matrix of the hemoglobin concentration signal is constructed, and the specific steps include:

[0055] There are 47 non-missing channels in the topological map. For each moment when the brain is active, the detection values ​​corresponding to the 47 non-missing channels are plotted in the topological map. Insert into the pre-built first grid image matrix to get 47 detection values The second grid image matrix.

[0056] like Figure 4 As shown, there is a detection value The 47 grids are in grey font. The channel grids and non-channel grids with missing channel values ​​in the second grid image matrix are assigned values. After the remaining grids are assigned values, the final grid image matrix corresponding to each moment is obtained.

[0057] The principle of bilinear interpolation is as follows:

[0058] If we want to get the value of the unknown function f at point P = (,y), we assume that the known function f is at Q 11 =(x 1 ,y 1 ), Q 12 =(x 1 ,y 2 ),Q 21 =(x 2 ,y 1 ) and Q 22 =(x 2 ,y 2 ) (the 4 known points closest to P). First, perform linear interpolation in the x direction to obtain

[0059]

[0060]

[0061] Then interpolate in the y direction and get:

[0062]

[0063] The grid image matrix corresponding to each moment is input into the task load recognition model in time sequence to perform task load recognition and obtain the brain task load result.

[0064] like Figure 5 As shown, in this embodiment, the task load identification model includes N parallel convolutional neural networks, a Transformer Encoder, a first fully connected layer, a second fully connected layer and a third fully connected layer;

[0065] The grid image matrix is ​​respectively input into N convolutional neural networks for spatial feature extraction to obtain N feature maps including spatial information. In this embodiment, multiple convolutional neural networks ensure that the spatial information extracted at each time point is independent of each other.

[0066] The N feature maps including spatial information are subjected to temporal feature extraction and splicing processing by the Transformer Encoder to obtain feature maps including spatial information and temporal information;

[0067] The feature map including spatial information and temporal information is sequentially classified and processed by the first fully connected layer, the second fully connected layer and the third fully connected layer to obtain a brain task load result.

[0068] In the specific implementation process, assume that the input dimension of the three fully connected layers is a and the output dimension is b, then:

[0069] First fully connected layer: a = Transformer Encorder output dimension, b = 1024.

[0070] Second fully connected layer: a=1024, b=128.

[0071] The third fully connected layer: a=128, b=3.

[0072] In this way, the brain task load results include three categories: high load, medium load and low load.

[0073] In this embodiment, the objective function of the task load identification model is a cross entropy loss function, and its expression is as follows:

[0074]

[0075] Among them, y i is the true label value of sample i, y i ′ is the predicted label value of sample i, and n is the total number of samples.

[0076] The brain task load identification method proposed in this embodiment uses a functional near-infrared light source and a detection probe to collect multi-channel hemoglobin concentration signals from the user's brain when the user is in an active state, and converts the multi-channel hemoglobin concentration signals into a grid image matrix according to the topological information between the functional near-infrared light source, the detection probe and the channels. The hemoglobin concentration signals of all channels can be linked together, so that the subsequent task load identification model can learn the time information and spatial information of the hemoglobin concentration signal, ensuring the high accuracy of the classification of the task load identification model, thereby improving the accuracy of brain task load identification.

[0077] The present invention adopts the task form of target tracking, rather than the form of puzzle solving and memory, to identify the task load of users. It not only uses the fNIRS signal of the prefrontal lobe, but also combines the fNIRS signal of the motor cortex to assess the task load, which can provide an objective assessment standard in scenarios such as rehabilitation training.

[0078] By converting fNIRS data into a grid image matrix, all channels are linked, and the spatial information of the grid image matrix is ​​extracted using a convolutional neural network. The convolutional neural network learns the relationship between channels, and uses the bilinear interpolation method to reasonably fill in the case of missing channel values, which can reduce the recognition error rate. Then the output of the convolutional neural network is used as the input of the Transformer Encoder. While using spatial features, it can also use temporal features to improve classification accuracy. Compared with other neural networks, the Transformer Encoder can ensure high classification accuracy in long sequences, and using the Transformer Encoder alone instead of the Transformer can greatly reduce the complexity of the neural network, ensuring the high classification accuracy of the task load recognition model while improving the calculation speed, thereby improving the efficiency and accuracy of brain task recognition.

[0079] Embodiment 3

[0080] This embodiment proposes an application of the brain task load identification method described in Embodiment 2 in rehabilitation training, including:

[0081] Using a functional near-infrared light source and a detection probe to perform multi-channel detection on the brain of a user undergoing rehabilitation training, and obtaining light intensity signals of M channels, where M is a positive integer;

[0082] Preprocessing the light intensity signals of the M channels to convert the light intensity signals into hemoglobin concentration signals;

[0083] Extract topological information between functional near-infrared light sources, detection probes, and channels;

[0084] constructing a grid image matrix of the hemoglobin concentration signal according to the topological information;

[0085] The grid image matrix is ​​input into a task load recognition model to perform task load recognition to obtain a brain task load result.

[0086] The brain task load identification method proposed in this embodiment is applied in rehabilitation training, using a functional near-infrared light source and a detection probe to perform multi-channel hemoglobin concentration signals on the brain of a user undergoing rehabilitation training, and converting the multi-channel hemoglobin concentration signals into a grid image matrix based on the topological information between the functional near-infrared light source, the detection probe and the channels, which can link the hemoglobin concentration signals of all channels, so that the subsequent task load identification model can learn the time and space information of the hemoglobin concentration signal, ensuring the high accuracy of the classification of the task load identification model, thereby improving the accuracy of brain task load identification.

[0087] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0088] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for identifying brain task load, It is characterized in that include: A functional near-infrared light source and a detection probe are used to perform multi-channel detection of the user's brain when the user is active. M The light intensity signal of each channel, M is a positive integer; Regarding the M Preprocess the light intensity signals of each channel and convert them into hemoglobin concentration signals; Extract topological information between functional near-infrared light sources, detection probes, and channels; constructing a grid image matrix of the hemoglobin concentration signal according to the topological information; The topological information is provided by at least one functional near-infrared light source, at least one detection probe and M The topological diagram of the channels is shown in Figure 1. M ; The at least one detection probe receives at least one adjacent functional near-infrared light source to form at least one channel; The specific steps of constructing the grid image matrix of the hemoglobin concentration signal include: Constructing an initialized first grid image matrix; For each moment when the brain is active, the detection values ​​of the non-missing channels are converted according to the topological information. inserting into the first grid image matrix to obtain a second grid image matrix; Using bilinear interpolation, assign values ​​to channel grids and non-channel grids with missing channel values ​​in the second grid image matrix to obtain a final grid image matrix corresponding to each moment; Inputting the grid image matrix into a task load recognition model to perform task load recognition to obtain a brain task load result; The task load identification model includes N A parallel convolutional neural network, a TransformerEncoder, the first fully connected layer, the second fully connected layer, and the third fully connected layer; The grid image matrix is ​​input in sequence respectively N A convolutional neural network is used to extract spatial features, and we get N A feature map including spatial information; Said N The feature graphs including spatial information are subjected to temporal feature extraction and splicing processing by the Transformer Encoder to obtain feature graphs including spatial information and temporal information; The feature map including spatial information and temporal information is sequentially classified and processed by the first fully connected layer, the second fully connected layer and the third fully connected layer to obtain a brain task load result.

2. The method for identifying brain task load according to claim 1, It is characterized in that The hemoglobin concentration signal includes an oxygenated hemoglobin concentration signal and a deoxygenated hemoglobin concentration signal; the light intensity signal is converted into the oxygenated hemoglobin concentration signal and the deoxygenated hemoglobin concentration signal using the improved Beer-Lambert law, and the specific expression is as follows: in, Indicates the current moment, Indicates the change in oxygenated hemoglobin concentration at the current moment, Indicates the change in deoxyhemoglobin concentration at the current moment, For the wavelength The extinction coefficient of oxygenated hemoglobin is For the wavelength The extinction coefficient of deoxyhemoglobin concentration at For the wavelength The extinction coefficient of oxygenated hemoglobin is For the wavelength The extinction coefficient of deoxyhemoglobin concentration at is the wavelength at the current moment The change in optical density at is the wavelength at the current moment The change in optical density at is the length of the path covered by the light, Indicated in wavelength The differential path length factor at .

3. The method for identifying brain task load according to claim 1, It is characterized in that After converting the light intensity signal into a hemoglobin concentration signal, the method further comprises: filtering the hemoglobin concentration signal through a filter.

4. The method for identifying brain task load according to claim 3, It is characterized in that The filter is a fourth-order zero-phase ButterWorth filter with a cut-off frequency of 0.3 Hz.

5. The method for identifying brain task load according to claim 3, It is characterized in that After converting the light intensity signal into a hemoglobin concentration signal, the method further comprises: Regarding the M The channels are normalized, and the expression is as follows: in, express i Normalized value of the channel, express i The value of the channel at the current moment, express i The minimum value of the channel, express i Maximum value of the channel.

6. The method for identifying brain task load according to claim 1, It is characterized in that The objective function of the task load identification model is the cross entropy loss function, which is expressed as follows: in, For sample i The true label value of For sample i The predicted label value of is the total number of samples.

7. An application of the brain task load identification method according to any one of claims 1 to 6 in rehabilitation training, It is characterized in that include: The functional near-infrared light source and detection probe are used to conduct multi-channel detection of the brain of users undergoing rehabilitation training, and the M The light intensity signal of each channel, M is a positive integer; Regarding the M Preprocess the light intensity signals of each channel and convert them into hemoglobin concentration signals; Extract topological information between functional near-infrared light sources, detection probes, and channels; constructing a grid image matrix of the hemoglobin concentration signal according to the topological information; The grid image matrix is ​​input into a task load recognition model to perform task load recognition to obtain a brain task load result.

8. A computing device, It is characterized in that The computing device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the operations performed by the brain task load identification method as described in any one of claims 1 to 6 are implemented.

Images