Brain load level detection method based on fNIRS signal rapid identification

Abstract

The invention belongs to the technical field of mental load evaluation, and particularly relates to a mental load level detection method based on fNIRS signal rapid identification, which comprises the following steps of: performing dynamic modeling on fNIRS signals based on a dynamic system theory to obtain a nonlinear dynamic model for describing cognitive behaviors of human beings under different mental load levels; introducing a dynamic learning algorithm on the basis of the nonlinear dynamic model, and establishing a dynamic neural network identifier; inputting the training set into a dynamic neural network identifier for training, optimizing the dynamic neural network identifier, generating a weight matrix # imgabs0 #, inputting the test set into the trained dynamic neural network identifier to generate a weight matrix # imgabs1 #, and calculating a dynamic error between the weight matrix # imgabs2 # and the weight matrix # imgabs3 #; and quickly identifying a new mental load level according to the dynamic error when the mental load level changes. According to the invention, clinical interpretable mental workload level detection is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of mental workload assessment, and specifically relates to a mental workload level detection method based on rapid identification of fNIRS signals. Background Art

[0002] Mental workload level is an important indicator that reflects the human cognitive state in human-computer interaction, and plays a vital role in monitoring human psychological state and cognitive behavior. For example, in high-risk task scenarios such as driving a vehicle or an airplane, evaluating the driver's mental workload level can avoid potential driving accidents caused by excessive cognitive load on the driver and operational errors when dealing with complex road conditions and vehicle conditions. In medical applications, mental workload level assessment not only helps to evaluate the participation of neurocognition in rehabilitation training and improve rehabilitation effects; it can also be used to evaluate the cognitive functions of patients with major depression and other mental illnesses, providing a reliable and accurate basis for disease diagnosis. Therefore, it is crucial to develop an objective and effective method for detecting mental workload levels.

[0003] In cognitive research, functional near-infrared spectroscopy (fNIRS) is widely used to measure brain workload during cognitive tasks. Its principle is to emit near-infrared light of different wavelengths to the brain area, then receive the light waves absorbed and scattered by hemoglobin, and then convert it through the modified Lambert-Beer formula to obtain the relative change of hemoglobin concentration in the corresponding area. Based on this principle, functional near-infrared spectroscopy can detect the activation state of the frontal cortex related to visual-spatial working memory (WM). However, most of the existing fNIRS-based mental workload level classification methods are mainly aimed at the binary classification of high and low mental workload levels. Moreover, these methods have certain limitations in classification accuracy and interpretability. This is because they fail to accurately extract the characteristics that describe human cognitive behavior, such as cerebral hemodynamic characteristics, and lack medical interpretability, so they are difficult to apply in clinical practice.

[0004] Dynamical Systems Theory (DST) is a potential tool for extracting characteristics that describe human cognition and behavior. The theory describes the state of human behavior over time by establishing mathematical expressions of dynamic systems. Specifically, it uses differential equations and difference equations to simulate complex systems, which can capture and quantify the development process of human behavior and explain the observed human patterns from a mathematical perspective. Existing literature has shown that the functional state of human brain regions is related to the neural dynamics corresponding to cognitive behavior. This association provides a potential theoretical tool, that is, the dynamics of fNIRS signals from human cognitive behavior can be used to characterize the level of human mental workload. However, most of the existing literature and inventions only provide theoretical and mathematical models, but in practical applications, these models are almost completely unknown and difficult to put into practical use.

[0005] Dynamic Learning (DL) is a new type of machine learning scheme, which is mainly used to identify unknown dynamic features of nonlinear dynamic systems. It provides a unified deterministic framework for the dynamic patterns of effective human cognitive behavior, defining their similarities and achieving rapid identification. The extraction of dynamic features of human cognitive behavior variables is a key step in the pattern recognition and classification of human mental workload levels.

[0006] In summary, in the detection of mental workload level based on fNIRS signals, it has become a technical problem to be solved at present to introduce dynamic learning algorithms and dynamic system theory to utilize their advantages to achieve rapid detection of mental workload level and clinical interpretability. Summary of the invention

[0007] The purpose of the present invention is to provide a method for detecting mental workload levels based on rapid identification of fNIRS signals, thereby realizing clinically interpretable mental workload level detection and helping clinicians monitor changes in patients' mental workload levels in real time, thus providing an effective tool for diagnosis of the condition and evaluation of rehabilitation effects of patients with mental illnesses such as major depression.

[0008] To achieve the above object, the present invention adopts the following technical solution:

[0009] A method for detecting mental workload level based on rapid identification of fNIRS signals, comprising the following steps;

[0010] Step 1: Obtain fNIRS signals of subjects in different psychological states;

[0011] Step 2: preprocess the acquired fNIRS signals and divide them into training set and test set;

[0012] Step 3: Based on the dynamic system theory, the fNIRS signal is dynamically modeled to obtain a nonlinear dynamic model for describing human cognitive behavior under different mental workload levels; a dynamic learning algorithm is introduced to establish a dynamic neural network identifier based on the nonlinear dynamic model to achieve dynamic feature extraction of the fNIRS signal;

[0013] Step 4: Input the training set into the dynamic neural network identifier, use the training set to optimize the dynamic neural network identifier, and generate a weight matrix Input the test set data into the trained dynamic neural network identifier and use the test set to generate the weight matrix Calculate the weight matrix With the weight matrix The dynamic error between them is calculated, and the mode with the smallest error is selected as the mental workload level corresponding to the test data, and it is output as the final mental workload level.

[0014] Furthermore, the process of performing dynamic modeling on the fNIRS signal based on the dynamic system theory to obtain a nonlinear dynamic model for describing human cognitive behavior under different mental workload levels includes:

[0015] Based on the differential equations used to explain the one-dimensional cerebral hemodynamics and oxygen transport model, the dynamic model of the fNIRS signal is derived as follows:

[0016]

[0017] Where m represents the concentration of free oxygen per unit volume; r represents the concentration of oxygenated hemoglobin; z represents the brain channel information; D, M and F are all dynamic model parameters, where D pl represents the diffusion coefficient of oxygen in plasma; M represents the cellular metabolic consumption, which is defined as the oxygen consumption rate per unit volume per unit time; F represents the oxygen flux diffused from the lumen to the tissue surface per unit time;

[0018] Based on the different values ​​of model parameters under different mental workload levels, the dynamic model of fNIRS signals is rewritten as the following equation to simulate the nonlinear dynamic model that describes human cognitive behavior under different mental workload levels, as shown in the following equation:

[0019]

[0020] Among them, f n represents the cerebral hemodynamics of individuals under different mental workload levels, also known as mental workload levels; n represents the existence of n different mental workload levels, is the first-order derivative of r, represents partial differential, and u represents the neuron activation strength.

[0021] Furthermore, the dynamic learning method is based on a radial basis function neural network dynamic learning algorithm.

[0022] Furthermore, the process of introducing a dynamic learning algorithm based on a radial basis function neural network and establishing a dynamic neural network identifier based on a nonlinear dynamic model to extract dynamic features of fNIRS signals includes:

[0023] Using the Euler sampling method and setting the sampling rate, a discrete-time system model is obtained to approximate the nonlinear dynamics model. The discrete-time system model is shown in the following formula:

[0024] x(k+1)=x(k)+T s f n (x(k))=v i (x(k))

[0025] Among them, x represents the system state vector, n represents the different human mental load levels through the human-computer interaction process, and f n (x(k)) represents the unknown system dynamics, v i (x(k))(i=0,1,2) is a dynamic function used to represent different levels of human mental workload, f n (x(k)) and v i (x(k)) are all unknown and need to be identified, T s is the system sampling time.

[0026] A dynamic learning algorithm based on radial basis function neural network (RBFNN) is used to approximate the v in discrete time system models. i (x(k)), as shown below:

[0027]

[0028] Where x = [x 1 ,x 2 ,…,x n ] T ∈R n represents the input vector of RBFNN, W = [W 1 ,W 2 ,…,W m ] T ∈R m represents the weight vector from the hidden layer to the output layer of the RBFNN, m>1 represents the number of neuron nodes in the RBFNN, S(x)=[s 1 (‖x-ξ 1 ‖),s 2 (‖x-ξ 2 ‖),…,sm (‖x-ξ m ‖)] T represents the regression vector composed of radial basis functions, s i (·)(i=1,…,m) is the nonlinear mapping function of the hidden layer-radial basis function, using the Gaussian function: ξ i (i=1,…,m) represents the neuron (node) position in the RBFNN network, and η represents the width of the receptive field;

[0029] Using the weight matrix To characterize the dynamic function of identifying the mental workload level based on the radial basis function neural network, the formula is as follows:

[0030] based on Construct a dynamic neural network recognizer as shown below:

[0031]

[0032] in, represents the state of the dynamic neural network recognizer, α i represents the recognizer gain, x fi [k] represents the connection data of oxygenated hemoglobin concentration saturation, represents a dynamic neural network identifier for modeling the dynamic information of the cerebral hemodynamic system, Represents the weight estimation of the dynamic neural network recognizer; taking the training set as input, the weight of the dynamic neural network recognizer is continuously updated; the weight update rule of the dynamic neural network recognizer is as follows:

[0033]

[0034] in, represents the tracking error signal of the dynamic neural network identifier, and γ represents the learning rate;

[0035] After multiple iterations and updates, a trained neural network identifier is obtained, thereby realizing the real-time extraction of dynamic features of fNIRS signals.

[0036] Furthermore, in step 4, the average L1 norm is also used to identify the weight matrix of the dynamic neural network identifier and The error between them is processed by dimensionality reduction to improve the recognition accuracy.

[0037] Furthermore, the fNIRS signals under different psychological states are obtained by collecting experimental paradigms.

[0038] The mental workload level detection method of the present invention is to perform dynamic modeling on the fNIRS signal based on the dynamic system theory (DST) to obtain a nonlinear dynamic model that describes the functional state of the working memory brain area under the fNIRS signal, and the model can describe the cognitive behavior of humans under different mental workload levels. Then, a dynamic learning algorithm (DL) is introduced to establish a dynamic neural network identifier based on the nonlinear dynamic model to complete the cerebral hemodynamic modeling of the fNIRS signal, thereby realizing the dynamic feature extraction of the fNIRS signal, and the extracted dynamic features are interpretable due to DST, and are similar to the characteristics of the human mental workload level. The fNIRS signals collected from the subjects under different psychological states are divided into a training set and a test set; the training set is used to optimize the dynamic neural network identifier and generate a weight matrix The test set is directly input into the trained dynamic neural network identifier to generate the weight matrix Calculate the weight matrix With the weight matrix The dynamic error between the two is used to quickly identify the new mental workload level when the mental workload level changes.

[0039] The method for detecting mental workload levels based on rapid identification of fNIRS signals realizes clinically interpretable mental workload level detection, as the extracted dynamic features are based on a model constructed on the basis of dynamic system theory (DST). At the same time, it can help clinicians monitor the changes in patients' mental workload levels in real time, and provides an effective tool for the diagnosis of the condition and rehabilitation effect evaluation of patients with mental illnesses such as major depression. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A method for quickly identifying mental workload levels based on fNIRS signals is provided as an embodiment;

[0041] Figure 2 N-back paradigm diagram used in the embodiment;

[0042] Figure 3 The flowchart of the mental workload level detection in the embodiment is as follows;

[0043] Figure 4 This is a diagram showing the dynamic learning convergence effect of fNIRS signals in the embodiment;

[0044] Figure 5 The confusion matrix diagram of the dynamic features extracted in the embodiment. DETAILED DESCRIPTION

[0045] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and embodiments.

[0046] This embodiment provides a system for detecting mental workload level based on rapid identification of fNIRS signals, which is composed of a dynamic pattern estimator. Figure 1 As shown, it includes: a paradigm presentation module, a data acquisition and preprocessing module, a dynamic feature extraction module and a recognition module. The data acquisition and preprocessing module is connected to the paradigm presentation module, the dynamic feature extraction module is connected to the data acquisition and preprocessing module, and the recognition module is connected to the dynamic feature extraction module.

[0047] The paradigm presentation module is used to present a guiding paradigm to the patient, instructing the patient to activate the brain area related to cognitive function through task behavior. The paradigm is configured as a psychological experimental paradigm for working memory ability test, which is produced by E-PRIME software. Figure 2 As shown, the paradigm is based on the classic psychological task N-back experiment, which tests the patient's attention and memory level. The experimental paradigm is divided into three groups according to difficulty: 0-back, 1-back, and 2-back. Participants sit in front of a computer and observe the sequence of numbers or letters presented on the screen. In each task, each letter is presented for half a second, and the screen is blank for two seconds before the next letter appears. Participants are required to judge as quickly and accurately as possible whether the currently presented number or letter is the same as the number or letter before n positions. If the same, press the "yes" key on the keyboard; if different, press the "no" key on the keyboard. The duration of a single experimental trial is 2.5s, including a prompt time of 0.5s and a rest time of 2s. The E-PRIME experimental paradigm consists of three rounds, each of which contains 3 blocks, and each block contains 30 experimental trials.

[0048] The data acquisition and preprocessing module is composed of a data acquisition unit and a preprocessing unit. The data acquisition unit is used to collect fNIRS signals of different psychological states of the subjects under different psychological experimental paradigms and send them to the data preprocessing unit; the data preprocessing unit is used to preprocess the collected fNIRS signals and divide the preprocessed fNIRS signals into a training set and a test set. The data acquisition unit collects fNIRS signals of different psychological states under three implementation paradigms through a near-infrared brain imaging device with a function of collecting fNIRS signals in the frontal lobe area. The near-infrared brain imaging device with a function of collecting fNIRS signals in the frontal lobe area includes: a multi-channel head-mounted device, and a terminal setting equipped with fNIRS signal acquisition software. The fNIRS signal acquisition and processing software can perform real-time presentation and signal processing, and has an event marking function; the head-mounted device is wirelessly connected to a notebook to transmit the collected fNIRS signals to the terminal notebook in real time.

[0049] The dynamic feature extraction module includes a dynamic modeling unit and a dynamic feature extraction unit. The dynamic modeling unit is used to construct a nonlinear dynamic model that describes the functional state of the working memory brain area under the fNIRS signal. The dynamic feature extraction unit is used to extract the dynamic features of the fNIRS signal. The dynamic feature extraction unit is a dynamic neural network identifier constructed by introducing a dynamic algorithm into the nonlinear dynamic model; the dynamic neural network identifier is optimized through a training set and generates a weight matrix that characterizes the dynamic features of the fNIRS signal. The dynamic neural network identifier generates a weight matrix that characterizes the dynamic characteristics of fNIRS signals through testing

[0050] The recognition module is used to calculate the weight matrix With the weight matrix The dynamic error between them is calculated, and the mode with the smallest error is selected as the mental workload level corresponding to the test data, and it is output as the final mental workload level, thereby completing the mental workload level test.

[0051] According to the above-mentioned mental workload level detection system based on rapid identification of fNIRS signals, a mental workload level detection method based on rapid identification of fNIRS signals is provided, comprising the steps of:

[0052] Step 1: Obtain fNIRS signals of subjects in different psychological states. In this embodiment, the fNIRS signals of five healthy subjects at rest and in N-back (N=0, 1, 2) tasks are divided into four mental workload levels.

[0053] Step 2: Preprocess the collected fNIRS signals. The preprocessing process includes:

[0054] Step 2.1: The collected fNIRS signals were preprocessed by: channel selection, filtering, epochs segmentation, hemoglobin concentration conversion, etc., where:

[0055] The channel was selected from the multi-depth hierarchical channels covering the prefrontal brain area, and the channel with a depth of 3 cm was specifically selected to obtain the fNIRS signal closest to the depth of the cerebral cortex area.

[0056] The filtering operation refers to filtering out noise interference such as motion artifacts, eye movements, and heartbeats in the signal and retaining the fNIRS signal data in a specific frequency band. The filtering operation uses a Butterworth bandpass filter with an upper limit frequency of 0.5 Hz and a lower limit frequency of 0.001 Hz.

[0057] Epochs segmentation means: the collected fNIRS signals are divided into fNIRS signal data of different experimental times according to the label points. The data from 0s to 30s after the label point corresponding to the task is taken as the fNIRS signal data of a single experiment.

[0058] The hemoglobin concentration conversion operation refers to: using the fNIRS acquisition device to continuously collect the change in infrared light intensity before and after transmission and reception, and using the modified Beer-Lambert relationship equation to convert the change in infrared light intensity into the relative change in oxygenated hemoglobin concentration and deoxygenated hemoglobin concentration. The operation process is shown in formula (1):

[0059]

[0060] In formula (1), d is the distance between the light source and the detector; a HbO HbO(λ i ) and a HbR HbR(λ i ) is the wavelength λ i The concentration of oxygenated hemoglobin and the extinction coefficient of deoxygenated hemoglobin; ΔOD(Δt,λ i ) is the wavelength λ after sampling time i The optical density change at DPF(λ i ) is the wavelength λ i The differential path length factor (i=0,1); ΔHbO and ΔHbR are the relative changes in oxygenated hemoglobin concentration and deoxygenated hemoglobin concentration, respectively; Δt is the time interval between two adjacent samplings. In addition, this method can be applied to various types of hemoglobin concentration data such as oxygenated hemoglobin concentration and deoxygenated hemoglobin concentration. However, in order to better illustrate the method of this embodiment, the fNIRS data described below are unified as oxygenated hemoglobin concentration data obtained based on fNIRS signals.

[0061] Step 2.2: After preprocessing the fNIRS signal, all channel signals are normalized using minimum-maximum scaling. This can speed up the gradient descent in the subsequent dynamic feature extraction process to achieve faster convergence. The normalization operation is expressed as shown in formula (1):

[0062]

[0063] In formula (2), x is the input signal of each channel, x min is the minimum value of the input signal, x max is the maximum value of the input signal, x ′ is the output signal, that is, the entire preprocessed data. They have M channel signals respectively, and each channel signal has N sampling points, and M and N are positive integers.

[0064] Step 2.3: Divide the normalized fNIRS signals into training set and test set in a ratio of 7:3.

[0065] Step 3: Construct a nonlinear dynamic model, and introduce a dynamic learning algorithm to establish a dynamic neural network identifier based on the nonlinear dynamic model. Figure 3 As shown:

[0066] Step 3.1, establish a dynamic theoretical model. In terms of dynamic modeling operations, when individuals perform N-back cognitive tasks of different difficulty levels, the oxygenated hemoglobin concentration data collected by the FNIRS signal will reflect different mental workload levels. In order to accurately assess the mental workload of an individual based on the fNIRS signal, this embodiment uses dynamic system theory to mathematically model the mental workload level of an individual. When modeling the mental workload level of an individual, the operation process is as follows:

[0067] Step 3.1.1. Since functional near-infrared spectroscopy (FNIRS) is a method for measuring oxygenated hemoglobin concentration based on the principle of cerebral hemodynamics, it is necessary to first combine the differential equations used to explain the one-dimensional cerebral hemodynamics and oxygen transport model to derive the dynamic model as shown in formula (3):

[0068]

[0069] In formula (3),

[0070] m represents the concentration of free oxygen per unit volume; r represents the concentration of oxygenated hemoglobin; z represents the brain channel information; D, M and F are all dynamic model parameters, where D pl represents the diffusion coefficient of oxygen in plasma; M represents cellular metabolic consumption, which is defined as the oxygen consumption rate per unit volume per unit time; F represents the oxygen flux diffused from the lumen to the tissue surface per unit time.

[0071] Step 3.1.2, under different mental workload levels, the model parameters will present different values. This embodiment is based on n = 3 types of experiments to carry out pattern recognition work for n mental workload levels. In view of this, the dynamic model parameters D, M and F also have n different values ​​accordingly to adapt to different mental workload levels. Based on this, the dynamic model of the fNIRS signal is rewritten as the following equation to simulate the nonlinear dynamic model describing human cognitive behavior under different mental workload levels, and the dynamic model of the fNIRS signal is rewritten as the equation described in formula (4) to simulate the nonlinear dynamic model describing human cognitive behavior under different mental workload levels, that is, formula (4) is used to describe the cognitive processes involved in professionals (such as drivers and pilots) when handling tasks of different difficulty levels, as well as the abnormal changes in mental workload levels in cognitive behaviors of patients with mental illnesses such as depression:

[0072]

[0073] In formula (4), f n represents the cerebral hemodynamics of individuals under different mental workload levels, also known as mental workload levels; n represents the existence of n different mental workload levels, is the first-order derivative of r, represents partial differential, and u represents the neuron activation strength.

[0074] In the process of human-computer interaction, mental workload is affected by task difficulty, task volume and interaction duration, thus changing the dynamic characteristics of the oxygenated hemoglobin concentration state vector x obtained by the data acquisition device. Therefore, by solving f in formula (4), n By using the specific expression of x, we can deduce the mental workload level corresponding to the dynamic change of x over time.

[0075] Step 3.2: Establish a dynamic neural network identifier. Due to the lack of specific parameters, f in formula (4) n It is impossible to directly solve it through mathematical methods, so it is impossible to directly use the model represented by formula (4) to extract features of the mental workload level. To address this problem, this embodiment adopts a method that combines dynamic system theory with dynamic learning to obtain a mathematical model of the mental workload level; that is, by introducing a dynamic learning method, a dynamic neural network identifier is established on the basis of a nonlinear dynamic model. The dynamic learning method is a machine learning algorithm designed for complex dynamic systems. It can use locally accurate neural networks to identify the dynamic characteristics of nonlinear dynamic models on periodic and regression trajectories. For the nonlinear dynamic model represented by formula (4), by using the Euler sampling method and setting the sampling rate to 10 Hz, the following discrete time system model can be obtained to approximate the system, as shown in formula (5):

[0076] x(k+1)=x(k)+T s f n (x(k))=v i (x(k))#(5)

[0077] In formula (5), x is the system state vector, n is the different human mental workload levels during the human-computer interaction process, and f n (x(k)) represents the unknown system dynamics, v i (x(k))(i=0,1,2) is a dynamic function used to represent different levels of human mental workload, f n (x(k)) and v i (x(k)) are all unknown and need to be identified, T s is the system sampling time. In order to accurately determine the unknown dynamics, the dynamic learning method of this embodiment adopts a dynamic learning algorithm based on radial basis function neural network (RBFNN). The specific operation process includes:

[0078] Step 3.2.1: Use RBFNN to approximate v in formula (5) i (x(k))(i=0,1), as shown in formula (6):

[0079]

[0080] In formula (6), x = [x 1 ,x 2 ,…,x n ] T ∈R n represents the input vector of RBFNN, W = [W 1 ,W 2 ,…,W m ] T ∈R m represents the weight vector from the hidden layer to the output layer of the RBFNN, m>1 represents the number of neuron nodes in the RBFNN, S(x)=[s 1 (‖x-ξ 1 ‖),s 2 (‖x-ξ 2 ‖),…,s m (‖x-ξ m ‖)] T represents the regression vector composed of radial basis functions, s i (·)(i=1,…,m) is the nonlinear mapping function of the hidden layer-radial basis function, using the Gaussian function: ξ i (i=1,…,m) represents the position of the neuron (node) in the network, and η represents the width of the receptive field.

[0081] Step 3.2.2: Use the weight matrix To characterize the dynamic function of identifying the mental workload level based on the radial basis function neural network, as shown in formula (7):

[0082]

[0083] In formula (7), x(k) represents the concentration of oxygenated hemoglobin, v n (x(k)) represents the kinetic features from the fNIRS signal.

[0084] Step 3.2.3: In order to use the dynamic learning method to learn the intrinsic dynamics v of the oxygenated hemoglobin concentration x(k) i (x(k)) is locally accurately modeled and a dynamic neural network recognizer based on sampled data is constructed, as shown in formula (8):

[0085]

[0086] In formula (8), represents the state of the dynamic neural network recognizer, α i represents the recognizer gain, x fi [k] represents the connection data of oxygenated hemoglobin concentration saturation, represents a radial basis function neural network used to model the dynamic information of the cerebral hemodynamic system, Represents the weight estimates for a radial basis function neural network.

[0087] Step 4: Using the dynamic neural network identifier obtained in step 3, the mental workload level detection based on the rapid identification of fNIRS signals is realized; the realization process includes:

[0088] Step 4.1: Taking the training set as input, the weight of the dynamic neural network recognizer is continuously updated. The weight update rule of the radial basis function neural network is performed as shown in formula (9):

[0089]

[0090] In formula (9), represents the tracking error signal of the dynamic neural network identifier, and γ represents the learning rate.

[0091] After the preset iteration update, a trained neural network identifier is obtained. The weight matrix in the trained RBFNN used in this embodiment is Reflects the unknown dynamics of the brain workload-cerebral hemodynamic system i (x(k)); where represents the average estimated value of the neural network weights over a period of time after the transient process. Since the dynamic learning algorithm converges exponentially, the intrinsic dynamics of the kinetic model obtained in step 3 can be accurately expressed in the form of a convergence constant. It can realize the real-time extraction of fNIRS signal dynamics and facilitate subsequent pattern recognition.

[0092] Step 4.2: Input the test set data into the trained dynamic neural network identifier and use the test set to generate the weight matrix

[0093] Step 4.3: Calculate the weight matrix With the weight matrix The dynamic error between i [k], and select the mode with the smallest error as the mental workload level corresponding to the test data, and output it as the final mental workload level. i The calculation formula of [k] is shown in formula (10):

[0094]

[0095] In this process, in order to accurately distinguish the dynamic modes corresponding to different data, this embodiment uses the average L1 norm to identify the weight matrix of the dynamic neural network identifier. and The error between them is processed by dimensionality reduction. This step can be shown as formula (11):

[0096]

[0097] In formula (11), express The average L1 norm of is the dynamic error data calculated in this example, t 0 is the specified time point after the weights converge, T is the time period for calculating the norm, ∫ represents the integration of the function, and dt represents t as the integration variable.

[0098] The average L1 norm of the pattern with the smallest error corresponds to a training pattern whose dynamic characteristics obtained based on the training set are most similar to the dynamic characteristics obtained based on the test set.

[0099] Finally, it should be noted that, for the different mental workload levels corresponding to the collected fNIRS signals in this embodiment, after the training set and the test set are divided, when the training set data completes the training of the training mode, the test set data does not need to be input into the same network for learning, but is directly generated with the training mode weight matrix In the real-time updated weight matrix And constantly compare it with training modes corresponding to different levels of mental workload.

[0100] Figure 4 is a graph showing the dynamic learning convergence effect of fNIRS signals in the embodiment; Figure 4 It can be seen that in the method for detecting mental workload level based on rapid identification of fNIRS signals provided in this embodiment, the processing effects of fNIRS signals of all channels have reached convergence, which also shows that the dynamic pattern estimator is effective and interpretable.

[0101] Table 1 Comparison of recognition effects of this embodiment and six existing mental workload level recognition methods

[0102]

[0103]

[0104] As shown in Table 1, the average classification accuracy of the four-category mental workload level rapid identification is 92.7%, which is superior to the existing mental workload level identification method in terms of identification accuracy and identification time. Figure 5 , which demonstrates the effectiveness of the mental workload level detection method based on rapid identification of fNIRS signals.

[0105] Figure 5 The confusion matrix diagram of the dynamic characteristics extracted in the embodiment. Figure 4 and Figure 5 It can be seen that the processing effect of the method proposed in the present invention on the fNIRS signals of all channels in the example has reached convergence. Figure 4 As shown, it is proved that the dynamic pattern estimator is effectively constructed and interpretable.

[0106] In summary, the method for detecting mental workload level based on rapid identification of fNIRS signals in this embodiment eliminates the model processing time during the identification process, and the dynamic characteristic error e i The calculation of [k] does not rely on complete fragment data, and can be classified in real time by reading data step by step, thereby realizing rapid identification of different mental workload levels. In addition, in online experiments, when the mental workload level changes, the real-time acquisition data is read step by step and the dynamic error is calculated according to the above formula. It is also possible to detect the change of the dynamic error of the input recognition data in real time and quickly identify the change of mental workload level.

Claims

1. A method for detecting mental workload level based on rapid identification of fNIRS signals, characterized in that: The steps include: Step 1: Obtain fNIRS signals of subjects in different psychological states; Step 2: preprocess the collected fNIRS signals and divide them into training set and test set; Step 3: Based on the dynamic system theory, the fNIRS signal is dynamically modeled to obtain a nonlinear dynamic model for describing human cognitive behavior under different mental workload levels; a dynamic learning algorithm is introduced to establish a dynamic neural network identifier based on the nonlinear dynamic model to achieve dynamic feature extraction of the fNIRS signal; Step 4: Input the training set into the dynamic neural network identifier, use the training set to optimize the dynamic neural network identifier, and generate a weight matrix Input the test set data into the trained dynamic neural network identifier and use the test set to generate the weight matrix Calculate the weight matrix With the weight matrix The dynamic error between them is calculated, and the mode with the smallest error is selected as the mental workload level corresponding to the test data, and it is output as the final mental workload level.

2. The method according to claim 1, characterized in that The process of performing dynamic modeling on fNIRS signals based on dynamic system theory to obtain a nonlinear dynamic model for describing human cognitive behavior under different mental workload levels includes: Based on the differential equations used to explain the one-dimensional cerebral hemodynamics and oxygen transport model, the dynamic model of the fNIRS signal is derived as follows: Where m represents the concentration of free oxygen per unit volume; r represents the concentration of oxygenated hemoglobin; z represents the brain channel information; D, M and F are all dynamic model parameters, where D pl represents the diffusion coefficient of oxygen in plasma; M represents the cellular metabolic consumption, which is defined as the oxygen consumption rate per unit volume per unit time; F represents the oxygen flux diffused from the lumen to the tissue surface per unit time; Based on the different values ​​of model parameters under different mental workload levels, the dynamic model of fNIRS signals is rewritten as the following equation to simulate the nonlinear dynamic model that describes human cognitive behavior under different mental workload levels, as shown in the following equation: Among them, f n represents the cerebral hemodynamics of individuals under different mental workload levels, also known as mental workload levels; n represents the existence of n different mental workload levels, is the first-order derivative of r, represents partial differential, and u represents the neuron activation strength.

3. The method according to claim 2, characterized in that The dynamic learning method is a dynamic learning algorithm based on a radial basis function neural network.

4. The method according to claim 3, characterized in that The process of introducing a dynamic learning algorithm based on a radial basis function neural network and establishing a dynamic neural network identifier based on a nonlinear dynamic model to realize the extraction of dynamic features of fNIRS signals includes: Using the Euler sampling method and setting the sampling rate, a discrete-time system model is obtained to approximate the nonlinear dynamics model. The discrete-time system model is shown in the following formula: x(k+1)=x(k)+T s f n (x(k))=v i (x(k)) Among them, x represents the system state vector, n represents the different human mental load levels through the human-computer interaction process, and f n (x(k)) represents the unknown system dynamics, v i (x(k))(i=0,1,2) is a dynamic function used to represent different levels of human mental workload, f n (x(k)) and v i (x(k)) are all unknown and need to be identified, T s is the system sampling time. A dynamic learning algorithm based on radial basis function neural network (RBFNN) is used to approximate the v in discrete time system models. i (x(k)), as shown below: Where x=[x1,x2,…,x n ] T ∈R n Represents the input vector of RBFNN, W=[W1,W2,…,W m ] T ∈R m represents the weight vector from the hidden layer to the output layer of the RBFNN, m>1 represents the number of neuron nodes in the RBFNN, S(x)=[s1(‖x-ξ1‖),s2(‖x-ξ2‖),…,s m (‖x-ξ m ‖)] T represents the regression vector composed of radial basis functions, s i (·)(i=1,…,m) is the nonlinear mapping function of the hidden layer-radial basis function, using the Gaussian function: represents the neuron (node) position in the RBFNN network, and η represents the width of the receptive field; Using the weight matrix To characterize the dynamic function of identifying the mental workload level based on the radial basis function neural network, the formula is as follows: based on Construct a dynamic neural network recognizer as shown below: in, represents the state of the dynamic neural network recognizer, α i represents the recognizer gain, x fi [k] represents the connection data of oxygenated hemoglobin concentration saturation, represents a dynamic neural network identifier for modeling the dynamic information of the cerebral hemodynamic system, Represents the weight estimation of the dynamic neural network recognizer; taking the training set as input, the weight of the dynamic neural network recognizer is continuously updated; the weight update rule of the dynamic neural network recognizer is as shown in the following formula: in, represents the tracking error signal of the dynamic neural network identifier, and γ represents the learning rate; After multiple iterations and updates, a trained neural network recognizer is obtained, thereby realizing the real-time extraction of dynamic features of fNIRS signals.

5. The method according to claim 1, characterized in that In step 4, the average L1 norm is also used to identify the weight matrix of the dynamic neural network identifier and The error between them is processed by dimensionality reduction to improve the recognition accuracy.

6. The method according to claim 1, characterized in that The fNIRS signals under the different psychological states are obtained by collecting experimental paradigms.

Images