An intention recognition method based on near-infrared brain function imaging technology

Abstract

This invention discloses an intention recognition method based on near-infrared brain functional imaging technology, comprising the following steps: S1. Acquiring raw multi-channel light intensity signals, preprocessing the raw multi-channel light intensity signals to obtain multi-channel blood oxygenation signals; S2. Extracting brain functional connectivity features and brain network topology features from the multi-channel blood oxygenation signals, and splicing the multi-channel blood oxygenation signals, brain functional connectivity features, and brain network topology features to obtain graph theory fusion features; S3. Performing global temporal modeling on the multi-channel blood oxygenation signals to capture long-distance dependencies between different time points and obtain temporal correlation features; S4. Performing comprehensive discrimination on the temporal correlation features and graph theory fusion features, outputting the corresponding probability of the motor imagery task category, and realizing automatic recognition of different motor intentions.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interface signal processing and intelligent information processing technology, specifically to a method for recognizing motor imagery intentions based on functional near-infrared spectral signals. Background Technology

[0002] Motor imagery brain-computer interface (MI-BCI) has become a hot topic in brain-computer interface, neural engineering, and rehabilitation medicine research because it enables control of external devices by decoding brain motor cortex activity without the need for peripheral nerves and muscles. As an important branch of non-invasive brain-computer interfaces, MI-BCI has driven the transformation of human-computer interaction from traditional peripheral device control to direct control based on brain signals, significantly promoting the development of intelligent human-computer interaction and neurorehabilitation technologies. For patients with motor dysfunction, such as stroke or spinal cord injury patients, MI-BCI holds promise for promoting neural remodeling and motor function recovery by recognizing motor-related brain region activity and providing neural feedback. Furthermore, the combination of MI-BCI and artificial intelligence models helps explore the human brain's motor control mechanisms and promotes the development of brain-inspired intelligent algorithms.

[0003] In recent years, the development of motor imagery brain-computer interfaces (MI-BCI) has primarily relied on electroencephalography (EEG), making EEG-based MI-BCI the current mainstream technology. However, related research indicates that the human motor imagery process involves a large-scale distributed brain network composed of multiple brain regions, including the sensorimotor cortex, supplementary motor area, and parietal lobe. Therefore, the current trend in the development of motor imagery brain-computer interfaces (MI-BCI) is to leverage the interpretability of deep learning models to explore the importance of motor-related regions in the brain and their contribution to decoding motor intentions.

[0004] Functional near-infrared spectroscopy (fNIRS) is an effective tool for capturing cortical activity. Certain characteristics of fNIRS make it suitable for motor imagery brain-computer interfaces (BCIs). First, fNIRS can be used to detect motor-related brain regions on the surface of the cerebral cortex, such as the primary motor cortex, supplementary motor area, and premotor cortex, which play crucial roles in motor imagery. Second, fNIRS has stronger resistance to motor and electromagnetic artifacts; in practical applications, fNIRS is suitable for children and environments with high electromagnetic interference; and fNIRS, by detecting changes in hemoglobin using near-infrared light, inherently possesses strong noise resistance. However, due to individual differences among subjects, coupled with the instability of brain activity and low signal-to-noise ratio, physiological signals can produce varying results. These factors may limit classification performance. Therefore, extracting useful features from complex fNIRS signals to improve the classification accuracy of MI-based BCI systems remains a significant challenge.

[0005] In recent years, deep learning methods have gradually become the mainstream approach for functional near-infrared spectroscopy (fNIRS) signal decoding. Existing research has shown that in complex brain state recognition tasks, deep learning models such as convolutional neural networks (CNNs) and long short-term memory networks (LSTMs) outperform traditional machine learning methods in feature extraction and classification. Meanwhile, in neuroscience research, graph-based methods construct brain functional connectivity networks and analyze these networks using connectivity metrics to uncover functional connections between brain regions. Existing methods, such as network models based on hemodynamic delay features and methods that convert fNIRS time series into two-dimensional images for feature learning, have made some progress in brain state recognition and brain disease diagnosis. However, existing methods still have certain limitations in the fNIRS signal decoding process. For example, some methods only focus on single information in time series features or spatial connectivity features, lacking joint modeling of local connectivity relationships and global topology of brain networks; some time modeling methods adopt a single time series model structure, making it difficult to capture long-term time series dependencies and short-term dynamic change features simultaneously; in addition, different feature sources such as raw blood oxygenation signal features, functional connectivity features and time series features are usually modeled independently, lacking an effective multi-source information fusion mechanism, resulting in insufficient ability of the model to express complex brain network features and weak cross-subject generalization ability.

[0006] Current brain-computer interfaces (BCIs) primarily rely on electroencephalography (EEG). However, research indicates that BCI systems based on motor imagery are considered one of the most promising BCI technologies due to their advantages, such as not requiring the presentation of stimulus signals, allowing for user autonomy, and providing a variety of signal types. This invention relates to the field of brain-computer interface signal processing and intelligent information processing technology, specifically to a motor imagery BCI system based on functional near-infrared spectroscopy (fNIRS) signals. More particularly, it relates to STGFNet (Spatial-Temporal Graph Fusion Network), a deep learning decoding method that integrates brain network topological features and temporal dynamic features, which can be applied to fields such as brain-computer interface control, intelligent unmanned system control, and neurorehabilitation training.

[0007] An existing fNIRS-based motor imagery brain-computer interface method (Hosni SM, Borgheai SB, Mclinden J, et al. An fNIRS-based motor imagery BCI for ALS: a subject-specific data-driven approach[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(12): 3063-3073.) estimates individualized spatial activation, optimizes channel sets, extracts subject-specific discriminant features, and combines linear support vector machines to achieve binary classification of motor imagery and resting state. This demonstrates that existing technology can utilize fNIRS signals to recognize motor imagery intentions and pays attention to the impact of individual differences on classification performance. However, this existing technology mainly focuses on individualized spatiotemporal activation features and statistical classification optimization, and has not further constructed a brain functional connectivity network and extracted brain functional connectivity features and whole-brain topological features, nor has it jointly modeled the long-distance temporal dependence and bidirectional dynamic relationship of blood oxygenation signals. This deficiency is precisely the technical problem that this invention aims to solve. Summary of the Invention

[0008] The purpose of this invention is to provide an intention recognition system based on near-infrared brain functional imaging technology to address the problems existing in current intention recognition technologies based on functional near-infrared spectral signals, such as susceptible signal quality to interference, insufficient mining of brain functional connectivity features, inadequate modeling of temporal dynamic dependencies, and weak multi-source feature fusion capabilities. This aims to improve the accuracy, stability, and interpretability of intention recognition. To achieve the above objective, this invention proposes an intention recognition method and system based on near-infrared brain functional imaging technology. The system includes a portable near-infrared brain functional imaging device for acquiring raw multi-channel light intensity signals from subjects under different task states. The software includes a data preprocessing module, a graph theory feature extraction module, a temporal dynamic modeling module fusing Transformer and Bi-LSTM, and an intention recognition and classification module. The system first acquires raw multi-channel light intensity signals using a near-infrared brain functional imaging device, which are then converted into multi-channel blood oxygenation time series data by a data preprocessing module. Subsequently, a graph theory feature extraction module extracts local functional connectivity features and global brain network topology features from the multi-channel blood oxygenation data to characterize the functional connectivity relationships between different brain regions and the overall information transmission efficiency of the brain network. Further, a time dynamic modeling module that integrates Transformer and Bi-LSTM performs joint modeling of the multi-channel blood oxygenation time series and graph theory fusion features to extract long-distance dependence information and bidirectional dynamic correlation information of blood oxygenation signals in the time dimension. Finally, an intent recognition and classification module comprehensively judges the graph theory features, time dynamic features, and raw signal graph theory fusion features, outputs the corresponding intent category label and its probability distribution results, and can further generate external device control commands.

[0009] The objective of this invention is achieved by at least one of the following technical solutions.

[0010] An intention recognition method based on near-infrared brain functional imaging technology includes the following steps: S1. Acquire raw multi-channel light intensity signals, preprocess the raw multi-channel light intensity signals, and obtain multi-channel blood oxygenation signals; S2. Extract brain functional connectivity features and brain network topology features from multi-channel blood oxygenation signals, and splice multi-channel blood oxygenation signals, brain functional connectivity features and brain network topology features to obtain graph theory fusion features; S3. Perform global temporal modeling on multi-channel blood oxygenation signals to capture long-distance dependencies between different time points and obtain temporal correlation features; S4. Perform comprehensive discrimination based on temporal correlation features and graph theory fusion features, and output the corresponding probability of motion imagination task category to achieve automatic recognition of different motion intentions.

[0011] Furthermore, the preprocessing includes the following steps: (1) Convert the original multi-channel light intensity signal into HbO and HbR signals; (2) Judging the channel signal quality based on the scalp coupling index: extract the HbO and HbR power spectra of each channel in the heart rate band, and define SCI as the absolute value of the correlation coefficient of the HbO and HbR heart rate power spectra. The higher the SCI value, the less interference the signal is subjected to. (3) Bandpass filtering: Filter the HbO and HbR signals using a filter bank; (4) Brain activity component extraction: Cortical activity components were extracted from HbO and HbR signals. The two hemodynamic signals, HbO and HbR, were decomposed into multi-channel blood oxygenation signal data and system noise, and the system noise component was removed.

[0012] Furthermore, the filter bank includes a third-order 0.01–0.2 Hz bandpass IIR Butterworth filter and a third-order IIR Butterworth notch filter with a center frequency of 0.1 Hz.

[0013] Furthermore, brain functional connectivity features include connectivity strength features, connectivity density features, and the reciprocal feature of the mean difference of functional signals for all channels; the connectivity strength feature is used to represent the correlation strength between two channel signals, the connectivity density feature is the proportion of significant correlations within a sliding window, used to measure the stability of the connection, and the reciprocal feature of the mean difference of functional signals is defined as the reciprocal of the mean difference between two channel signals, used to quantify the efficiency of functional information transmission between channel pairs.

[0014] Furthermore, the brain network topological features were extracted as follows: First, construct a functional connectivity network: use an adjacency matrix to indicate whether there are significant functional connections between channels.

[0015] in Represents the adjacency matrix of the nth element. The first channel and the first Each channel corresponds to a functional connection. Indicates the first Blood oxygenation signals from each channel, Indicates the first Blood oxygenation signals from each channel, This represents the Pearson correlation coefficient; Then, the average of the reciprocals of the shortest path lengths between any two nodes in the functional connectivity network is calculated to measure the efficiency of information transmission in the brain network.

[0016] in, This represents the total number of nodes in the functional connectivity network, that is, the total number of channels involved in constructing the brain's functional network. and This indicates the IDs of two different nodes in the network. Represents a node and The shortest path length between them, based on the adjacency matrix. Calculations show that This represents the global efficiency feature within the topological structure of brain networks. A higher value indicates a higher efficiency in global information transmission within the network.

[0017] Further, step S3 includes the following steps: S31. After linear mapping, the high-dimensional features obtained from the multi-channel blood oxygen time series are used to establish the dependency relationship between different time points in the time series based on the self-attention mechanism to obtain time-coded features. S32. The graph theory fusion features are mapped to a feature space that matches the time-coded features through a linear transformation to obtain the projected features; S33. Concatenate the temporal coding features with the projection features to obtain the concatenated features; S32. A bidirectional long short-term memory network structure is used to recursively model the spliced ​​features from both the forward and reverse directions to obtain time-related features.

[0018] Further, step S4 includes the following steps: First, the graph theory fusion features and the temporal correlation features are concatenated to form the classification input features, which are then transformed into intermediate classification features through linear mapping. The intermediate classification features are then subjected to nonlinear activation to obtain the activated classification features. Subsequently, the activated classification features are mapped to the target category space, and then... The function calculates the probability distribution of each category to obtain the final recognition result.

[0019] A system for implementing the intent recognition method based on near-infrared brain functional imaging technology includes: The signal acquisition module is used to collect raw multi-channel optical intensity signal data; The data preprocessing module is used to convert the raw multi-channel light intensity signal into a multi-channel blood oxygen signal.

[0020] The graph theory feature extraction module is used to extract brain functional connectivity features and brain network topology features from multi-channel functional near-infrared spectral blood oxygenation signals to characterize the functional connectivity between different brain regions and the overall information transmission efficiency of the brain network, and fuse them into graph theory fusion features to provide spatial structural feature information for subsequent motor image recognition. The time-dynamic modeling module is used to fuse multi-channel blood oxygen time series and graph theory fusion features to output a time-dynamic feature representation vector; The intent recognition and classification module is used to comprehensively judge the graph theory fusion features and the temporal correlation features output by the temporal dynamic modeling module, and output the corresponding motion imagination task category probability to achieve automatic recognition of different motion intents.

[0021] A computer device according to the present invention includes a memory and a processor, the memory being electrically connected to the processor, the memory storing a computer program, which, when executed by the processor, causes the processor to implement the method described herein.

[0022] The present invention provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor implements the method described herein.

[0023] Compared with existing technologies, the beneficial effects of the present invention are as follows: This invention constructs a complete system architecture from raw signal acquisition, data preprocessing, spatial topological feature extraction, temporal dynamic modeling to intent classification output, which can realize high-precision intent recognition based on near-infrared brain functional imaging signals and has good robustness, scalability and application prospects. Attached Figure Description

[0024] Figure 1 This is a framework diagram of an intent recognition system based on near-infrared brain functional imaging, as an example.

[0025] Figure 2 This is a flowchart of graph theory feature extraction for an example.

[0026] Figure 3 The flowchart of the time dynamic modeling module in the embodiment is shown. Detailed Implementation

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] like Figure 1As shown in this embodiment, an intention recognition system based on near-infrared brain functional imaging includes: a signal acquisition module, a preprocessing module, a graph theory feature extraction module, a temporal dynamic modeling module fusing Transformer and Bi-LSTM, and a multi-source feature fusion intention recognition classification module. After acquiring raw multi-channel light intensity signals, the signal acquisition module processes them through the preprocessing module, graph theory feature extraction module, temporal dynamic modeling module, and intention recognition classification module, outputting category labels and control commands.

[0029] The signal acquisition module is used to collect raw multi-channel optical intensity signal data; The data preprocessing module is used to convert the raw multi-channel light intensity signal into a multi-channel blood oxygen signal.

[0030] The graph theory feature extraction module is used to extract brain functional connectivity features and brain network topology features from multi-channel functional near-infrared spectral blood oxygenation signals to characterize the functional connectivity between different brain regions and the overall information transmission efficiency of the brain network, providing spatial structural feature information for subsequent motor image recognition.

[0031] The graph theory feature extraction module includes a local functional connectivity feature extraction module, a global brain network topology feature extraction module, and a feature fusion module. The local functional connectivity feature extraction module is used to extract brain functional connectivity features, while the global brain network topology feature extraction module is used to construct a brain functional network from a preprocessed multi-channel blood oxygenation time series matrix and extract brain network topology features.

[0032] The local functional connectivity feature extraction module includes a functional connectivity network construction module, which is used to indicate the functional connectivity between channels through an adjacency matrix. The global brain network topology feature extraction module includes a global efficiency feature module; the global efficiency feature module is used to calculate the average of the reciprocals of the shortest path length between any two nodes in the functional connectivity network, which is used to measure the information transmission efficiency of the brain network.

[0033] The feature fusion module is used to connect the original temporal signal features, brain functional connectivity features, and brain network topology features into a graph theory fusion feature.

[0034] The time dynamic modeling module is used to fuse multi-channel blood oxygen time series and graph theory fusion features to output a time dynamic feature representation vector.

[0035] The time dynamic modeling module includes a Transformer time encoding module and a Bi-LSTM time correlation modeling module. The Transformer time encoding module is used to perform global time series modeling on multi-channel blood oxygen time series to capture long-distance dependencies between different time points. The Bi-LSTM time correlation modeling module uses a bidirectional long short-term memory network structure to further combine graph theory features with the Transformer time encoding results to model the dynamic correlation of time series features in both forward and backward directions.

[0036] The intent recognition and classification module is used to comprehensively judge the graph theory fusion features and the temporal correlation features output by the temporal dynamic modeling module, and output the corresponding motion imagination task category probability to achieve automatic recognition of different motion intents.

[0037] like Figures 1-3 As shown, an intention recognition method based on near-infrared brain functional imaging in this embodiment includes the following steps: S1. Acquire raw multi-channel light intensity signals using near-infrared brain functional imaging equipment, preprocess the raw multi-channel light intensity signals, and obtain multi-channel blood oxygenation signals.

[0038] Raw multi-channel light intensity signals were acquired: A near-infrared brain functional imaging device was worn on the brain to acquire raw multi-channel light intensity signal data in both resting and task states. As one embodiment, the motor task in this embodiment involved flexion of the left arm, right arm, left hand, and right hand. The acquisition system contained 768 functional near-infrared spectral channels with a sampling frequency of 10.2 Hz. Each task trial lasted 12 seconds, including 6 seconds of rest and 6 seconds of movement, with 25 repetitions per set. The 6-second movement data was used for subsequent processing.

[0039] Preprocessing includes the following steps: (1) Multi-channel light intensity signal data is converted into HbO and HbR data. The original light intensity is converted into light density using the existing technology of the modified Beer–Lambert Law (MBLL), and then the HbO and HbR data are obtained.

[0040] (2) Channel signal quality is judged based on the scalp coupling index (SCI). The principle is as follows: In the heart rate frequency band (0.8-2.5Hz), changes in HbO and HbR signals are caused by cardiac activity. If fNIRS detection is ideal, HbO and HbR signals should be highly negatively correlated; if fNIRS detection is interfered with (e.g., poor probe-scalp contact, excessive skull thickness, etc.), the correlation will decrease. Therefore, the HbO and HbR power spectra of each channel in the heart rate frequency band are extracted, and the SCI is defined as the absolute value of the correlation coefficient between the HbO and HbR heart rate power spectra. The higher the SCI value, the less interference the signal is subjected to. An SCI value less than 0.8 is considered to have poor signal quality, and the channel will be discarded.

[0041] (3) Bandpass filtering. To remove instrument noise and physiological noise, a filter bank is used for the HbO / HbR signal. In one embodiment, the filter bank includes a third-order 0.01-0.2Hz bandpass IIR Butterworth filter and a third-order IIR Butterworth notch filter with a center frequency of 0.1Hz.

[0042] (4) Brain Activity Component Extraction: Further cortical activity components were extracted from the HbO / HbR signals. Using existing correlation-based signal improvement algorithms, the two hemodynamic signals, HbO and HbR, were decomposed into negatively linearly correlated components (multi-channel oxygenation signal data) and positively linearly correlated components (system noise), and the system noise component was removed. Each measurement channel contains hemodynamic signals of HbO and HbR, which are considered as basic functionally coupled units—a total of [number missing] One channel pair.

[0043] S2. Brain functional connectivity features and brain network topological features are extracted from multi-channel blood oxygenation signals to characterize the functional connectivity relationships between different brain regions and the overall information transmission efficiency of the brain network, providing spatial structural feature information for subsequent motor imagery recognition. The specific steps are as follows: S21. Extract brain functional connectivity features for all channels. These features include connectivity strength, connectivity density, and the inverse of the mean difference of functional signals for all channels. Connection strength characteristics are used to represent the strength of the correlation between two channel signals, and are calculated using the Pearson correlation coefficient:

[0044] in Indicates connection strength. They represent the first i HbO and HbR signals of each channel, The Pearson correlation coefficient is represented as follows:

[0045] in, Indicates signal The average value, Indicates the first The signal value at each time point, Indicates a point-in-time index. This indicates the total number of time points contained in the signal segment. After normalization, the connection strength... The value range is (0,1), and the larger the value, the stronger the functional connection between the two channels.

[0046] The connection density feature is defined as the proportion of significant correlations within a sliding window and is used to measure the stability of connections.

[0047]

[0048] in, This represents the connectivity density characteristic, used to measure the proportion of two channel signals that remain significantly correlated across all sliding windows; The length of the sliding window. Indicates the first Signal segments in a sliding window Indicates and The corresponding other channel is in the Signal segments in a sliding window Indicates the first The first channel in the The signal values ​​for each sliding window, with 0.3 as the significance threshold. This is an indicator function; it takes the value 1 when the condition is true, and 0 otherwise. The larger the value, the higher the frequency of effective connections between the two channels.

[0049] The reciprocal characteristic of the difference in the mean of the functional signals is defined as the reciprocal of the difference in the mean of the two channel signals, and is used to quantify the efficiency of functional information transfer between channel pairs:

[0050] in, The reciprocal characteristic of the difference between the mean values ​​of the functional signals is used to characterize the average difference between two channel signals and the efficiency of functional information transmission. The larger the value, the smaller the average difference between the two channel signals, the more similar their functional activities, and the higher the information transmission efficiency. It is a very small constant used to avoid division by zero. The smaller the constant, the higher the similarity between the two channel signals, and the higher the information transmission efficiency.

[0051] S22. A brain functional network was constructed using multi-channel blood oxygenation signals, and the topological features of the brain network were extracted. A functional connectivity map was constructed at the full-channel scale, and topological indices of the entire brain network were extracted.

[0052] The functional connectivity network is constructed by using an adjacency matrix to indicate whether there are significant functional connections between channels.

[0053] in Represents the adjacency matrix of the nth element. The first channel and the first The element corresponding to each channel Indicates the first Blood oxygenation signals from each channel, Indicates the first Blood oxygenation signals from each channel, This represents the Pearson correlation coefficient, with 0.3 being the global connectivity threshold. This indicates that there is a significant functional connection between the two corresponding channels.

[0054] The average of the reciprocals of the shortest path lengths between any two nodes in a brain network is used to measure the efficiency of information transmission in the brain network. Higher global efficiency indicates a more tightly connected overall topology of the brain network and higher information transmission efficiency.

[0055]

[0056] in, This represents the total number of nodes in the functional connectivity network, that is, the total number of channels involved in constructing the brain's functional network. and This indicates the IDs of two different nodes in the network. Represents a node and The shortest path length between them, based on the adjacency matrix. Calculated. Global efficiency. A higher value indicates a higher efficiency in global information transmission within the network.

[0057] S23. This invention designs a feature fusion mechanism that connects multi-channel blood oxygenation signals, brain functional connectivity features, and brain network topology features into a unified representation:

[0058] in, Indicates multi-channel blood oxygenation signal; The brain functional connectivity feature vector is composed of the local features of all channel pairs stacked sequentially. Represents the global efficiency feature in the topological structure of brain networks; Indicates will , and The unified graph theory fusion feature obtained after splicing; express 3D real vector space and It is a vector formed by stacking the local features of all channel pairs in sequence.

[0059] S3. Perform global temporal modeling on the preprocessed multi-channel blood oxygenation signals to capture long-distance dependencies between different time points and obtain temporal correlation features. The specific steps are as follows: S31. Establishing dependencies between different time points in a time series based on self-attention mechanisms: First, input the blood oxygen time series data. Represented as:

[0060] in, Indicates the number of channel pairs. Indicates the length of time. Representing feature dimension, This represents the three-dimensional real-valued tensor corresponding to the channel pairs in the blood oxygen time series data. A high-dimensional feature representation is obtained by mapping the channel pairs in the blood oxygen time series data to a high-dimensional feature space through a linear mapping.

[0061] For the input features, which are high-dimensional feature representations obtained by linearly mapping the blood oxygen time series data to the channels, the query matrix Q, key matrix K, and value matrix V are first generated, and the attention output in the time dimension is calculated using the following formula:

[0062] in, Represents the dimension of the key vector. This represents the normalization function. Through the self-attention calculation described above, the correlation weights between different time points can be obtained, thus enabling the modeling of long-term dependencies. Furthermore, a multi-head attention mechanism is employed to jointly model the temporal dependency features from multiple subspaces, with the expression:

[0063] in,

[0064] In the formula, Indicates the number of heads of attention. and These are the learnable parameter matrices. After multi-head self-attention and feedforward network encoding, the Transformer encoded output is obtained:

[0065] in, This represents the initial feature representation input to the Transformer encoder, which is the high-dimensional feature obtained by linearly mapping the blood oxygen time series data to the channels; This represents the Transformer encoded output feature obtained after multi-head self-attention mechanism and feedforward network encoding, used to characterize the global dependency of the input sequence in the time dimension.

[0066] To obtain a unified global temporal feature representation, the encoding results are aggregated along the temporal dimension, preferably using average pooling to obtain the temporal encoded feature vector. :

[0067] in, , This represents the encoded feature corresponding to the t-th time point. This represents the total number of time steps in the time series. Indicates by Each channel pair and The space of real matrixes formed by the features of the dimension.

[0068] S32. Based on the Transformer temporal encoding results, further incorporate graph theory fusion features. This involves modeling the dynamic relationships between temporal features in both forward and backward directions. First, graph theory is used to fuse these features. After linear transformation and mapping to a feature space that matches the time-coded features, the projected features are obtained. :

[0069] in, The weight matrix represents the linear mapping used to fuse graph theory features. Projected onto time-encoded feature vectors Matching feature space; This represents the bias vector corresponding to the linear mapping, used to adjust the mapping result by translation.

[0070] S33. Encode the time-encoded feature vector With projection features splicing results in splicing features ,Right now:

[0071] in, The weight matrix represents the linear mapping. This represents the bias vector corresponding to the linear mapping, used to adjust the mapping result by translation.

[0072] 34. A bidirectional long short-term memory network structure is adopted to recursively model the input sequence from both the forward and backward directions, so as to simultaneously capture the influence of historical moments on the current moment and the supplementary contextual information of future moments on the current moment. Specifically: At any given time, the forward LSTM unit calculates the forward hidden state for the current time step. The LSTM unit includes a forget gate, an input gate, a candidate state, and an output gate, and their calculation processes are as follows:

[0073] in, For time step index, For the first The splicing features of each time step This is the forward hidden state of the previous time step. Output for the forget gate. For input gate output, The candidate cell state is... This represents the current cell state. For output gate output, This is the forward hidden state at the current time step; This represents the Sigmoid activation function. This represents the hyperbolic tangent activation function. This represents element-wise multiplication. , , , These represent the weight matrices corresponding to each gating control. , , , These represent the corresponding bias terms.

[0074] Accordingly, the backward LSTM unit performs the same recursive operation in reverse chronological order to obtain the backward hidden state. Finally, the forward hidden state and the backward hidden state are concatenated to obtain the bidirectional temporal correlation feature representation:

[0075] in, , This represents the dimension of the hidden layer in a one-way LSTM.

[0076] Time-related features It can simultaneously reflect the bidirectional temporal dependence information of multi-channel blood oxygenation signals and the prior information of brain network topology. The temporal correlation features can be further input into the classification module for the recognition and discrimination of motor imagery tasks.

[0077] S4. Perform comprehensive discrimination based on temporal correlation features and graph theory fusion features, and output the probability distribution results corresponding to each intention category to achieve automatic recognition of different motion intentions.

[0078] This step is implemented using an intent recognition and classification module, whose input is graph theory fusion features. Time-related features The output is the probability distribution results corresponding to each intent category. The intent recognition and classification module is implemented using a multilayer perceptron (MLP), which includes a first fully connected layer, a non-linear activation layer, a second fully connected layer, and a softmax output layer. Specifically, firstly, graph theory fusion features Time-related features The features are concatenated to form the categorical input features, and then transformed into intermediate categorical feature representations through linear mapping:

[0079] in, This is the weight matrix of the first fully connected layer. For the corresponding bias vector This indicates a feature concatenation operation. This represents intermediate classification features. Through the above processing, brain network topology information, original signal fusion information, and temporal dynamic correlation information can be uniformly mapped into the classification space.

[0080] To enhance the model's ability to represent nonlinear classification boundaries, the intermediate classification features are... For nonlinear activation processing, a modified linear unit function is preferred. The activated classification features are obtained as follows:

[0081] in, This represents a modified linear unit activation function. This nonlinear transformation enhances the sparsity and discriminative power of feature representation, improving the separability between different motion visualization tasks.

[0082] Subsequently, the activated classification features The input and output layers are mapped to the target category space through a second fully connected layer, and then... The function calculates the probability distribution of each category to obtain the final recognition result:

[0083] in, This is the output layer weight matrix. This is the output layer bias vector. The output of the intent recognition and classification module represents the predicted probability for each motion imagery category. For Motion intent recognition tasks include:

[0084] in, This indicates that the input sample belongs to the first... The predicted probability of the type of motion intent. The final identification category can be determined according to the principle of maximum probability, that is:

[0085] in, This indicates the category label for the final output motion intent.

[0086] The intent recognition and classification module can uniformly model and output the extracted multi-source discriminative features, realizing the classification and recognition of left-hand motor imagery, right-hand motor imagery, foot motor imagery, and other preset task states. Compared with methods that only use single temporal or single spatial features for classification, the intent recognition and classification module of this invention fully integrates brain network topology information, temporal dynamic dependence information, and raw blood oxygenation signal representation information, thereby effectively improving the recognition accuracy, robustness, and practicality of the motor imagery brain-computer interface system.

[0087] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, enabling those skilled in the art to better understand and utilize the invention.

Claims

1. An intention recognition method based on near-infrared brain functional imaging technology, characterized in that, Includes the following steps: S1. Acquire raw multi-channel light intensity signals, preprocess the raw multi-channel light intensity signals, and obtain multi-channel blood oxygenation signals; S2. Extract brain functional connectivity features and brain network topology features from multi-channel blood oxygenation signals, and splice multi-channel blood oxygenation signals, brain functional connectivity features and brain network topology features to obtain graph theory fusion features; S3. Perform global temporal modeling on multi-channel blood oxygenation signals to capture long-distance dependencies between different time points and obtain temporal correlation features; S4. Perform comprehensive discrimination based on temporal correlation features and graph theory fusion features, and output the corresponding probability of motion imagination task category to achieve automatic recognition of different motion intentions.

2. The intention recognition method based on near-infrared brain functional imaging technology according to claim 1, characterized in that, Preprocessing includes the following steps: (1) Convert the original multi-channel light intensity signal into HbO and HbR signals; (2) Judging the channel signal quality based on the scalp coupling index: extract the HbO and HbR power spectra of each channel in the heart rate band, and define SCI as the absolute value of the correlation coefficient of the HbO and HbR heart rate power spectra. The higher the SCI value, the less interference the signal is subjected to. (3) Bandpass filtering: Filter the HbO and HbR signals using a filter bank; (4) Brain activity component extraction: Cortical activity components were extracted from HbO and HbR signals. The two hemodynamic signals, HbO and HbR, were decomposed into multi-channel blood oxygenation signal data and system noise, and the system noise component was removed.

3. The intention recognition method based on near-infrared brain functional imaging technology according to claim 2, characterized in that, The filter bank consists of a third-order 0.01–0.2 Hz bandpass IIR Butterworth filter and a third-order IIR Butterworth notch filter with a center frequency of 0.1 Hz.

4. The intention recognition method based on near-infrared brain functional imaging technology according to claim 1, characterized in that, Brain functional connectivity features include connectivity strength features, connectivity density features, and the reciprocal feature of the difference between the means of functional signals for all channels. The connectivity strength feature is used to represent the strength of the correlation between two channel signals. The connectivity density feature is the proportion of significant correlations within a sliding window, used to measure the stability of the connection. The reciprocal feature of the difference between the means of functional signals is defined as the reciprocal of the difference between the means of two channel signals, used to quantify the efficiency of functional information transmission between channel pairs.

5. The intention recognition method based on near-infrared brain functional imaging technology according to claim 1, characterized in that, The brain network topology features extracted are as follows: First, construct a functional connectivity network: use an adjacency matrix to indicate whether there are significant functional connections between channels. in Represents the adjacency matrix of the nth element. The first channel and the first Each channel corresponds to a functional connection. Indicates the first Blood oxygenation signals from each channel, Indicates the first Blood oxygenation signals from each channel, This represents the Pearson correlation coefficient; Then, the average of the reciprocals of the shortest path lengths between any two nodes in the functional connectivity network is calculated to measure the efficiency of information transmission in the brain network. in, This represents the total number of nodes in the functional connectivity network, that is, the total number of channels involved in constructing the brain's functional network. and This indicates the IDs of two different nodes in the network. Represents a node and The shortest path length between them, based on the adjacency matrix. Calculations show that This represents the global efficiency feature within the topological structure of brain networks. A higher value indicates a higher efficiency in global information transmission within the network.

6. The intention recognition method based on near-infrared brain functional imaging technology according to claim 1, characterized in that, Step S3 includes the following steps: S31. After linear mapping, the high-dimensional features obtained from the multi-channel blood oxygen time series are used to establish the dependency relationship between different time points in the time series based on the self-attention mechanism to obtain time-coded features. S32. The graph theory fusion features are mapped to a feature space that matches the time-coded features through a linear transformation to obtain the projected features; S33. Concatenate the temporal coding features with the projection features to obtain the concatenated features; S32. A bidirectional long short-term memory network structure is used to recursively model the spliced ​​features from both the forward and reverse directions to obtain time-related features.

7. The intention recognition method based on near-infrared brain functional imaging technology according to claim 1, characterized in that, Step S4 includes the following steps: First, the graph theory fusion features and the temporal correlation features are concatenated to form the classification input features, which are then transformed into intermediate classification features through linear mapping. The intermediate classification features are then subjected to nonlinear activation to obtain the activated classification features. Subsequently, the activated classification features are mapped to the target category space, and then... The function calculates the probability distribution of each category to obtain the final recognition result.

8. A system for implementing the intention recognition method based on near-infrared brain functional imaging technology as described in claim 1, characterized in that, include: The signal acquisition module is used to collect raw multi-channel optical intensity signal data; The data preprocessing module is used to convert the raw multi-channel light intensity signal into a multi-channel blood oxygen signal. The graph theory feature extraction module is used to extract brain functional connectivity features and brain network topology features from multi-channel functional near-infrared spectral blood oxygenation signals to characterize the functional connectivity between different brain regions and the overall information transmission efficiency of the brain network, and fuse them into graph theory fusion features to provide spatial structural feature information for subsequent motor image recognition. The time-dynamic modeling module is used to fuse multi-channel blood oxygen time series and graph theory fusion features to output a time-dynamic feature representation vector; The intent recognition and classification module is used to comprehensively judge the graph theory fusion features and the temporal correlation features output by the temporal dynamic modeling module, and output the corresponding motion imagination task category probability to achieve automatic recognition of different motion intents.

9. A computer device comprising a memory and a processor, the memory being electrically connected to the processor, the memory storing a computer program, characterized in that: When the computer program is executed by the processor, it causes the processor to implement the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor implements the method as described in any one of claims 1 to 8.

Images