Personnel state prediction method based on brain function connection, edge computing device and medium

Abstract

The invention provides a personnel state prediction method based on brain function connection, edge computing equipment and a medium, and relates to the technical field of artificial intelligence. The personnel state prediction method based on brain function connection comprises the steps that electroencephalogram signals of multiple channels are obtained, initial adjacency data are obtained, the electroencephalogram signals of different channels correspond to different brain areas, and the initial adjacency data represent the function association strength between the electroencephalogram signals of different channels; performing attention fusion processing on the initial adjacent data to obtain target adjacent data; based on the target adjacent data, feature fusion extraction and personnel state prediction processing are carried out on the electroencephalogram signals, and a personnel state prediction result is obtained.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a method for predicting human state based on brain functional connectivity, an edge computing device, and a medium. Background Technology

[0002] Brain connectomics is a rapidly developing sub-discipline of neuroscience that aims to characterize the structural and functional connectivity patterns between different brain regions at a macroscopic scale, providing a network-level perspective for understanding human condition. Based on brain functional connectivity, it can examine the functional and structural connectivity relationships between different brain regions at a macroscopic scale, thereby predicting human condition.

[0003] In electroencephalogram (EEG) studies, relying solely on local features of EEG signals from a single channel for predicting a person's state may affect the accuracy of such predictions. Summary of the Invention

[0004] One of the technical problems this disclosure aims to solve is the low accuracy of personnel status prediction.

[0005] To address the aforementioned technical problems, this disclosure provides a method for predicting the state of a person based on brain functional connectivity, comprising: acquiring EEG signals from multiple channels and acquiring initial adjacency data, wherein the EEG signals from different channels correspond to different brain regions, and the initial adjacency data represents the functional correlation strength between the EEG signals from different channels; performing attention fusion processing on the initial adjacency data to obtain target adjacency data; and performing feature fusion extraction and person state prediction processing on the EEG signals based on the target adjacency data to obtain the person state prediction result.

[0006] In some embodiments, based on target adjacency data, feature fusion extraction and personnel state prediction processing are performed on EEG signals to obtain personnel state prediction results, including: extracting temporal features from EEG signals of different channels to obtain initial temporal features of different channels; fusing spatial features from the initial temporal features of different channels based on target adjacency data to obtain spatial fusion features; extracting temporal features from the spatial fusion features to obtain deep spatiotemporal features; and predicting personnel state based on the deep spatiotemporal features to obtain personnel state prediction results.

[0007] In some embodiments, attention fusion processing is performed on initial adjacency data to obtain target adjacency data, including: performing feature mapping processing on the initial adjacency data to obtain at least one mapping processing result, wherein each mapping processing result includes query features, key features, and value features; performing attention calculation on the query features, key features, and value features corresponding to each mapping processing result to obtain at least one attention calculation result; and obtaining target adjacency data based on at least one attention calculation result.

[0008] In some embodiments, before performing attention fusion processing on the initial adjacency data to obtain the target adjacency data, the method further includes: performing sparsification processing on the initial adjacency data.

[0009] In some embodiments, the multiple channels include n channels, and the initial adjacency data includes n×n sub-adjacency data; the initial adjacency data is sparsified by: comparing any one of the n×n sub-adjacency data with a preset functional association strength threshold; when the sub-adjacency data is less than the preset functional association strength threshold, the sub-adjacency data is set to a preset value.

[0010] In some embodiments, the plurality of channels includes n channels, and the initial adjacency data includes n×n sub-adjacency data, wherein any one of the n×n sub-adjacency data is represented as... Where 1≤i≤n, 1≤j≤n; obtaining initial adjacency data includes: performing correlation calculations on the EEG signals of the i-th channel and the j-th channel among the multiple channels to obtain the sub-adjacency data corresponding to the i-th channel and the j-th channel. .

[0011] In some embodiments, based on target adjacency data, feature fusion extraction and personnel state prediction processing are performed on EEG signals to obtain personnel state prediction results, including: constructing corresponding nodes for each channel to obtain multiple nodes corresponding to multiple channels, and constructing edges for any two nodes among the multiple nodes; determining the EEG signal of each channel as the node feature of the corresponding node, and determining edge features based on target adjacency data; generating graph structure data based on node features and edge features; and performing graph convolution processing on the graph structure data to obtain personnel state prediction results.

[0012] This disclosure also provides a personnel state prediction model device based on brain functional connectivity, including: an attention network module, a first temporal convolutional network module, a spatial graph convolutional network module, a second temporal convolutional network module, and a personnel state prediction network module. The attention network module performs attention fusion processing on initial adjacency data to obtain target adjacency data, wherein EEG signals from different channels correspond to different brain regions, and the initial adjacency data represents the functional correlation strength between EEG signals from different channels; the first temporal convolutional network module extracts temporal features from EEG signals from different channels to obtain initial temporal features for different channels; the spatial graph convolutional network module fuses spatial features based on the target adjacency data and the initial temporal features of different channels to obtain spatial fusion features; the second temporal convolutional network module extracts temporal features from the spatial fusion features to obtain deep spatiotemporal features; and the personnel state prediction network module predicts personnel state based on the deep spatiotemporal features to obtain a personnel state prediction result.

[0013] This disclosure also provides an edge computing device, including a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the method described above.

[0014] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0015] Through the above technical solution, this disclosure acquires EEG signals from multiple channels and obtains initial adjacency data; performs attention fusion processing on the initial adjacency data to obtain target adjacency data; and based on the target adjacency data, performs feature fusion extraction and personnel state prediction processing on the EEG signals to obtain the personnel state prediction result. The method of this disclosure improves the accuracy of personnel state prediction. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a method for predicting the state of a person based on brain functional connectivity provided in an embodiment of this disclosure; Figure 2 This is a schematic diagram of the personnel status prediction model device provided in the embodiments of this disclosure; Figure 3This is a schematic diagram of a method for predicting the state of a person based on brain functional connectivity provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram of another method for predicting the state of a person based on brain functional connectivity provided in this embodiment of the disclosure; Figure 5 This is a block diagram of an edge computing device provided in an embodiment of this disclosure. Detailed Implementation

[0018] The embodiments of this disclosure will be further described in detail below with reference to the accompanying drawings and examples. The detailed description of the embodiments and the accompanying drawings are used to illustrate the principles of this disclosure by way of example, but should not be used to limit the scope of this disclosure. This disclosure can be implemented in many different forms and is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

[0019] These embodiments are provided to make the disclosure thorough and complete, and to fully express the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specifically stated, the relative arrangement of components and steps, material composition, numerical expressions, and values ​​set forth in these embodiments should be interpreted as exemplary only and not as limiting.

[0020] All terms used in this disclosure have the same meaning as understood by one of ordinary skill in the art to which this disclosure pertains, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant art, and not as idealized or highly formalized, unless expressly defined herein.

[0021] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0022] Brain connectomics is a rapidly developing subfield of neuroscience that allows for the examination of functional and structural connections between different brain regions on a macroscopic scale. Electrophysiological functional connectivity can be defined as a set of metrics describing "high-level couplings across low-level networks." Low-level networks consist of interconnected cells distributed across the cortex at a depth of 1 cm or larger—a spatial scale that varies throughout the brain depending on local cell density, region depth, and the dominant direction of electrical currents within that region. Because these local circuits are relatively homogeneous in function, they are termed "brain regions." High-level couplings constitute the signaling interactions between these local regions. These interacting regions are separated by distances generally greater than the spatial extent of each region in the network, forming the higher-level network. In EEG research, relying solely on local features of EEG signals from a single channel to predict a person's state may overlook crucial information about coordinated activity between different brain regions, thus affecting the accuracy of the prediction. Therefore, introducing features of brain functional connectivity or brain network features often helps to improve prediction performance.

[0023] This disclosure proposes a method for predicting the state of individuals based on brain functional connectivity. It performs sparsification and attention fusion processing on the initial adjacency data representing brain functional coupling, improving the feedback of the adjacency data on the functional coupling relationships between different brain regions. Furthermore, it combines the adjacency data representing brain functional coupling with a graph convolutional network to predict the state of individuals, thus improving the accuracy of the prediction. If only the initial adjacency matrix is ​​sparsified (e.g., by thresholding or retaining the top k strongest connections), this process may filter out some weak but physiologically significant functional connections. Since the sparsified adjacency matrix cannot fully represent the complex functional coupling patterns between different brain regions, its use as feature input may limit the model's ability to capture the dynamics of the network related to the state of individuals, thereby restricting the accuracy of the prediction to some extent. Therefore, this disclosure requires attention fusion processing on the sparsified adjacency matrix.

[0024] Figure 1 This is a flowchart of a method for predicting human state based on brain functional connectivity provided in this disclosure, as shown in the embodiment. Figure 1 As shown, the method 100 for predicting the state of a person based on brain functional connectivity includes steps S110 to S130.

[0025] Step S110: Obtain EEG signals from multiple channels and obtain initial adjacency data, wherein the EEG signals from different channels correspond to different brain regions, and the initial adjacency data represents the functional correlation strength between the EEG signals from different channels.

[0026] Electroencephalogram (EEG) signals are examples of brainwave signals. Functional connectivity (FC) characterizes the statistical correlation of information flow between cortical regions, reflecting the synchronicity or synergy of functional signals between different brain regions.

[0027] Specifically, EEG signals from multiple channels (e.g., n channels) are acquired and segmented by n leads (electrodes) placed in different brain regions. Initial adjacency data represents the functional correlation strength between the n channels of EEG signals, such as the functional correlation strength between the first and second channels, the functional correlation strength between the first and third channels, etc., thus obtaining n×n functional correlation strengths. The strengths of ×n functional associations are represented in matrix form as the initial adjacency data. The channel number n indicates that signals from n brain regions are being acquired, and signals from each brain region can be acquired through leads.

[0028] Segmentation of acquired multi-channel EEG signals includes: selecting fixed-length, overlapping segments for training or prediction. For example, for an n-channel EEG signal, segmentation is performed based on the sampling rate Fs (Hz), window length w seconds, and window shift t seconds. The length of an EEG signal segment (number of sampling points) in one channel is then M = Fs × w, and the EEG signals corresponding to the n channels are represented by matrix X. . Functional correlation strength can be obtained by calculating the correlation coefficients (e.g., Pearson correlation coefficient, Spearman correlation coefficient, partial correlation coefficient, cross-correlation coefficient) of EEG signals from different channels. It can also be obtained by calculating the coherence coefficients (e.g., spectral coherence coefficient, imaginary part coherence coefficient, wavelet coherence coefficient) of EEG signals from different channels. Furthermore, it can be obtained by calculating the phase-based functional connectivity coefficients (e.g., phase lock value, hysteresis index, phase slope index) of EEG signals from different channels. Finally, it can be obtained by calculating the Granger causal directed connectivity coefficients of EEG signals from different channels. Functional correlation strength typically ranges from 0 to 1.

[0029] Step S120: Perform attention fusion processing on the initial adjacency data to obtain the target adjacency data.

[0030] Specifically, attention fusion processing is implemented through the attention layers of a spatiotemporal convolutional network. These attention layers can be single-head or multi-head attention layers. The target adjacency data obtained after attention layer fusion processing has the same feature dimensions as the initial adjacency data. Performing attention fusion processing on the initial adjacency data can improve the feedback degree of the target adjacency data on the functional coupling relationships between different brain regions or improve the accuracy of the target adjacency data in representing the strength of functional associations between different brain regions.

[0031] Step S130: Based on the target adjacency data, feature fusion extraction and personnel state prediction processing are performed on the EEG signal to obtain the personnel state prediction result.

[0032] Specifically, the feature fusion and extraction process is implemented by a spatiotemporal convolutional network. This network can include a single spatiotemporal convolutional block or multiple spatiotemporal convolutional blocks (e.g., 2, 3, 4, etc.), connected in series. Each spatiotemporal convolutional block includes a temporal convolutional network and a spatial graph convolutional network, with two temporal convolutional networks and one spatial graph convolutional network. After the spatiotemporal convolutional network completes the feature fusion and extraction process, a flattening network layer flattens the multidimensional features to obtain one-dimensional data. Based on this one-dimensional data, the flattening network layer performs personnel state prediction to obtain the prediction result.

[0033] Personnel status prediction results include multiple states such as workload, mood, stress, attention, and distraction. Personnel status prediction results can include predictions of a single person's status, or predictions of multiple people's statuses can be considered.

[0034] The embodiments of this disclosure process the initial adjacency data by introducing an attention fusion mechanism, which can adaptively strengthen the functional connections between channels that are significantly related to the current task according to different personnel states, suppress noise or irrelevant connections, so that the final target adjacency data more accurately reflects the real functional coupling relationship between brain regions. Based on the target adjacency data, feature fusion extraction and personnel state prediction processing of EEG signals are performed to improve the accuracy and robustness of personnel state prediction.

[0035] In some embodiments of this disclosure, the multiple channels include n channels, and the initial adjacency data includes n×n sub-adjacency data, where any one of the n×n sub-adjacency data is represented as... Where 1≤i≤n, 1≤j≤n; obtaining initial adjacency data includes: performing correlation calculations on the EEG signals of the i-th channel and the j-th channel among the multiple channels to obtain the sub-adjacency data corresponding to the i-th channel and the j-th channel. .

[0036] Specifically, correlation calculations include calculating correlation coefficients (such as Pearson correlation coefficient, Spearman correlation coefficient, partial correlation coefficient, and cross-correlation coefficient) and coherence coefficients (such as spectral coherence coefficient, imaginary part coherence coefficient, and wavelet coherence coefficient) of EEG signals from different channels, phase-based functional connectivity coefficients (such as phase lock value, hysteresis index, and phase slope index), and Granger causal directed connectivity coefficients. For example, correlation calculations are performed on the EEG signals from the i-th channel and the j-th channel, and the correlation results are used as sub-adjacent data. Each child adjacency can be a connection value. When i=j, the child adjacency data... This can represent the correlation of the EEG signals within that channel, i.e., sub-adjacent data. It can be 1.

[0037] The embodiments of this disclosure generate sub-adjacent data by directly calculating the correlation between the EEG signals of the i-th channel and the j-th channel. This can quantify the functional coupling strength between any two channels, enabling the adjacency matrix to be updated adaptively with the task state. It automatically reduces the edge weights of weak connections and artifacts, thereby improving the accuracy of personnel state prediction. The edge weights of artifacts reflect the synchronicity or functional connectivity strength of artifact signals in different brain regions and are used to distinguish between real neural activity and interference signals.

[0038] In some embodiments of this disclosure, before performing attention fusion processing on the initial adjacency data to obtain the target adjacency data, the method further includes: performing sparsification processing on the initial adjacency data.

[0039] Specifically, the initial adjacency data representing brain functional coupling can be represented as an initial adjacency matrix. There are many methods for constructing the initial adjacency matrix. EEG signals from n channels can be collected through n lead processes, meaning that the EEG signals between the n leads can all be used to calculate a relational value (connection value), which is the value of the initial adjacency matrix. Each value represents a value of a brain region. The size of the initial adjacency matrix is: number of leads × number of leads = n × n. The connection value is usually between 0 and 1, but it contains some spurious connections. For example, some different brain regions may not actually have functional coupling, but the calculated corresponding connection value is not 0 (a non-zero value indicates that the calculated result shows functional coupling).

[0040] Specifically, sparsity processing includes thresholding the initial adjacency data. For example, each sub-adjacency in the initial adjacency matrix is ​​compared with a threshold; sub-adjacency data smaller than the threshold can be discarded or set to 0. Therefore, the larger the threshold, the fewer valid sub-adjacency data remain in the initial adjacency data, and the sparser the generated initial adjacency data (high-level network); conversely, the smaller the threshold, the more valid sub-adjacency data remain in the initial adjacency data, and the denser the generated initial adjacency data (high-level network). The threshold is a value set empirically, and thresholding processing includes binarization methods, weighted network methods, and sparsity methods.

[0041] For binarization methods, sub-neighbor data larger than a threshold can be set to 1, and sub-neighbor data smaller than the threshold can be set to 0. For weighted network methods, sub-neighbor data larger than the threshold is retained as its original value, and sub-neighbor data smaller than the threshold is set to 0. For sparsity methods, only the sub-neighbor data of the top few percent of brain regions with the largest initial neighbor data is retained, and the remaining sub-neighbor data is set to 0.

[0042] The embodiments of this disclosure first sparsify the initial adjacency data before attention fusion, which can remove weakly correlated connections, significantly reducing the number of parameters and memory usage in subsequent mapping and attention calculations. The strong connection edges retained by sparsification make the target adjacency data more focused on the coupling of key brain regions, thereby improving the accuracy of personnel state prediction.

[0043] In some embodiments of this disclosure, the multiple channels include n channels, and the initial adjacency data includes n×n sub-adjacency data; the initial adjacency data is subjected to sparsification processing, including: comparing any one of the n×n sub-adjacency data with a preset functional association strength threshold; when the sub-adjacency data is less than the preset functional association strength threshold, the sub-adjacency data is set to a preset value.

[0044] Specifically, the preset functional association strength threshold is a threshold preset based on experience. For example, the median of n×n sub-adjacent data is taken, and the value that is greater than the median by one standard deviation is used as the preset functional association strength threshold. The n×n sub-adjacent data is compared with the preset functional association strength threshold. When the sub-adjacent data is less than the preset functional association strength threshold, the sub-adjacent data is set to a preset value (such as 0).

[0045] The embodiments of this disclosure compare each of the n×n sub-adjacency data with a preset functional association strength threshold and set the elements below the threshold to a unified preset value. This can eliminate weak connection noise in one go, improve the sparsity of the adjacency matrix, and significantly reduce the number of parameters required for subsequent attention calculations.

[0046] Next, the initial adjacency data after sparsification will be processed. Compared with segmented EEG signals X The data is input into the personnel status prediction model for personnel status prediction processing to obtain the personnel status prediction results.

[0047] Figure 2 This is a schematic diagram of the personnel status prediction model device provided in the embodiments of this disclosure.

[0048] like Figure 2 As shown, the personnel state prediction model device includes an attention network module 210, a first temporal convolutional network module 220, a spatial graph convolutional network module 230, a second temporal convolutional network module 240, and a personnel state prediction network module 250.

[0049] Attention network module 210 is used to perform attention fusion processing on initial adjacency data to obtain target adjacency data, wherein the EEG signals of different channels correspond to different brain regions, and the initial adjacency data represents the functional correlation strength between the EEG signals of different channels.

[0050] The first temporal convolutional network module 220 is used to extract temporal features from EEG signals of different channels to obtain initial temporal features of different channels.

[0051] The spatial graph convolutional network module 230 is used to perform spatial feature fusion on the initial temporal features of different channels based on target adjacency data to obtain spatial fused features.

[0052] The second temporal convolutional network module 240 is used to extract temporal features from spatial fusion features to obtain deep spatiotemporal features.

[0053] The personnel status prediction network module 250 is used to predict personnel status based on deep spatiotemporal features and obtain personnel status prediction results.

[0054] In some embodiments of this disclosure, attention fusion processing is performed on initial adjacency data to obtain target adjacency data, including: performing feature mapping processing on the initial adjacency data based on attention network 210 to obtain at least one mapping processing result, wherein each mapping processing result includes query features, key features, and value features; performing attention calculation on the query features, key features, and value features corresponding to each mapping processing result to obtain at least one attention calculation result; and obtaining target adjacency data based on at least one attention calculation result.

[0055] Specifically, for the initial adjacency matrix Feature mapping is performed, including single-head and multi-head feature mapping. Single-head feature mapping yields one mapping result, while multi-head feature mapping yields multiple mapping results. For single-head feature mapping, attention is calculated on the mapping result to obtain one attention result, which is used as the target adjacency data. For multi-head feature mapping, attention is calculated on multiple mapping results to obtain multiple attention results, which are then concatenated and linearly transformed to obtain the target adjacency data. The initial adjacency data and the target adjacency data have the same feature dimension.

[0056] Assuming initial adjacency data is input The size is n×n, and the feature dimension after attention fusion is... =n.

[0057] For single-head feature mapping, the initial adjacency data of the input... Through three different linear transformation matrices The mapping yields the query feature (Q), key feature (K), and value feature (V), as shown in formula (1): , (1) in ∈ The feature dimension after mapping is still n.

[0058] Attention is calculated on the query feature (Q), key feature (K), and value feature (V), and the result is shown in formula (2): (2) in, This represents the scaled dot product attention, with attention weights normalized using softmax. This represents the result of attention calculation, which can be used as target adjacency data.

[0059] For multi-head feature mapping, assuming there are h heads, the feature dimension of each head is... = The i-th head is transformed by a linear transformation matrix. For initial adjacency data Mapping is performed to obtain query features ( ), key features ( Sum value characteristics ( As shown in formula (3): , (3) in, ∈ , i=1,2,…,h.

[0060] Query features for each head i ( ), key features ( Sum value characteristics ( Attention calculations were performed, and multiple attention calculation results were obtained as shown in formula (4): (4) in, This represents the scaled dot product attention, with attention weights normalized using softmax. This represents the results of multiple attention calculations.

[0061] Feature dimensions of each head = The attention calculation results of each head are concatenated to obtain n. The splicing result is shown in formula (5): (5) in, = This is the attention output of each head, and the concatenated result is n. matrix.

[0062] The concatenated result is then subjected to a linear transformation matrix. ∈ Mapping back to the original feature space, as shown in Equation (6): (6) in, This represents the target adjacency matrix.

[0063] The relationship between any two leads differs under different cognitive load states. For example, under high load, the functional association strength between electrode points in the prefrontal cortex (one brain region) and parietal cortex (another brain region) is higher, while under low load, there is no significant enhancement in functional association strength. Initial adjacency data usually only reflects the pairwise connections between electrode points, but the target adjacency matrix obtained after multi-head attention fusion processing can reveal the connection relationship between the current electrode point and all other electrode points, that is, the functional association strength of the current electrode point in the whole brain, which is closer to the true state of brain network connectivity.

[0064] The embodiments of this disclosure dynamically highlight significant functional connections and suppress noisy edges by mapping initial adjacency data to query-key-value triples and performing multi-level attention calculations, thereby improving the accuracy of target adjacency data. Based on a multi-head attention mechanism, it can capture information at different spatial scales, with each attention head focusing on the strength of functional associations in different spatial ranges. Furthermore, it can form a hierarchical organization, identifying multi-level functional modules from local brain regions to the global network. For example, it can obtain the functional connectivity of the current electrode point in its local brain region, as well as its functional connectivity throughout the whole brain. This hierarchical result can reveal dynamic information about brain network connections, and by identifying brain regions that play key roles in different cognitive states, it reveals the ability of brain regions to switch roles in different tasks. It can obtain detailed results of inter-network coupling, revealing that functional network boundaries are not fixed but change with cognitive load, and identifying specific paths of information transmission between different networks.

[0065] Figure 3 This is a schematic diagram of a method for predicting the state of a person based on brain functional connectivity provided in an embodiment of this disclosure.

[0066] like Figure 3As shown, the personnel state prediction method can be implemented by a personnel state prediction model. The personnel state prediction model can include a graph convolutional neural network model. For example, the personnel state prediction model includes an input network, a spatiotemporal convolutional network, and a personnel state prediction network. The personnel state prediction network can include a flattened layer network. The spatiotemporal convolutional network can include at least one spatiotemporal convolutional block. This embodiment uses a spatiotemporal convolutional network including one spatiotemporal convolutional block as an example for explanation. One spatiotemporal convolutional block includes an attention network (attention layer), a first temporal convolutional network, a first spatial graph convolutional network, and a second temporal convolutional network. The input network is used to input the initial adjacency data after sparsification. Compared with segmented EEG signals X The input is fed into a spatiotemporal convolutional network for attention fusion processing (attention network or attention layer), feature fusion extraction (first temporal convolutional network, first spatial graph convolutional network, second temporal convolutional network), and personnel state prediction processing (personnel state prediction network) to obtain the personnel state prediction result.

[0067] Attention layers can include multi-head attention layers, which process the initial adjacent data. Attention fusion processing is performed to obtain target adjacency data. .

[0068] Based on target adjacency data, feature fusion extraction and personnel state prediction processing are performed on EEG signals to obtain personnel state prediction results. This includes: extracting temporal features from EEG signals of different channels to obtain initial temporal features of different channels; fusing spatial features from the initial temporal features of different channels based on target adjacency data to obtain spatial fusion features; extracting temporal features from the spatial fusion features to obtain deep spatiotemporal features; and predicting personnel state based on the deep spatiotemporal features to obtain personnel state prediction results.

[0069] Specifically, X-rays of EEG signals from different channels The input is fed into a first temporal convolutional network for temporal feature extraction, yielding initial temporal features for different channels. For example, temporal feature extraction is performed on n-channel EEG signals to obtain initial temporal features for each channel. The target adjacency data is then used. The initial temporal features are input into a first spatial convolutional network for spatial feature fusion to obtain spatial fusion features. For example, during spatial fusion, since the target adjacent data represents the degree of EEG correlation between different channels (sub-adjacent data can be regarded as a kind of weight), when calculating the spatial fusion features of a certain channel, if the target adjacent data indicates that the channel has a high degree of correlation with other channels, other channels can be fused with this channel, and the fusion weight can be larger, thereby achieving cross-channel and cross-spatial fusion. The spatial fusion features are input into a second temporal convolutional network for temporal feature extraction to obtain deep spatiotemporal features. The deep spatiotemporal features are input into a flattening layer network for flattening to flatten the deep spatiotemporal features into one-dimensional data, and the softmax of the flattening layer network is used for personnel state prediction.

[0070] Figure 4 This is a schematic diagram of another method for predicting the state of a person based on brain functional connectivity provided in this embodiment of the disclosure.

[0071] like Figure 4 As shown, in Figure 3 Based on the illustrated embodiment, this embodiment uses a spatiotemporal convolutional network comprising two spatiotemporal convolutional blocks as an example for explanation. (Of course, there can be three, four, or more spatiotemporal convolutional blocks). The two spatiotemporal convolutional blocks can share a single attention network or each have its own attention network. The initial adjacency data after sparsification... Compared with segmented EEG signals X The input is fed into a spatiotemporal convolutional network for attention fusion processing (attention network), feature fusion extraction (spatiotemporal convolutional block 1 (first temporal convolutional network, first spatial graph convolutional network, second temporal convolutional network), spatiotemporal convolutional block 2 (third temporal convolutional network, second spatial graph convolutional network, fourth temporal convolutional network)) and personnel state prediction processing (personnel state prediction network) to obtain the personnel state prediction result.

[0072] Attention layers can include multi-head attention layers, which process the initial adjacent data. Attention fusion processing is performed to obtain target adjacency data. .

[0073] Specifically, X-rays of EEG signals from different channels Input is fed into the first temporal convolutional network (with K convolutional kernels). t Temporal feature extraction was performed on channels C1 (32 channels) to obtain the first initial temporal features of different channels (dimension: The first-order temporal convolutional network directly processes the EEG signal X. Convolution is performed, so the corresponding channel C1 can be the same as the channel n mentioned above, and subsequent convolution operations can change the number of channels.

[0074] Target adjacency data and the first initial temporal features (dimension: The input is fed into a first spatial convolutional network (with 32 channels C2) for first spatial feature fusion, resulting in first spatial fused features (dimension: ...). ).

[0075] The first spatial fusion feature (dimension is) Input the second temporal convolutional network (with K convolutional kernels) t Temporal feature extraction was performed on channels C3 (64 channels) to obtain the first deep spatiotemporal feature (dimension: ).

[0076] Let the first depth spacetime dimension be Input a third temporal convolutional network (with K convolutional kernels) t The second initial temporal feature (dimension 64) was obtained by extracting temporal features from channels C4 (64 channels). ).

[0077] The second initial temporal feature (dimension is...) Adjacency data with target The input is fed into a second-space convolutional network (with 32 channels C5) for second-space feature fusion, resulting in second-space fused features (dimension: ...). ).

[0078] The second space fusion feature (dimension is) The input is fed into the fourth temporal convolutional network (the convolutional kernel is K). t Temporal feature extraction was performed on channels C5 (128 channels) to obtain the second deep spatiotemporal feature (dimension: ).

[0079] The second depth spatiotemporal features (dimension: The output is flattened by a flattening layer network to flatten the second-depth spatiotemporal features into one-dimensional data, and the softmax of the flattening layer network is used to predict the state of people.

[0080] The embodiments of this disclosure extract initial temporal features independently from each channel's EEG signal to pre-suppress artifacts and noise within the channel. Then, based on target adjacency data, spatial weighted fusion is performed on the initial temporal features to accurately strengthen effective connections between functionally related brain regions and eliminate redundant edges, reducing parameter scale and computational complexity. Subsequently, deep temporal extraction is performed again on the obtained spatially fused features, and personnel state prediction is performed based on these deep spatiotemporal features, improving the accuracy of personnel state prediction. Convolutional operations using multiple spatiotemporal convolutional blocks enable the extraction of deep-level features, further enhancing prediction accuracy.

[0081] In some embodiments of this disclosure, based on target adjacency data, feature fusion extraction and personnel state prediction processing are performed on EEG signals to obtain personnel state prediction results, including: constructing a corresponding node for each channel to obtain multiple nodes corresponding to multiple channels, and constructing an edge for any two nodes among the multiple nodes; determining the EEG signal of each channel as the node feature of the corresponding node, and determining the edge feature based on the target adjacency data; generating graph structure data based on the node feature and the edge feature; and performing graph convolution processing on the graph structure data to obtain personnel state prediction results.

[0082] Specifically, each channel corresponds to a node, and any two nodes are connected by an edge. Each channel corresponds to an EEG signal X. As a node characteristic of the corresponding node, the target adjacency data As an edge feature, for example, the sub-adjacent data of the i-th channel and the j-th channel. As the edge feature between the i-th node and the j-th node, based on node feature X Sum of edge features The graph-structured data is input into a spatiotemporal convolutional network for feature fusion and extraction, as well as personnel status prediction, to obtain the personnel status prediction results.

[0083] The embodiments of this disclosure construct a graph structure by converting channels into nodes and target adjacency data into edge features. This allows for the unified embedding of EEG signals, spatial topology, and functional coupling strength into the graph domain, thereby achieving high-precision prediction of human state.

[0084] The proposed method for predicting the state of persons based on brain functional connectivity combines brain functional connectivity with graph convolutional networks. It also improves the spatiotemporal convolutional network by adding an attention layer. First, a feature map is obtained on the brain network graph through the attention layer, and then combined with the temporal characteristics, it is input into the network. The residual connection mechanism is used to increase the stability of the network and improve the accuracy of the prediction of the state of persons.

[0085] Figure 5 This is a block diagram of an edge computing device provided in an embodiment of this disclosure.

[0086] like Figure 5 As shown, the edge computing device in this embodiment includes a processor 501 and a memory 502, wherein the memory 502 stores programs or instructions that can run on the processor 501. When the program or instructions corresponding to the edge computing device are executed by the processor 501, the person state prediction method based on brain functional connectivity described above can be implemented.

[0087] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0088] The embodiments of this disclosure have now been described in detail. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein based on the above description.

[0089] While specific embodiments of this disclosure have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments or equivalent substitutions can be made to some technical features without departing from the scope and spirit of this disclosure. In particular, as long as there is no structural conflict, the technical features mentioned in the various embodiments can be combined in any manner.

Claims

1. A method for predicting the state of individuals based on brain functional connectivity, characterized in that, include: The method acquires EEG signals from multiple channels and acquires initial adjacency data, wherein the EEG signals from different channels correspond to different brain regions, and the initial adjacency data represents the functional correlation strength between the EEG signals from different channels. Attention fusion processing is performed on the initial adjacency data to obtain the target adjacency data; Based on the target adjacency data, feature fusion extraction and personnel state prediction processing are performed on the EEG signal to obtain the personnel state prediction result.

2. The method according to claim 1, characterized in that, The step of performing feature fusion extraction and personnel state prediction processing on the EEG signal based on the target adjacency data to obtain the personnel state prediction result includes: Temporal features were extracted from the EEG signals of different channels to obtain the initial temporal features of different channels; Based on the target adjacency data, spatial feature fusion is performed on the initial temporal features of the different channels to obtain spatial fusion features; Temporal feature extraction is performed on the spatial fusion features to obtain deep spatiotemporal features; Based on the aforementioned deep spatiotemporal features, personnel status prediction is performed to obtain the personnel status prediction result.

3. The method according to claim 1, characterized in that, The attention fusion processing of the initial adjacency data to obtain the target adjacency data includes: The initial adjacency data is subjected to feature mapping processing to obtain at least one mapping processing result, wherein each mapping processing result includes query features, key features and value features; Attention is calculated for the query feature, the key feature, and the value feature corresponding to each mapping processing result to obtain at least one attention calculation result; Based on the at least one attention calculation result, the target adjacency data is obtained.

4. The method according to claim 1, characterized in that, Before performing attention fusion processing on the initial adjacency data to obtain the target adjacency data, the method further includes: The initial adjacency data is then subjected to sparsification processing.

5. The method according to claim 4, characterized in that, The plurality of channels includes n channels, and the initial adjacency data includes n×n sub-adjacency data; the sparsity processing of the initial adjacency data includes: Compare any one of the n×n sub-adjacent data with a preset functional association strength threshold; When the sub-adjacent data is less than the preset functional association strength threshold, the sub-adjacent data is set to a preset value.

6. The method according to claim 1, characterized in that, The plurality of channels includes n channels, and the initial adjacency data includes n×n sub-adjacency data, wherein any one of the n×n sub-adjacency data is represented as... Where 1≤i≤n, 1≤j≤n; the process of obtaining initial adjacency data includes: The EEG signals of the i-th channel and the j-th channel are correlated to obtain the sub-adjacent data corresponding to the i-th channel and the j-th channel. .

7. The method according to any one of claims 1-6, characterized in that, The step of performing feature fusion extraction and personnel state prediction processing on the EEG signal based on the target adjacency data to obtain the personnel state prediction result includes: Build a corresponding node for each channel to obtain multiple nodes corresponding to the multiple channels, and build an edge for any two nodes among the multiple nodes; The EEG signal of each channel is determined as the node feature of the corresponding node, and the edge feature is determined based on the target adjacency data; Graph structure data is generated based on the node features and edge features; The graph structure data is subjected to graph convolution processing to obtain the personnel state prediction result.

8. A device for predicting human state based on brain functional connectivity, characterized in that, include: An attention network module is used to perform attention fusion processing on initial adjacency data to obtain target adjacency data, wherein the EEG signals of different channels correspond to different brain regions, and the initial adjacency data represents the functional correlation strength between the EEG signals of different channels. The first temporal convolutional network module is used to extract temporal features from the EEG signals of different channels to obtain the initial temporal features of different channels. A spatial graph convolutional network module is used to perform spatial feature fusion on the initial temporal features of the different channels based on the target adjacency data to obtain spatial fusion features; The second temporal convolutional network module is used to extract temporal features from the spatial fusion features to obtain deep spatiotemporal features; The personnel status prediction network module is used to predict personnel status based on the deep spatiotemporal features and obtain personnel status prediction results.

9. An edge computing device, characterized in that, The method includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method according to any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-8.

Images