EEG signal analysis and decoding service platform

Abstract

This invention relates to the fields of electroencephalogram (EEG) signal processing and brain-computer interface (BCI) technology, specifically to an EEG signal analysis and decoding service platform. The platform includes an EEG signal acquisition module, an action fingerprint map construction module, and a Bayesian path selection and online decoding module. The acquisition module extracts Riemann trajectory fingerprint vectors from EEG signals and calculates fingerprint statistics; the map construction module uses actions as nodes and clusters similar actions into multiple action groups based on Wasserstein distance and graph convolutional networks; the decoding module calculates group probabilities and action probabilities, updates the probability vectors through Bayesian recursion, and outputs action labels, which are then converted into control commands for the intelligent prosthesis. This invention, through Riemann trajectory fingerprint extraction, adaptive map clustering, and Bayesian recursive decoding, enables brain-controlled intelligent prostheses to accurately recognize various fine movements such as finger bending and wrist rotation, solving the problems of limited action types, low recognition rates, and difficulty in expansion associated with traditional methods.

Description

Technical Field

[0001] This invention relates to the field of electroencephalogram (EEG) signal processing and brain-computer interface technology, and more specifically, to an EEG signal analysis and decoding service platform. Background Technology

[0002] Electroencephalogram (EEG) signals are a comprehensive reflection of the electrical activity of neurons in the brain on the surface of the scalp. They can represent the user's thought intentions and neural state in real time. Effective analysis and decoding of EEG signals is a key link in realizing brain-computer interface technology and human-computer interaction applications.

[0003] In brain-controlled intelligent prosthetic applications, users generate corresponding brain signals by imagining different limb movements. The system then decodes these brain signals and converts them into control commands for the prosthetic limb.

[0004] However, when it comes to six or more types of fine control movements, such as finger bending and wrist rotation, the differences in EEG characteristics between different movements are subtle, making decoding and recognition difficult and easily leading to a decrease in accuracy. This makes it impossible to meet the high-precision control requirements of multiple free limbs.

[0005] To address the above problems, existing technologies mainly employ the following two methods: ① The complex decoding task of multi-category fine motor skills of prostheses is broken down into several binary sub-tasks. Spatial and temporal features of brain waves are extracted, and then machine learning is combined to complete the signal discrimination and decoding of each sub-task, thereby reducing the difficulty of multi-category synchronous recognition. ② Adopting a coarse-to-fine hierarchical decoding logic, first, the limb movements corresponding to the EEG signals are classified into broad categories, and then fine movements are subdivided and identified within each category, thus completing the EEG decoding of multi-dimensional fine movements step by step. However, both of the above solutions have certain limitations: The task splitting and decoding method generates a large number of subclassifiers as the number of action categories increases, resulting in high computational overhead. Furthermore, the subtasks are optimized independently, making it impossible to effectively utilize the correlation features of EEG signals from different actions. The hierarchical decoding method relies heavily on human experience to group actions, and the errors generated in the coarse classification stage will accumulate and propagate layer by layer, ultimately affecting the overall decoding accuracy of fine actions. Therefore, there is an urgent need for an EEG signal analysis and decoding scheme that balances computational efficiency and decoding accuracy and is compatible with the recognition of various fine motor actions. Summary of the Invention

[0006] This invention provides a brainwave signal analysis and decoding service platform. It converts motor imagery brainwave signals into Riemann trajectory fingerprints, automatically constructs action fingerprint maps based on fingerprint similarity, clusters similar actions, and then progressively updates the probability estimates for each action using a Bayesian recursive model. Finally, it outputs action labels according to decision rules and converts them into control commands for an intelligent prosthetic limb, thereby solving the problems mentioned in the background art, namely: In the typical human-computer interaction application of brain-controlled intelligent prostheses, when it is necessary to recognize more than six types of fine movements such as finger bending and wrist rotation, the differences in EEG characteristics between different movements are slight, and existing technologies are difficult to achieve high-accuracy decoding and are difficult to flexibly expand to new movements.

[0007] To achieve the above objectives, the service platform includes an EEG signal acquisition module, which is used to acquire the raw EEG signals of the user when performing fine motor imagery tasks and extract Riemann trajectory fingerprint vectors from them. Based on multiple Riemann trajectory fingerprint vectors of the same category, the platform calculates the fingerprint mean vector and scatter matrix of that category of fingerprints. The platform also includes an action fingerprint map construction module and a Bayesian path selection and online decoding module. The action fingerprint map construction module takes each fine action as a node, calculates the Wasserstein distance between nodes based on the fingerprint mean vector and scatter matrix, and constructs the adjacency matrix of the undirected graph. Based on the adjacency matrix, a graph convolutional network is used to learn the embedding of nodes to obtain node embedding vectors. By maximizing the modularity, the node embedding vectors are clustered into multiple action groups, and the embedding center and group radius of each action group are calculated. The Bayesian path selection and online decoding module calculates the group probability of the current fingerprint belonging to each action group and the action probability of each fine action based on the Riemann trajectory fingerprint vector of the current test segment and the embedding center and group radius of each action group. Maintaining an initial uniformly distributed probability vector, Bayesian recursion is performed in the current test segment based on the probability vector before the update, the current action probability, and the pre-statistical action transition probability matrix to obtain the probability vector of the current test segment. Action labels are output according to the decision rules, converted into control commands, and sent to the intelligent prosthesis to drive the prosthesis to perform corresponding fine movements.

[0008] In the aforementioned technical solution, a large number of fine motor actions are first automatically clustered into multiple action groups based on the similarity of EEG features. Without this clustering process, each fine motor action is treated in isolation. As the number of action categories increases, the number of classification models required will increase significantly, and the correlation information between similar actions cannot be used to enhance the discriminative power. Based on this, this solution uses a probabilistic recursive approach to fuse historical information and current observations, avoiding the blindness of making judgments based on a single trial. Without this probabilistic recursive mechanism, if the fingerprint of the current trial is similar to a certain action part, it is easy to directly misjudge, making it impossible to accumulate evidence through multiple trial segments to correct errors, and also making it difficult to wait for more information or trigger resampling when confidence is insufficient. These two steps work together; the former decomposes the complex problem into a hierarchical recognition structure, and the latter achieves robust dynamic decision-making on this structure, thereby solving the problems of low decoding accuracy and poor scalability of multi-class fine motor actions.

[0009] Based on this, the graph convolutional network in the action fingerprint map construction module includes two graph convolutional layers. The first layer multiplies the normalized adjacency matrix with the node feature matrix and then multiplies it by the first layer weight matrix, and applies a linear rectified activation function to the product result to obtain the first layer node embedding matrix. The second layer multiplies the normalized adjacency matrix with the first layer node embedding matrix and then multiplies it by the second layer weight matrix to obtain the second layer node embedding matrix as the final node embedding vector.

[0010] In another technical solution, when the action fingerprint map construction module performs clustering by maximizing modularity, each node is regarded as an independent community. The node is moved to the adjacent community in turn and the modularity after the move is calculated. If the modularity after the move is greater than that before the move, the move is retained; otherwise, the move is canceled. The above process is repeated until the modularity no longer increases, thus completing the clustering of the node embedding vector.

[0011] This technical solution employs a two-layer graph convolutional network, with the second layer not using an activation function. This design aims to preserve node features while avoiding excessive nonlinearization, ensuring that the embedding vectors more accurately reflect the original similarity relationships between nodes. If an activation function were added to the second layer, subtle differences in features might be suppressed by the nonlinear transformation, causing actions that were originally clearly distinguishable to become blurred in the embedding space. The clustering step uses iterative node movement and real-time calculation of modularity gain. Each step only accepts moves that improve modularity, ensuring that the final partitioning result is reasonable within the context of local optima. If a one-time partitioning is used without iterative adjustments, the clustering result is easily influenced by the initial settings, making it difficult to converge to a stable grouping structure. The two-layer graph convolutional network and iterative modularity optimization work together; the former provides high-quality node embeddings, while the latter finds the optimal community partitioning based on these, jointly guaranteeing the accuracy and stability of the action fingerprint map.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention first maps motor imagery EEG signals to trajectories on Riemannian manifolds and extracts geometric fingerprints, amplifying the subtle differences between different fine movements into distinguishable feature vectors, thus solving the problem of insufficient separability of traditional time-frequency spatial domain features for more than 6 types of fine movements. On this basis, it automatically constructs action fingerprint maps based on fingerprint similarity and performs adaptive clustering, dividing a large number of fine movements into multiple action groups, mitigating the defect of existing technologies where the number of classification models increases significantly due to the increase in action categories.

[0013] 2. This invention employs a Bayesian recursive model to fuse historical probabilities with current observations and uses a dual-threshold decision rule to determine the timing of output. This avoids blind judgment in a single trial and allows for three different strategies—output, waiting, or resampling—based on confidence levels, thus achieving a balance between accuracy and response speed in human-computer interaction. Furthermore, the joint judgment mechanism of group probability and action probability further ensures the reliability of the output results.

[0014] 3. This invention supports incremental registration of new actions and adaptive adjustment of decision thresholds. Users only need a small number of samples to expand new fine-grained actions. The system can also continuously optimize the grouping structure and decoding parameters based on feedback during use, so that the service platform can maintain stable decoding performance for different users and at different stages of use, and has good scalability and individual adaptability. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the overall structure of the EEG signal analysis and decoding service platform of the present invention; Figure 2 This is a schematic diagram illustrating the working principle of the EEG signal analysis and decoding service platform of the present invention; Figure 3 This is a flowchart illustrating the structure of the action fingerprint mapping construction module of the present invention. Figure 4 This is a flowchart illustrating the structure of the Bayesian path selection and online decoding module of the present invention.

[0016] The meanings of the labels in the diagram are as follows: 100. EEG signal acquisition module; 200. Action fingerprint map construction module; 300. Bayesian path selection and online decoding module; 400. Map evolution and parameter update module. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Here are some explanations of technical terms: Mü rhythms refer to brain electrical rhythms with a frequency of 8 to 12 Hz, which mainly occur in the sensorimotor cortex of the brain; beta rhythms refer to brain electrical rhythms with a frequency of 13 to 30 Hz, which are closely related to the brain's active motor intentions and attentional state. Affine-invariant Riemann distance is a distance metric on Riemannian manifolds used to calculate the degree of difference between two covariance matrices. Its characteristic is that it is invariant to affine transformations of the covariance matrix, and can more accurately reflect the true geometric distance between the covariance matrices of EEG signals. Modularity is an indicator for measuring the quality of network community segmentation. Its value ranges from -0.5 to 1. The larger the value, the tighter the connection within the community and the sparser the connection between communities.

[0019] Currently, in the typical human-computer interaction application of brain-controlled intelligent prostheses, when it is necessary to recognize more than six types of fine movements such as finger bending and wrist rotation, the differences in EEG characteristics between different movements are slight. Existing technologies struggle to achieve high-accuracy decoding and are difficult to flexibly extend to new movements. This invention provides an EEG signal analysis and decoding service platform, see [link to relevant documentation]. Figure 1 , Figure 2 As shown, it includes an EEG signal acquisition module 100, an action fingerprint map construction module 200, a Bayesian path selection and online decoding module 300, and a map evolution and parameter update module 400.

[0020] By converting motor imagery EEG signals into trajectories on Riemannian manifolds and extracting geometric fingerprints, an action fingerprint atlas is automatically constructed based on the similarity between fingerprints, and similar fine movements are adaptively aggregated. During online decoding, a Bayesian recursive model is used to progressively update the probability estimates of each movement, and a dual threshold mechanism is combined to determine the output timing. Finally, the recognition results are sent to the intelligent prosthesis, achieving high accuracy and scalable EEG decoding services for multiple types of fine movements.

[0021] In this embodiment, the EEG signal acquisition module 100 first uses a multi-lead EEG acquisition device to acquire the EEG signals generated when the user performs a fine motor imagination task. The module specifically includes a signal receiving and preprocessing unit, a trajectory fingerprint generation unit, and a fingerprint packaging unit.

[0022] The signal receiving and preprocessing unit first acquires the user's brain signals when performing fine motor imagery tasks through a multi-channel EEG acquisition device.

[0023] Specifically, the signal receiving and preprocessing unit is connected to an EEG cap worn on the user's head. The EEG cap has multiple acquisition electrodes, each corresponding to an independent acquisition channel. These channels cover areas of the cerebral cortex related to motor imagery. The unit receives raw EEG signals from these channels, and the signal from each channel records the potential changes of the electrical activity of neurons in the brain at the corresponding electrode location.

[0024] Furthermore, the signal receiving and preprocessing unit performs independent component extraction on the received raw EEG signal to separate independent source signal components from the mixed signal. The operation steps are as follows: ① The raw EEG signal is modeled as a linear mixture of multiple independent source signals. Specifically, the observed multichannel EEG signal is considered to be generated by a linear combination of several statistically independent source signals through an unknown mixing matrix. These source signals include brain neural electrical activity, electrooculography artifacts, electromyography artifacts, and environmental noise.

[0025] ② The original EEG signal is processed to remove the mean. The time average value of the signal in each channel is calculated and then subtracted from each sampling point of each channel to obtain a signal matrix with zero mean.

[0026] ③ Whitening is performed on the zero-mean signal matrix. Specifically, first, the covariance matrix of the signal matrix is ​​calculated; then, eigenvalue decomposition is performed on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors; a whitening matrix is ​​constructed based on the eigenvalues ​​and eigenvectors, specifically by multiplying the reciprocal of the square root of the eigenvalue by the corresponding eigenvector; finally, the original zero-mean signal matrix is ​​multiplied by the whitening matrix to obtain the whitened signal.

[0027] ④ A fixed-point iterative algorithm is used to extract independent components from the whitened signal. Specifically, this algorithm employs a fast independent component analysis method, using negative entropy as a measure of non-Gaussianity. The approximate formula for calculating negative entropy is: ; In the formula, w is the weight vector to be solved; z is the whitened signal vector; v is a Gaussian variable with zero mean and unit variance; G(·) is a non-quadratic function, usually taken as or ; E{·} represents the mathematical expectation.

[0028] The larger the negative entropy value, the stronger the non-Gaussianity of the projected signal, and the closer it is to the independent source signal.

[0029] The specific extraction steps are as follows: For the first independent component, randomly initialize a weight vector w1 of unit length, and then iteratively update it according to the following formula: ; In the formula, g(·) is the first derivative of G(·); g'(·) is the second derivative of G(·).

[0030] After each iteration, for Normalization is performed: .

[0031] The convergence condition for the iteration is that the absolute value of the dot product between two consecutive updated weight vectors is close to 1, i.e. Or, until the preset maximum number of iterations is reached, such as 1000 times.

[0032] Then, the weight vector is multiplied with the whitened signal to obtain the time series of the first independent component. After obtaining the first independent component, the orthogonalization method is used to eliminate the influence of the extracted components on the subsequent extractions, so as to ensure that the extracted components are independent of the extracted components.

[0033] It should be noted that when using the orthogonalization method, after extracting the weight vector corresponding to each independent component, this weight vector is projected onto the orthogonal complement space spanned by all the extracted weight vectors. Specifically, when extracting the second independent component, after each iteration updating the current weight vector, the inner product of the current weight vector and the first extracted weight vector is calculated. Then, the component of the first weight vector multiplied by the inner product is subtracted from the current weight vector to make the current weight vector orthogonal to the first weight vector. Finally, the orthogonalized weight vector is normalized.

[0034] When extracting the third and subsequent independent components, the current weight vector is subtracted sequentially from its projected components in the direction of each extracted weight vector, and then normalization is performed. This orthogonalization process ensures that the extraction directions are mutually orthogonal, thus making the independent components statistically independent.

[0035] Repeat the above process of randomly initializing the weight vector, updating iteratively using Newton, and orthogonalizing to extract the weight vectors corresponding to all independent components in turn.

[0036] ⑤ Combine the weight vectors obtained in each iteration into a demixing matrix. Perform a linear transformation on the whitened signal using the demixing matrix to obtain a matrix of independent source signal components. Each row of this matrix corresponds to a time series of an independent source signal, i.e., an independent component.

[0037] The time series of each independent component contains the time-domain waveform information of that component. After performing a Fourier transform on the time series, its frequency-domain energy distribution is obtained. Based on the time-domain waveform characteristics and frequency-domain energy distribution of each independent component, the signal receiving and preprocessing unit identifies the components corresponding to electrooculography (EOG) artifacts and electromyography (EMG) artifacts.

[0038] Among them, the time-domain waveform of the electrooculography artifact component is characterized by large-amplitude periodic fluctuations in the low-frequency band, and its waveform shape is synchronized with the user's blinking action in time; the time-domain waveform of the electromyography artifact component is characterized by short-term bursts in the high-frequency band, and its spectral energy is mainly distributed in the high-frequency range.

[0039] Furthermore, the rows corresponding to the identified electrooculography (EOG) and electromyography (EMG) artifacts in the independent component matrix are all set to zero, while the rows corresponding to brain neural electrical activity remain unchanged.

[0040] Based on this, the signal receiving and preprocessing unit performs an inverse transformation on the zeroed independent component matrix using a hybrid matrix, transforming the signal from the independent component space back to the original channel space, and reconstructing a pure EEG signal matrix after artifact removal.

[0041] After blind source separation and artifact removal are completed, the signal receiving and preprocessing unit performs bandpass filtering on the pure EEG signal. A digital filter is used, with the passband frequency set to 8 to 30 Hz and the stopband attenuation to 60 dB.

[0042] This passband covers the Mü and Beta rhythms, which are closely related to motor imagery tasks, effectively preserving useful neural electrical activity components while filtering out low-frequency drift and high-frequency interference outside the passband. The filtered, clean EEG signal achieves a further improved signal-to-noise ratio while maintaining the characteristics relevant to motor imagery.

[0043] Furthermore, the signal receiving and preprocessing unit extracts the corresponding signal segment from the filtered signal based on the start and end times of each test segment. The start time of each test segment is triggered by a prompt signal, and the end time is determined by a preset duration. The extracted signal segments are then arranged into a lead-time matrix according to lead order and time order, where the first dimension is the lead index and the second dimension is the sampling time point.

[0044] The lead time matrix corresponding to each test segment is output to the trajectory fingerprint generation unit in the order of test execution.

[0045] Based on this, the trajectory fingerprint generation unit processes the lead time matrix of each test segment. This unit first divides the continuous signal into several overlapping time windows according to a fixed window length and a fixed step size.

[0046] Specifically, let the window length be L sampling points and the step size be S sampling points. The step size is smaller than the window length to ensure that there is an overlapping area between adjacent windows. The signal data in the k-th window is denoted as matrix X. k The number of rows is C, and the number of columns is L, which is the window length.

[0047] The covariance matrix of the signal matrix within each time window is calculated using the following formula: ; In the formula, Let X be a matrix k The transpose of X, i.e., the matrix of X k The new matrix is ​​obtained by swapping the rows and columns.

[0048] Calculated P k It is a positive definite symmetric matrix with C rows and C columns. Its diagonal elements reflect the signal energy of the corresponding leads, and its off-diagonal elements reflect the signal correlation between different leads.

[0049] Furthermore, all covariance matrices obtained from each experimental segment are arranged in chronological order to form a covariance matrix sequence. , where M is the total number of time windows within the test segment.

[0050] This unit treats all covariance matrices as points on a Riemannian manifold. Taking the first covariance matrix P1 as a reference point, each covariance matrix P... k Mapped into the tangent space of the reference point, the formula for calculating the mapped tangent vector is as follows: ; In the formula, log represents the matrix logarithm operation.

[0051] After the above mapping, each covariance matrix is ​​transformed into a symmetric matrix form of tangent vector. Connecting the tangent vectors of adjacent time windows in chronological order yields a trajectory on the Riemannian manifold, which represents the dynamic evolution path of the user's EEG state during a single imagination task.

[0052] Based on this, three types of geometric features are extracted from the above trajectories: ① A geodesic curvature sequence used to describe the degree of curvature of a trajectory on a manifold.

[0053] Specifically, for three adjacent points on the trajectory Its geodesic curvature The calculation formula is as follows: ; In the formula, Δt is the time interval between adjacent time windows; δ R The formula for calculating the affine-invariant Riemann distance is as follows: ; In the formula, The Frobenius norm of a matrix is ​​the square root of the sum of the squares of all its elements, used to measure the overall size of the matrix.

[0054] ② The total arc length of the trajectory used to reflect the intensity of changes in EEG state throughout the entire imagination task is calculated using the following formula: ;

[0055] ③ The torsion sequence of the trajectory used to describe the tendency of the trajectory to deviate from the local plane, with torsion τ k The calculation formula is: ; In the formula, V k Let be the tangent vector at the k-th point; det represents the determinant of a matrix; × represents the cross product of vectors.

[0056] A larger torsion indicates that the trajectory has been distorted at that location, suggesting a reversal in the direction of the change in the brain electrical state.

[0057] The trajectory fingerprint generation unit concatenates the geodesic curvature sequence, total arc length, and torsion sequence into a one-dimensional numerical vector in sequence, and defines the vector as the Riemann trajectory fingerprint of the imagined action corresponding to the current test segment; at the same time, the unit directly outputs the Riemann trajectory fingerprint vector of the current test segment to the subsequent Bayesian path selection and online decoding module 300.

[0058] Furthermore, the fingerprint packaging unit collects Riemann trajectory fingerprints from multiple test segments within the same fine-machining category. Specifically, suppose there are N test segments in a certain fine-machining category, and the Riemann trajectory fingerprint vector of the i-th test segment is denoted as f. i The mean vector μ of this type of fingerprint is calculated using the following formula: ; The fingerprint scatter matrix ∑ is calculated using the following formula: ; The fingerprint packaging unit encapsulates the mean vector and scatter matrix of all fine action categories into a fingerprint feature package, and the EEG signal acquisition module 100 transmits the fingerprint feature package to the action fingerprint map construction module 200.

[0059] like Figure 3 As shown, the action fingerprint map construction module 200 receives fingerprint feature packets from the EEG signal acquisition module 100. This module consists of a node construction unit, a similarity calculation unit, a graph convolution embedding unit, and an adaptive clustering unit.

[0060] The node construction unit extracts fingerprint statistics for each fine-move category from the fingerprint feature package. Specifically, assuming there are K fine-move categories, the fingerprint mean vector of the i-th category is denoted as μ. i The fingerprint scatter matrix is ​​denoted as S. i This unit defines each fine action category as a node, and the features of the node are composed of the mean vector and scatter matrix of the fingerprint of that category.

[0061] Furthermore, the similarity calculation unit calculates the similarity between any two nodes to construct an association graph between actions.

[0062] Based on this, the similarity calculation unit uses the Wasserstein distance to measure the degree of difference between the fingerprint distributions of two actions. The fingerprints of each action category are assumed to follow a Gaussian distribution. Under this condition, the Wasserstein distance has a closed-form solution, and the calculation formula is as follows: ; In the formula, This represents the square of the Wasserstein distance between the i-th node and the j-th node; The Euclidean norm of a vector; Tr represents the trace operation of a matrix; It is the square root matrix of the scatter matrix.

[0063] The smaller the Wasserstein distance, the more similar the fingerprint distributions of the two actions are.

[0064] Next, the Wasserstein distance is converted into similarity weights using the following formula: ; In the formula, σ is a scale parameter used to control the rate at which the similarity weight decays as the Wasserstein distance increases. The larger the value of σ, the slower the decay, and the smaller the value of σ, the faster the decay.

[0065] Similarity weight w ij The value ranges from 0 to 1. The smaller the w ij The larger the value, the more similar the fingerprint distributions of the two actions. The similarity weights between all pairs of nodes form a K-row, K-column weight matrix W.

[0066] Furthermore, the similarity calculation unit constructs an undirected graph based on the weight matrix to describe the fingerprint similarity relationship between each fine action, which serves as the input to the subsequent graph convolution embedding unit.

[0067] Specifically, first set a similarity threshold θ w Then, the similarity weight w between each pair of nodes is determined one by one. ij : When w ij Greater than θ w When connecting the i-th node and the j-th node, an edge is formed, and the weight of the edge is w. ij ; When w ij Less than or equal to θ w At this time, no edge is connected between the two nodes.

[0068] After traversing all node pairs, we obtain an undirected graph containing K nodes and several edges.

[0069] This undirected graph intuitively reflects the similarity between fingerprints of different fine movements; the closer the connection between movements, the closer their fingerprint distribution.

[0070] The graph convolutional embedding unit performs graph convolution operations on the constructed undirected graph, mapping each action node to a low-dimensional embedding space. Specifically, it includes the following steps: ① Obtain the adjacency matrix of the undirected graph. Specifically, during the construction of the undirected graph in the similarity calculation unit, it has been determined whether there is an edge between each pair of nodes and the weight of the edge. This information is organized into a K-row, K-column adjacency matrix, where the element in the i-th row and j-th column represents the similarity weight w between the i-th node and the j-th node. ij The value of a position without an edge connection is 0.

[0071] ② Normalize the adjacency matrix. Specifically, let the adjacency matrix be A. First, add the identity matrix I to the adjacency matrix A to obtain... This adds an edge pointing to itself to each node; then calculate... degree matrix degree matrix It is a diagonal matrix, and its i-th diagonal element is equal to The sum of all elements in the i-th row.

[0072] Finally, the normalized adjacency matrix is ​​calculated according to the following formula: .

[0073] ③ Construct a node feature matrix X. Specifically, the fingerprint mean vector of each action node is used as the initial feature of that node. The features of all nodes are stacked in node order to form a feature matrix X with K rows and D columns, where D is the dimension of the fingerprint mean vector.

[0074] ④ Perform the first layer of graph convolution on the node feature matrix. Specifically, this involves converting the normalized adjacency matrix... Multiply by the feature matrix X, then multiply by the weight matrix W of the first layer. (1) Then, a linear rectified activation function is applied to the product result to obtain the node embedding matrix H of the first layer. (1) : ; In the formula, ReLU is a linear rectified activation function used to set negative numbers in the matrix to zero and keep positive numbers unchanged.

[0075] ⑤ Perform a second-layer graph convolution on the node embedding matrix of the first layer. This unit will normalize the adjacency matrix. With the node embedding matrix H of the first layer (1) Multiply by, then multiply by the weight matrix W of the second layer. (2) The node embedding matrix H of the second layer is obtained. (2) : ; Unlike the first layer, the second layer of graph convolution does not use an activation function and directly outputs the result after linear transformation in order to retain more complete node feature information for subsequent clustering.

[0076] The training method for graph convolutional networks is as follows: First, the training data uses multiple sets of fine motor imagery test data collected offline by users, and the fingerprint mean vector μ for each action category is extracted according to the aforementioned process. i and scatter matrix S i Construct the node feature matrix X and adjacency matrix Using these as training samples, the action category label corresponding to each node is known and used for supervised training.

[0077] Next, the cross-entropy loss function is used to measure the accuracy of classification after node embedding. The node embedding matrix H output by the second layer graph convolution is then used. (2) Input a softmax classification layer, output the probability of each node belonging to each action class, and then calculate the cross-entropy loss with the true label. The calculation formula is as follows: ; In the formula, K is the number of nodes; C represents the total number of action categories; y ic One-hot encoding for the real label; This represents the predicted probability output by the softmax function.

[0078] Furthermore, the Adam optimizer is used for gradient descent updates, with an initial learning rate of 0.01 and weight decay of 0.0005, and 200 training iterations are performed. During each training iteration, the complete node feature matrix X and adjacency matrix are... The input network is forward-propagated to obtain the prediction results, the loss is calculated, and then the weight matrix W is updated via backpropagation. (1) and W (2) .

[0079] After training is complete, the weight matrix parameters are fixed for subsequent node embedding inference.

[0080] It should be noted that the above training process is completed during the offline phase of the system and will not be updated during online decoding to ensure the real-time performance and stability of the decoding.

[0081] Based on this, H is calculated by the second layer graph convolution. (2) This is the final node embedding matrix, where each row corresponds to the coordinate vector of an action node in the low-dimensional embedding space. The graph convolutional embedding unit stores these coordinate vectors in node order for use by the subsequent adaptive clustering unit.

[0082] The adaptive clustering unit clusters nodes in the embedding space, grouping similar fine-grained actions into the same action group. Specifically, each action node is first treated as an independent community, resulting in k communities, each containing only one node.

[0083] Furthermore, the modularity of the current community division is calculated. Modularity is a commonly used indicator to measure the quality of community division. Its value ranges from -0.5 to 1. The larger the value, the tighter the connection between nodes within the community and the sparser the connection between different communities.

[0084] Based on this, each node is traversed sequentially, and the current node is moved from its current community to an adjacent community. After each move, the modularity of the current partition is recalculated, and the modularity after the move is compared with the modularity before the move.

[0085] It should be noted that if the modularity after the move is greater than the modularity before the move, the move is retained; if the modularity after the move does not increase, the move is canceled, and the node is moved to the next adjacent community.

[0086] The adaptive clustering unit repeats the above movement operation for all nodes. After completing one round of traversal, if any node has been moved in this round, the next round of traversal begins; if no node has been moved in this round, it means that the modularity has reached the maximum value, and the iteration stops.

[0087] At this point, the adaptive clustering unit has obtained the final community partitioning result, containing a total of M communities, each community corresponding to an action group, denoted as . .

[0088] Furthermore, the embedding center and group radius of each action group are calculated.

[0089] The embedding center is the average of the embedding vectors of all nodes within the group, and its calculation formula is as follows: ; The group radius is the maximum distance from the embedding vector of a node within the group to the embedding center, i.e. .

[0090] Finally, the adaptive clustering unit establishes a mapping table from each fine-grained action to its corresponding action group, encapsulates the embedding center, group radius, and mapping table of each action group into an action graph package, and passes it to the Bayesian path selection and online decoding module 300.

[0091] like Figure 4 As shown, the Bayesian path selection and online decoding module 300 receives the action map packet output by the action fingerprint map construction module 200, and the Riemann trajectory fingerprint vector of the current test segment output by the EEG signal acquisition module 100, denoted as f. test This module mainly consists of a probability calculation unit, a probability update unit, and a dual-threshold decision unit. Through the collaborative work of the probability update unit and the dual-threshold decision unit, it completes the decoding of the EEG signal of the current test segment and outputs the corresponding fine motor labels.

[0092] The probability calculation unit first reads the fingerprint statistics of each action group from the action map package, including the mean vector and scatter matrix of all fingerprints in the group.

[0093] Specifically, for the m-th action group, the probability calculation unit uses a Gaussian distribution model to describe the distribution characteristics of the fingerprint within the group, and calculates the current fingerprint vector f according to the following formula. test The probability density value belonging to this action group is denoted as the group probability. : ; In the formula, D is the dimension of the fingerprint vector; μ m and S m These are the fingerprint mean vector and scatter matrix of the m-th action group, respectively; The determinant of the scatter matrix.

[0094] Furthermore, the fingerprint mean vector and scatter matrix for each fine-grained action category are read from the action graph package. For the i-th fine-grained action, this unit also uses a Gaussian distribution model to describe the distribution characteristics of its fingerprint, and calculates the current fingerprint vector f using the following formula. test The probability density value belonging to this action is denoted as the action probability. : ; In the formula, μ i and S i Let be the fingerprint mean vector and scatter matrix of the i-th fine action, respectively.

[0095] The probability calculation unit will calculate the group probabilities. and action probability Output to subsequent steps.

[0096] The probability update unit receives the action probability. This is used to successively update the probability estimates for each fine-grained action. This unit maintains a probability vector b of length K. t Where K is the total number of fine movements, and the probability vector b t The i-th element represents the probability that the system considers the current action to be the i-th fine action at time t of the trial period. The specific update steps are as follows: ① Initialize the probability vector: First, initialize the probability vector b t Assume a uniform distribution, with the initial probability of each action being 1 / K; ② Determine the action transition relationship. Specifically, construct a K-order transition matrix, where the element in the j-th row and i-th column is denoted as... , representing the probability of switching from the j-th action in the previous test segment to the i-th action in the current test segment; ③ Calculate the prior probability at the current time step: combine with the probability vector b from the previous time step. t-1 Based on the above transition relationship, solve for the prior probability of the i-th type of action. : ; ④ The probability of the action With corresponding prior probability Multiplying these together yields the unnormalized posterior probability of the i-th action class: ; ⑤ Normalize the unnormalized posterior probabilities of all actions to obtain the final probability of the i-th type of action at time t: ; Combining steps ② to ⑤ above, we obtain the Bayesian recurrence formula as follows: ; .

[0097] At this point, the probability update unit has completed the initial decoding of the EEG signal for the current test segment, obtaining the updated probability vector b for each action. t The probability vector is then output to the dual-threshold decision unit.

[0098] The dual-threshold decision unit receives the updated probability vector b from the probability update unit. t and group probabilities from the probability calculation unit It is used to determine whether to output the decoding result and which fine action to output based on the current probability distribution and the degree of group matching.

[0099] This unit sets three thresholds: the output threshold δ out Waiting threshold δ wait The probability threshold δ of the group group , where δ out Greater than δ wait In the initial state, δ out Set to 0.7, δ wait Set to 0.4, δ group Set it to 0.5.

[0100] The basis for this value is as follows: when the output threshold is 0.7, the decoding accuracy and output rate reach a better balance, requiring the action probability to reach a high confidence level before outputting, in order to avoid misjudgment; when the waiting threshold is 0.4, it can effectively filter low-confidence samples and avoid excessive waiting. If it is lower than this value, it is considered that the information is insufficient and needs to be re-collected; when the group probability threshold is 0.5, it can ensure that the consistency between the output action and the action group to which it belongs is not less than half, requiring that the current fingerprint matches the overall distribution of the selected action group by not less than half, in order to further ensure the reliability of the output.

[0101] The specific decision-making steps of the dual-threshold decision unit are as follows: ①From the updated probability vector b t The maximum value searched in the middle is denoted as And record the action index i* corresponding to the maximum value.

[0102] ② Locate the action group G to which the action belongs based on the action index i*. m And retrieve the group probability corresponding to the action group. . ③ Perform graded judgment based on various thresholds: like and This indicates that the matching degree between the current feature and the corresponding action and action group has met the standard. The fine action corresponding to the action index i* is output as the decoding result. This result is the action label of the current test segment. The probability vector is reset to a uniform distribution and waits to process the next test segment. like and ,or If the current matching result does not meet the output conditions, the system enters a waiting mode, retains the existing probability vector, and uses it as the prior probability for the next test segment to participate in the iterative update. like If the system determines that there is insufficient valid information and reliable identification cannot be completed, the dual-threshold decision unit issues a resampling instruction to prompt the user to re-execute the action imagination task and resets the probability vector to a uniform distribution.

[0103] ④ Count the number of test segments that consecutively enter the waiting mode. When three consecutive segments are in the waiting state and the maximum probability does not increase significantly, output the threshold δ. out The threshold is lowered by 0.05, with a lower limit of 0.5. If the group probability continues to fail to meet the judgment requirements, the group probability threshold δ is appropriately lowered. group Its lower limit is 0.3. The adjusted threshold is applied to the decision-making process of all subsequent test segments.

[0104] It should be noted that the output threshold is lowered by 0.05 each time because an excessively large step size would lead to an overly lenient output, increasing the risk of misjudgment, while an excessively small step size would result in a slow convergence speed. 0.05 strikes a balance between response speed and accuracy. The lower limit of the output threshold is set to 0.5 because the confidence of the output result is too low below 0.5, making it difficult to guarantee the reliability of decoding. The lower limit of the group probability threshold is set to 0.3 because the matching degree between the current fingerprint and the action group is already very low below 0.3, and further reduction would lose its significance as a criterion.

[0105] The dual-threshold decision unit decodes the action label and its corresponding maximum probability p. max Output to the graph evolution and parameter update module 400.

[0106] Meanwhile, the Bayesian path selection and online decoding module 300 converts the action label into control instructions that the smart prosthesis can execute. Specifically, each action label corresponds to a set of control parameters in advance, including the movement direction, movement speed, movement angle and duration of the prosthesis. The corresponding control parameters are obtained by looking up the table according to the output action label, and these parameters are encapsulated into control instruction frames and sent to the smart prosthesis through the communication interface to drive the prosthesis to perform the corresponding fine movements.

[0107] The graph evolution and parameter update module 400 receives the action label and corresponding maximum probability p output by the Bayesian path selection and online decoding module 300.max At the same time, it obtains feedback information from users during the usage process.

[0108] This module consists of an incremental registration unit, a transition probability update unit, a threshold adaptation unit, and a group structure review unit. It is used to continuously iterate and optimize the action fingerprint spectrum and decoding parameters to adapt to the dynamic changes of newly added actions and user EEG characteristics.

[0109] The incremental registration unit is used to complete the incremental registration of new fine actions: when a new action needs to be added, the unit first collects multiple sets of test samples for the action, extracts the Riemann trajectory fingerprint of each set of samples according to the signal processing flow described above, and then calculates the mean vector and scatter matrix corresponding to the fingerprint.

[0110] Based on this, the Euclidean distance between the mean vector of the new action fingerprint and the embedding center of each existing action group is calculated to determine the action's affiliation: if the calculated distance is less than the group radius of the corresponding action group, the new action is assigned to that group, and the embedding center, group radius, and fingerprint statistics within the group are updated simultaneously; if the calculated distance is greater than or equal to the corresponding group radius, a new action group is created separately, the embedding center of the new group is set to the mean vector of the new action fingerprint, and the group radius is initialized to zero.

[0111] Furthermore, the unit updates the action fingerprint map as a whole, recalculates the embedding vectors of the associated nodes and the weights of the inter-group edges, and sends the updated action map package back to the Bayesian path selection and online decoding module 300.

[0112] The transition probability update unit is used to maintain and iterate the action transition probability matrix. During system initialization, this unit first counts the action transition frequency between adjacent test segments when the user performs a fine-grained action task, thereby determining the initial transition probability matrix. Specifically, it collects test data of several sets of preset fine-grained action sequences performed by the user, records the pairing of the preceding and following actions in each pair of adjacent test segments, counts the occurrence frequency of each action pair, and then divides it by the total occurrence frequency of the preceding action to obtain the initial estimate of the transition probability, denoted as . .

[0113] This unit is configured with a length of L h The action history queue is used to record the sequence of actions recently completed by the user. During system use, whenever a decoding result is received from the dual-threshold decision unit and the result is confirmed by the user or corrected based on feedback, the corresponding action tag is stored in the history queue.

[0114] Based on this, after the queue has accumulated sufficient new data, the unit counts the transition frequency between adjacent actions and updates the matrix parameters using an exponentially weighted moving average method. The update formula is as follows: ; in, The transfer frequency is based on the latest queue statistics; The transition probability before the update is given, and the smoothing factor α is set to 0.3.

[0115] It should be noted that the smoothing factor α is set to 0.3 to strike a balance between rapidly adapting to changes in user habits and maintaining the stability of historical information. A value that is too large would be overly sensitive to short-term fluctuations and easily affected by random errors; a value that is too small would result in too slow an update speed, failing to reflect changes in user habits in a timely manner.

[0116] The iterative transition probability matrix can be tailored to user operating habits, improving the accuracy of prior estimation for probability update units.

[0117] The threshold adaptive unit is used to dynamically adjust the three types of threshold parameters within the dual-threshold decision unit. Specifically, after every 50 decoding outputs, the unit calculates the recent decoding error rate and the waiting mode trigger rate. The decoding error rate represents the proportion of output results with deviations, and the waiting mode trigger rate represents the proportion of test segments entering the waiting state. If the decoding error rate is higher than 0.1, it indicates that the current output threshold is too low and there are many misjudgments. The unit will then adjust the output threshold δ. out Increase by 0.02, and set the upper limit of the parameter to 0.85.

[0118] Furthermore, if the waiting mode trigger rate is higher than 0.15, it indicates that the waiting threshold is too high, and most test segments cannot output results normally. The unit will then lower the waiting threshold δ. wait The value was reduced by 0.02, and the lower limit of the parameter was set to 0.3.

[0119] When both types of indicators exceed the reasonable range simultaneously, the unit simultaneously performs threshold adjustment operations, triggering the group structure review unit to conduct detection. The adjusted threshold parameters are synchronized to the dual-threshold decision unit and applied to the subsequent decoding and judgment process.

[0120] The group structure review unit is used to verify the rationality of the internal clustering of each action group and to dynamically split and optimize failed groups. The unit can actively inspect according to a fixed period, or it can be passively triggered to detect when the threshold adaptive unit determines that the parameters are abnormal.

[0121] During the detection process, the group structure review unit traverses all action groups and calculates the Riemann distance between any two action fingerprints within a group. If the maximum Riemann distance within an action group exceeds twice the radius of the current group, it indicates that the dispersion of action features within that group is too high, and the original clustering structure can no longer accurately distinguish action differences.

[0122] Based on this, the group structure review unit re-performs community clustering operations on all action nodes within the abnormal group, and performs a secondary division of the original group according to feature similarity, ultimately generating at least two sub-action groups with more reasonable structures. After the grouping adjustment is completed, the unit updates the node embeddings, group centers, group radii, and mapping relationship tables between actions and groups in the action fingerprint map, completing the map structure iteration.

[0123] Furthermore, the group structure review unit sends the updated action graph package back to the Bayesian path selection and online decoding module 300, enabling the subsequent online decoding process to complete feature matching and probabilistic inference based on the latest clustering group structure, thus ensuring long-term decoding stability and adaptability.

[0124] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A brainwave signal analysis and decoding service platform, comprising a brainwave signal acquisition module (100), wherein the brainwave signal acquisition module (100) is used to acquire the raw brainwave signals of a user performing a fine motor imagery task, and extract Riemann trajectory fingerprint vectors from them, and calculate the fingerprint mean vector and scatter matrix of the fingerprint based on multiple Riemann trajectory fingerprint vectors of the same category, characterized in that, It also includes an action fingerprint map construction module (200) and a Bayesian path selection and online decoding module (300). The action fingerprint map construction module (200) takes each fine action as a node, calculates the Wasserstein distance between nodes based on the fingerprint mean vector and the scatter matrix, and constructs the adjacency matrix of the undirected graph. Based on the adjacency matrix, a graph convolutional network is used to learn the embedding of nodes to obtain node embedding vectors. By maximizing the modularity, the node embedding vectors are clustered into multiple action groups, and the embedding center and group radius of each action group are calculated. The Bayesian path selection and online decoding module (300) calculates the group probability of the current fingerprint belonging to each action group and the action probability of each fine action based on the Riemann trajectory fingerprint vector of the current test segment and the embedding center and group radius of each action group. Maintaining an initial uniformly distributed probability vector, Bayesian recursion is performed in the current test segment based on the probability vector before the update, the current action probability, and the pre-statistical action transition probability matrix to obtain the probability vector of the current test segment. Action labels are output according to the decision rules, converted into control commands, and sent to the intelligent prosthesis to drive the prosthesis to perform corresponding fine movements.

2. The EEG signal analysis and decoding service platform according to claim 1, characterized in that: The EEG signal acquisition module (100) divides the acquired raw EEG signal into time windows, calculates the covariance matrix within each time window, maps the covariance matrix sequence to a trajectory on a Riemann manifold, and extracts geometric features from the trajectory to splice them into the Riemann trajectory fingerprint vector. The geometric features include geodesic curvature sequence, total arc length of the trajectory, and torsion sequence.

3. The EEG signal analysis and decoding service platform according to claim 1, characterized in that: The EEG signal acquisition module (100) performs independent component extraction on the original EEG signal, decomposes it into multiple independent components, identifies the electrooculogram (EOG) artifact components and electromyogram (EMG) artifact components based on the time-domain waveform characteristics and frequency-domain energy distribution of each component, and reconstructs the signal after setting the corresponding rows of these components to zero, thus obtaining a pure EEG signal.

4. The EEG signal analysis and decoding service platform according to claim 1, characterized in that: The graph convolutional network in the action fingerprint map construction module (200) includes two graph convolutional layers. The first layer multiplies the normalized adjacency matrix with the node feature matrix and then multiplies it by the first layer weight matrix. The linear rectified activation function is applied to the product result to obtain the first layer node embedding matrix. The second layer multiplies the normalized adjacency matrix with the first-layer node embedding matrix, and then multiplies it by the second-layer weight matrix to obtain the second-layer node embedding matrix as the final node embedding vector.

5. The EEG signal analysis and decoding service platform according to claim 1, characterized in that: The action fingerprint graph construction module (200) sets a similarity threshold, converts the Wasserstein distance between nodes into similarity weights, and retains only edges with weights greater than the threshold for constructing the adjacency matrix of the undirected graph.

6. The EEG signal analysis and decoding service platform according to claim 1, characterized in that: When the action fingerprint map construction module (200) performs clustering by maximizing modularity, it treats each node as an independent community, moves the node to the adjacent community in turn and calculates the modularity after the move. If the modularity after the move is greater than before the move, the move is retained; otherwise, the move is canceled. The above process is repeated until the modularity no longer increases, thus completing the clustering of the node embedding vector.

7. The EEG signal analysis and decoding service platform according to claim 1, characterized in that: The decision rules for outputting action labels according to the decision rules include: If the maximum value of the current probability vector is greater than or equal to the output threshold and the group probability of the action corresponding to the maximum value is greater than or equal to the group probability threshold, then output the action label; If the maximum value of the current probability vector is between the waiting threshold and the output threshold, then the current probability vector is retained and the information fusion is waited for the next test segment. If the maximum value of the current probability vector is less than the waiting threshold, a resampling prompt is triggered.

8. The EEG signal analysis and decoding service platform according to claim 7, characterized in that: When multiple consecutive test segments are in a waiting state and the maximum probability does not increase, the output threshold will be lowered; when the group probability continues to fail to meet the judgment requirements, the group probability threshold will be lowered; the adjusted threshold will be applied to the decision-making process of subsequent test segments.

9. The EEG signal analysis and decoding service platform according to claim 1, characterized in that: It also includes a graph evolution and parameter update module (400), which is used to receive action tags. When a user needs to add an action, it collects multiple test segment fingerprints of the new action, calculates its mean vector, and compares it with the embedding center of the existing action group. If the distance is less than the group radius, it is merged into the group; otherwise, a new action group is created.

10. The EEG signal analysis and decoding service platform according to claim 1, characterized in that: The graph evolution and parameter update module (400) periodically checks the maximum distance between all action fingerprints in each action group. When the distance exceeds a preset threshold, the action group is split into multiple subgroups, and the embedding center and group radius are updated.

Images