Electroencephalogram intention recognition and robot control method based on contrastive learning

Abstract

This invention relates to the field of EEG information processing technology, and discloses a method for EEG intention recognition and robotic arm control based on contrastive learning, comprising the following steps: Step 1, acquiring multi-channel EEG signals from a subject and segmenting them into multiple EEG data segments; Step 2, constructing a training set; Step 3, constructing a pre-trained model, performing contrastive learning pre-training to obtain a pre-trained encoder; Step 4, extracting intermediate features from the intermediate layer of the pre-trained encoder and calculating the average energy feature, inputting it into a multilayer perceptron classifier for training, and outputting the motor imagery intention classification result; Step 5, acquiring online EEG signals to be tested in real time, performing real-time inference, decoding and outputting motor imagery intention labels; Step 6, generating control commands to control an external robotic arm to perform the target action. This invention can effectively improve the recognition accuracy and online inference stability of six-class classification of cross-time-segment motor imagery, thereby achieving efficient real-time control of the robotic arm in six directions.

Description

Technical Field

[0001] This invention relates to the field of EEG information processing technology, specifically to a method for EEG intention recognition and robotic arm control based on contrastive learning. Background Technology

[0002] Motor imagery (MI) recognition and closed-loop control based on electroencephalogram (EEG) signals are core directions in the field of brain-computer interfaces, but they face multiple technical bottlenecks in the practical application of streaming online control.

[0003] Early methods, such as Common Spatial Patterns (CSP) and the patent technology published in CN103793058B, primarily rely on supervised decomposition of multiple signal covariance matrices to find the optimal spatial projection direction to maximize the variance difference between the two signal classes. This type of method suffers from three inherent drawbacks: First, its performance is highly dependent on prior human selection of specific frequency bands, but the optimal frequency band dynamically changes with the subject, electrode position, and experimental period, making manual selection unsuitable for adaptive matching. Second, the algorithm is essentially based on binary classification; when expanded to three or more classes, multiple binary classification sub-tasks need to be constructed, and the spatial patterns of each sub-task interfere with each other, leading to degradation of the projection basis matrix and blurring of the feature space boundaries. Third, this method completely ignores the temporal evolution characteristics of the signal, utilizing only spatial variance information, and cannot capture the dynamic time-frequency patterns of event-related desynchronization / synchronization (ERD / ERS) during motion visualization. Subsequent improvements, such as the Chinese patent with publication number CN110765920B, introduced convolutional neural networks to automatically extract features to replace manual design. However, its supervised learning paradigm relies on fixed hard labels to drive parameter updates. Under conditions of non-stationary EEG signals and high noise levels, the network struggles to learn deep representations with sufficient inter-class discrimination from limited labeled samples. Furthermore, the four-class classification architecture cannot adapt to the instruction granularity required for six-directional control of the robotic arm. The insufficient number of classifications directly leads to a severe limitation on the degree of freedom of control.

[0004] At the system implementation level of brain-controlled robotic arms, robotic arm control schemes based on passive evoked paradigms such as P300 or SSVEP have also been developed concurrently, such as Chinese patents with publication numbers CN106671084B and CN108304068B, respectively. However, both of them rely heavily on high-frequency flashing external visual stimuli to induce specific potentials, which not only easily causes severe visual fatigue and occupies the subject's field of vision, but also makes it easy for stimulus artifacts to mix in and distort the intrinsic EEG characteristics, making it impossible to achieve continuous and smooth control that is completely driven by the brain.

[0005] Although recent studies have introduced contrastive learning into motor imagery EEG decoding to improve feature discriminativeness and employed multi-scale spatiotemporal convolution to capture multi-resolution time-frequency features, such as Chinese patents CN114997229A and CN117009780A, existing solutions have three clear shortcomings: First, the contrastive learning and feature extractor are simply connected in series, with each optimizing independently. This fails to form a deep closed-loop collaboration driven by gradient feedback, limiting feature alignment efficiency and leaving the improvement in discrimination ability far below its theoretical limit. Second, existing solutions primarily rely on binary or quadruple classification, while robotic arms require at least six independent motion directions (left / right / up / down / forward / backward) in three-dimensional space for flexible control. The coarse classification granularity directly restricts the freedom and accuracy of downstream control. Third, in the face of high-amplitude sudden artifacts such as blinking and electromyography during continuous online acquisition, as well as non-stationary distribution drift caused by long-term cross-period acquisition, there is currently a lack of a method that can combine adaptive threshold filtering and dynamic normalization processing logic to suppress sudden pulse interference. When the feature distribution of the signal is disturbed, the classifier will produce continuous misclassifications, seriously threatening the operational safety of the robotic arm. Summary of the Invention

[0006] The present invention aims to provide a method for EEG intention recognition and robotic arm control based on contrastive learning, which can effectively improve the recognition accuracy and online reasoning stability of six categories of cross-time motion imagery, thereby achieving efficient real-time control of the robotic arm in six directions.

[0007] The basic solution provided by this invention is: a method for EEG intention recognition and robotic arm control based on contrastive learning, comprising the following steps: Step 1: Acquire multi-channel EEG signals from the subject, and after preprocessing, divide them into multiple EEG data segments according to time windows; Step 2: Construct a training set by pairing EEG signal fragments of the same motor imagery category collected from the same subject at different time periods to construct positive sample pairs, and EEG signal fragments of different motor imagery categories to construct negative sample pairs. Step 3: Construct a pre-trained model, which includes a multi-scale spatiotemporal convolution module and an attention mechanism module. The multi-scale spatiotemporal convolution module extracts local temporal features and multi-scale spatiotemporal composite features of the EEG signal. The attention mechanism module calculates channel attention weights for the extracted features and performs element-wise weighting. With the goal of minimizing the InfoNCE loss function, comparative learning pre-training is performed based on the positive and negative sample pairs. The parameters of the multi-scale spatiotemporal convolution module and the attention mechanism module are iteratively updated to obtain the pre-trained encoder. Step 4: Extract intermediate features from the intermediate layer of the pre-trained encoder and calculate the average energy features. Input the average energy features into the multilayer perceptron classifier for training and output the motion imagination intention classification result. Step 5: Real-time acquisition of online EEG signals to be tested; after the pre-trained encoder extracts intermediate features and calculates average energy features, the signals are input into the trained multilayer perceptron classifier for real-time reasoning, and the decoded output is a motor imagery intention label. Step 6: Generate control commands based on the spatial motion mapping relationship corresponding to the motion imagination intention tag, and control the external robotic arm to perform the target action.

[0008] The working principle and advantages of this invention are as follows: This invention presents a contrastive learning-based EEG intention recognition and robotic arm control method that effectively improves the recognition accuracy and online inference stability of six-category classification of cross-temporal motor imagery, thereby achieving efficient real-time control of the robotic arm in six directions. The key points are: First, the closed-loop collaborative optimization architecture between feature extraction and contrastive learning significantly improves feature discrimination capabilities. Features extracted by the multi-scale spatiotemporal convolution module are weighted by the attention mechanism and then input into contrastive learning. The gradient feedback of contrastive learning is simultaneously backpropagated to both the attention mechanism and the multi-scale spatiotemporal convolution module, forming a complete closed loop of extraction-weighting-alignment-feedback-re-extraction. This allows the attention weights to be dynamically adjusted in each iteration under the guidance of contrastive loss, gradually strengthening core brain region features that are beneficial for inter-class differentiation and suppressing task-irrelevant responses. Its performance ceiling is significantly higher than that of a cascaded structure optimized independently. Furthermore, by connecting contrastive learning after the dynamic attention module, with clear input features, the spatial distance between samples of the same class and samples of different classes is more effectively reduced within the feature space, thereby achieving better generalization and discrimination.

[0009] Second, a cross-time period positive sample pair construction strategy is adopted, which can effectively address non-stationary drift. Existing contrastive learning methods typically apply data augmentation to the same original sample to construct positive sample pairs, which is essentially limited to data augmentation within the same time period and cannot cope with the distribution drift caused by cross-time period collection. This approach extends positive sample pairs from "same time period" to "different time periods"—pairing EEG segments of the same category from the same subject in different experimental rounds, forcing the model to learn stable class discrimination features across time periods, rather than just fitting the data distribution of a specific time period. This helps to specifically address the long-term non-stationary drift problem of EEG signals caused by factors such as changes in electrode impedance and fluctuations in subject fatigue.

[0010] Third, selecting average energy features as intermediate features allows for seamless integration with dynamic attention mechanisms. This scheme extracts intermediate features from multi-scale spatial convolutional layers and calculates their temporal mean. The output has a clear neurophysiological interpretation: positive values ​​indicate channel activation (ERD phenomenon), negative values ​​indicate inhibition (ERS phenomenon), and the amplitude reflects the activation intensity. This characteristic naturally matches the essential function of attention mechanisms—filtering and amplifying discriminative channel responses. The combination of these two approaches completes feature integration in a very short time, satisfying the hard constraint of low-latency inference in online control.

[0011] Fourth, this solution extends contrastive learning-driven EEG intention recognition to six categories and establishes a one-to-one correspondence mapping with the six-directional movements of the robotic arm in three-dimensional space. The average recognition accuracy reaches 56%, far exceeding the 16.67% random guessing level, enabling efficient real-time control of the robotic arm in six directions. The leap from four to six categories faces the challenge of blurred inter-class boundaries due to fine-grained feature space partitioning—the more categories there are, the more easily decision boundaries intrude, and the difficulty of discrimination increases non-linearly. In this solution, it is precisely because of the significantly improved feature distinguishability based on the closed-loop collaborative architecture that the six-class classification task achieves usable accuracy at the engineering level, making it possible for the robotic arm to achieve flexible control covering three degrees of freedom—yaw, pitch, and extension—in three-dimensional space. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the method flow of an embodiment of the EEG intention recognition and robotic arm control method based on contrastive learning of the present invention. Figure 2 This is a schematic diagram of the raw EEG data before preprocessing in an embodiment of the EEG intention recognition and robotic arm control method based on contrastive learning of the present invention. Figure 3 This is a schematic diagram of preprocessed EEG data from an embodiment of the EEG intention recognition and robotic arm control method based on contrastive learning of the present invention. Figure 4 This is a schematic diagram of the average energy feature extracted by a pre-trained encoder, representing an embodiment of the EEG intention recognition and robotic arm control method based on contrastive learning of the present invention. Figure 5 This is a schematic diagram of the model structure of an embodiment of the EEG intention recognition and robotic arm control method based on contrastive learning of the present invention. Figure 6 This is a confusion matrix diagram of the classification results in a classification test of six types of motor imagery (left hand, right hand, both hands, both feet, tongue, and resting) in an embodiment of the EEG intention recognition and robotic arm control method based on contrastive learning of the present invention. Detailed Implementation

[0013] The following detailed explanation illustrates the specific implementation methods: The basic implementation examples are as follows: Figure 1 As shown: A method for EEG intention recognition and robotic arm control based on contrastive learning includes the following steps: Step 1: Acquire multi-channel EEG signals from the subject, and after preprocessing, segment the EEG data into multiple segments according to time windows, such as... Figure 2 and Figure 3 As shown.

[0014] The multi-channel EEG signal has 63 channels and a sampling rate of 1000Hz. In this embodiment, during each experiment, the subject performs six types of motor imagery tasks according to screen prompts: left hand motor imagery, right hand motor imagery, tongue motor imagery, foot motor imagery, hand motor imagery, and resting state. Each type of task appears 4 times in each experiment, and each imagery task lasts for 4 seconds, with rest intervals between tasks. Each subject performs 20 runs of the experiment, with each run containing 24 trials (6 types of tasks × 4 repetitions), and a total of 480 trials of EEG data are collected from each subject. The data from each run is stored independently in a file, with the trials in the file arranged in the order of the 6 types and repeated 4 times.

[0015] The preprocessing includes: (1) Bandpass filtering and power frequency notch filtering.

[0016] The multi-channel EEG signals are preprocessed using bandpass filtering and power frequency notch filtering to remove power frequency interference and suppress low-frequency drift and high-frequency noise. The frequency range of the bandpass filtering is 0.5–40 Hz, and the frequency of the power frequency notch filtering is 50 Hz.

[0017] (2) Adaptive threshold removal of abnormal artifacts.

[0018] Electroencephalogram (EEG) signals often contain sudden, high-amplitude artifacts caused by blinking, electromyography (EMG), head movements, or poor electrode contact. To avoid these extreme outliers contaminating the statistics required for normalization, an adaptive thresholding strategy is used to remove outliers before data slicing. Specifically: Calculate the median of the absolute values ​​of EEG data for the current time period, and determine 30 times the median as the amplitude threshold. Data with absolute values ​​less than the amplitude threshold are retained as valid data.

[0019] It should be noted that normal EEG amplitudes typically fall within the range of ±50-100 μV, while amplitudes caused by poor electrode contact or EMG artifacts often exceed 1000 μV. Setting a threshold of 30 times the median is a lenient exclusion criterion—when the median is 50 μV, the threshold is approximately 1500 μV, eliminating only the most extreme outliers. This threshold setting aims to prevent extreme outliers from affecting the distribution of normal data through subsequent normalization operations, rather than eliminating data within the normal amplitude range. In practice, this threshold is usually not triggered and only comes into play when severe signal abnormalities are observed.

[0020] (3) Hierarchical normalization.

[0021] The overall amplitude level of the EEG signal may drift between different experimental rounds due to factors such as changes in electrode impedance and differences in fatigue levels. To eliminate the differences in data distribution between different time periods, this embodiment adopts a stratified normalization strategy grouped by time period.

[0022] The data from each round are grouped together, and the mean is calculated using the retained valid data. and standard deviation According to the formula Normalize the data for the current time period, where, To correct for small constants.

[0023] This hierarchical normalization unifies the data amplitude scale of different rounds to a similar order of magnitude, while avoiding the phenomenon that extreme outliers in one round contaminate the data of other rounds.

[0024] Furthermore, to increase the training sample size and facilitate the capture of temporal dynamic features, the preprocessed EEG data is segmented without overlap according to a fixed time window. The segmentation rule varies between the training and inference phases: the length of the time window is 1 second in the contrastive learning pre-training phase and 0.5 seconds in the multilayer perceptron classifier classification phase.

[0025] Step 2: Construct a training set by pairing EEG signal fragments of the same motor imagery category collected from the same subject at different time periods to construct positive sample pairs, and EEG signal fragments of different motor imagery categories to construct negative sample pairs.

[0026] The preprocessed samples were grouped according to the collection time period (run). A 10-fold cross-validation strategy was adopted: the data from the 20 runs were divided into 10 groups according to the chronological order of the collection time period, with each group containing data from 2 runs. One group was selected as the validation set in turn, and the remaining 9 groups were used as the training set to complete 10 rounds of cross-validation. This method of dividing and grouping by time period can effectively verify the model's generalization ability in cross-time period scenarios.

[0027] When constructing the training set, a contrastive learning-based sample pairing strategy is employed. The specific rules are as follows: For the same subject, EEG signal segments of the same motor imagery category are paired between different runs to construct positive sample pairs. For example, assuming that both the first and second runs contain EEG segments of left-hand motor imagery, these segments are paired to form positive sample pairs. Positive sample pairs are constructed between any two different runs according to this rule.

[0028] EEG signal segments from different motor imagery categories were constructed as negative sample pairs. Specifically, negative sample pairs were formed between samples of different categories within the same run, and between samples of different categories across different runs.

[0029] Each sample has a label space of 6 classes: Class 0 (left hand), Class 1 (right hand), Class 2 (tongue), Class 3 (feet), Class 4 (both hands), and Class 5 (resting).

[0030] Step 3: Build a pre-trained model.

[0031] like Figure 5 As shown, the pre-trained model is composed of a multi-scale spatiotemporal convolution module, an attention mechanism module, and a projection head module cascaded together.

[0032] The multi-scale spatiotemporal convolution module uses temporal convolutional layers and multiple sets of spatial convolutional layers with different dilation coefficients to extract local temporal features and multi-scale spatiotemporal composite features of EEG signals in parallel.

[0033] Specifically, in this embodiment, the temporal convolutional layer uses 16 one-dimensional convolutional kernels of size 1×30 to extract local temporal features along the time axis.

[0034] The specific steps of temporal convolution processing are as follows: Obtain the input feature matrix ,in, This indicates the batch size, and 1 indicates the initial number of input channels (in this embodiment, the default is single-channel input). Indicates the total number of electrode channels. Indicates the time step; the input feature matrix The data is fed into a temporal convolution kernel for one-dimensional temporal convolution, and the output is a temporal feature matrix. ,in, This indicates the set number of convolutions over time.

[0035] The formula for calculating the temporal convolution operation is: ; in, The weight tensor representing the temporal convolution kernel; Indicates the window length for temporal convolution; This represents the padding length, which is used to maintain consistency across time. It is calculated as follows: ; The tensor representing the bias term of the temporal convolution.

[0036] The output of the temporal convolutional layer The spatial convolutional layer consists of four groups of dilated convolutional layers with dilation coefficients of 1, 3, 6, and 12, each containing 16 spatial convolutional kernels. The results of the four convolutional layers are concatenated along the channel dimension to output multi-scale spatiotemporal composite features. The number of convolutional groups is set to be equal to the number of temporal convolutions (i.e., 16 groups), so that each group of spatial convolutions can independently and isolatedly process the corresponding temporal channel data.

[0037] For the i-th parallel branch (i=1,2,3,4), its corresponding expansion rate is The mathematical analytical expression for this branch is: ; in, This represents the weight tensor of the i-th spatial convolution kernel. This indicates the coverage length of the spatial convolution on the time axis ( ), This represents the expansion coefficient of each parallel branch. Indicates the fill length of the i-th branch (corresponding to respectively) (The values ​​are 1, 3, 6, and 12), their function is to maintain consistency in the time dimension, and their calculation method satisfies: , Let represent the bias term tensor of the i-th spatial convolution.

[0038] After parallel computation of four sets of spatial convolutions, the outputs of each set are concatenated and stitched together along the channel dimension to finally output a multi-scale spatial feature matrix. ( That is, 4 groups × 16 spatial convolution kernels per group × 16 temporal channels per group / number of groups 16).

[0039] It should be noted that four sets of spatial convolutional kernels with different dilation coefficients (dilation coefficients 1, 3, 6, and 12) were designed to cover EEG features at different time scales: a smaller dilation coefficient (e.g., 1) results in a smaller receptive field, suitable for capturing high-frequency, transient, fast-changing rhythmic features (e.g., β rhythm, 13-30Hz); a larger dilation coefficient (e.g., 12) results in a larger receptive field, suitable for capturing slow-changing rhythmic features within a longer time window (e.g., μ rhythm, 8-12Hz). This multi-scale design ensures that features of different frequency components are fully extracted, providing a rich feature base for subsequent attention mechanisms.

[0040] Furthermore, the attention mechanism module processes the multi-scale spatial feature matrix. Calculate the channel attention weights and perform element-wise weighting, including the following sub-steps: First, a depthwise separable temporal convolution is used to extract local temporal features along the time axis. Specifically, a 1×15 depthwise separable convolution kernel is used to filter the input features, and mean pooling is used to compress the time dimension, aggregating the temporal information of each channel into a scalar value (the pooling window size is 1×15).

[0041] Secondly, the attention values ​​of each channel are calculated by pointwise convolution (1×1 convolution), and the channel attention weights are obtained by normalization using the Softmax function. The weight vector has a dimension of 256.

[0042] Finally, the channel attention weights are combined with the output features of the multi-scale spatiotemporal convolution module. Perform element-wise dot product and output attention-weighted features. Through this attention mechanism, the core brain regions associated with the current motor imagery task are amplified and enhanced, while background and environmental noise are suppressed.

[0043] Furthermore, the projection head module is used to map the attention-weighted features to the embedding space required for contrastive learning, i.e., to perform contrastive learning pre-training. The processing flow is as follows: Attention-weighted features Temporal average pooling (pooling window size set to 15 sampling points, stride of 15, compressing the time dimension to...) Each time step has an output dimension of [number] dimensions. The input is then fed into two temporally smoothed convolutional layers.

[0044] The temporally smoothed convolutional layer employs a 2x channel expansion factor for inter-layer smoothing evolution (the first layer expands the number of channels from 256 to 512, and the second layer expands the number of channels from 512 to 1024, both with an expansion factor of 2). During forward propagation, hierarchical normalization constraints are embedded to flatten the output feature map, resulting in a 1024×1024 convolutional layer. The basic feature vectors of the dimensional model are obtained by performing L2 normalization mapping, resulting in the basic feature vectors for contrastive learning.

[0045] Contrastive learning aims to minimize the InfoNCE loss function and is pre-trained based on the positive and negative sample pairs. It then iteratively trains using the Adam optimizer and cosine annealing scheduler, updating the parameters of the multi-scale spatiotemporal convolutional module and the attention mechanism module to minimize the InfoNCE loss function, thus obtaining the pre-trained encoder.

[0046] Specifically, for a sample pair (i,j), the expression for its InfoNCE loss function is: ; in, For sample pairs and The comparative loss value; and A positive sample pair (two enhanced views from the same original sample); This refers to all samples in the current batch except for the positive samples; This is the function for calculating cosine similarity. The temperature parameter (set to 0.07 in this embodiment) is used to control the sensitivity of contrastive learning in distinguishing negative samples; This represents the total number of samples in the current batch.

[0047] The formula for calculating the cosine similarity function is: ; The Adam optimizer was used for parameter optimization, with an initial learning rate of 0.0007 and a weight decay coefficient of 0.00015. A cosine annealing learning rate scheduler was used for 30 epochs of iterative training, aiming to minimize the InfoNCE loss function and learn discriminative features that distinguish six types of motion intentions. After training, the projector head was discarded, and the multi-scale spatiotemporal convolutional module and attention mechanism module were retained as pre-trained encoders.

[0048] In this embodiment, the multi-scale spatiotemporal convolution module, the attention mechanism module, and the contrastive learning pre-training form a closed-loop collaborative optimization architecture. The collaborative mechanism is as follows: after the multi-scale spatiotemporal convolution module extracts multi-scale spatiotemporal features, the attention mechanism module weights and outputs the features to the contrastive learning pre-training. The gradient backpropagation of the contrastive learning pre-training simultaneously updates the parameters of both the multi-scale spatiotemporal convolution module and the attention mechanism module. The updated parameters then influence the feature extraction of the multi-scale spatiotemporal convolution module and the weight allocation of the attention mechanism module in the next iteration.

[0049] These three elements are interconnected: the rich features of multi-scale spatiotemporal convolution provide an optional foundation for the attention mechanism; the attention mechanism provides clean feature input for contrastive learning; and the gradient feedback of contrastive learning forcibly corrects the feature extraction direction of multi-scale spatiotemporal convolution and the parameter selection of the attention mechanism. Removing any one of these modules will significantly degrade the decoding performance.

[0050] Step 4: Extract intermediate features from the intermediate layers of the pre-trained encoder and calculate the average energy features, such as... Figure 4 As shown, the average energy features are input into a multilayer perceptron classifier for training, and the output is a classification result of the motion imagination intention.

[0051] Specifically, the multi-scale spatial feature matrix is ​​selected after the multi-scale spatial convolutional layer and before the attention mechanism module. This feature serves as an intermediate feature. It acquires the ability to cluster similar EEG patterns through contrastive learning, while preserving the multi-scale spatiotemporal features of the original EEG to the greatest extent possible.

[0052] The reason for choosing this layer instead of the final layer (projection head layer) as the source of feature extraction is: (1) In the field of self-supervised EEG representation learning, a large number of experiments have shown that the accuracy of downstream classification using intermediate layer features of the basic encoder is higher than that of directly using the final layer features of contrastive learning; (2) The multi-scale spatial convolution in this embodiment essentially performs multi-resolution frequency domain and time domain filtering on the μ rhythm (8-12Hz) and β rhythm (13-30Hz) of EEG. The intermediate features still retain the mapping relationship with the original physiological signal, while the features output by the projection head layer will lose this mapping relationship after nonlinear compression.

[0053] After extracting intermediate features from the intermediate layers of the pre-trained encoder and before calculating the average energy features, a normalization step is also included. The purpose of this step is to dynamically normalize the intermediate features extracted by the pre-trained encoder, enabling the model to have stronger noise resistance and adaptability to new data distributions when predicting new samples. Specifically, this includes: Using the global mean of the training data and global variance As a reference benchmark, an exponential decay factor is used. The data statistics are updated in real time, and the intermediate features are dynamically normalized. The calculation formula is as follows: ; in, For the first Original intermediate feature data at each time point; The feature data is normalized after dynamic operation; The preset exponential decay factor is, and =0.990; Let t be the decay weight at step t; This serves as a global mean reference benchmark for offline training data. This serves as a global variance reference benchmark for offline training data. This represents the average value at time point t. Up to the current time point Historical feature data summation; This is the sum of squared historical feature data up to the current time point t; To prevent negative values, the function is truncated; This is a small correction constant used to avoid the denominator being zero.

[0054] The exponential decay factor in the normalization of the operation The selection of α balances stability and adaptability: a larger α (e.g., 0.999) results in stronger model stability and better noise resistance, but a slower adaptation speed to new data distributions; a smaller α (e.g., 0.9) allows the model to adapt to new data distributions more quickly, but it becomes more sensitive to noise. 0.990 is the equilibrium value taken under the experimental conditions of this embodiment. 0.9-0.999 is a common range of values ​​in the field of dynamic normalization.

[0055] Furthermore, after extracting features from the intermediate layers of the pre-trained encoder, two feature transformation modes are provided: First, the arithmetic mean of the intermediate features over time is calculated to obtain the average energy feature. Specifically, for each sample and each feature channel, the mean is calculated over all time steps.

[0056] Second, calculate the variance of intermediate features over the time dimension. The differential entropy (DE) feature is then converted based on the following formula to amplify the discriminative power of nonlinear weak spontaneous EEG rhythms during event-related desynchronization (ERD). The formula for calculating the differential entropy feature is as follows: ; in, It is a characteristic of differential entropy. The variance of the intermediate feature over time. Pi It is a natural constant.

[0057] The average energy feature or differential entropy feature is input into the multilayer perceptron (MLP) classifier, and the classifier outputs the probability distribution of six types of motor imagination action intentions.

[0058] In this embodiment, the average energy feature is selected, and the differential entropy feature is only used as a backup feature option and is not used in the actual online inference of this embodiment. The advantages of using the average energy feature instead of the original feature are: (1) A positive value of the mean value indicates that the feature channel is in an active state in the whole time period, and a negative value indicates an inhibited state. The magnitude of the value reflects the activation intensity, which is consistent with the physiological meaning of ERD / ERS (Event-Related Desynchronization / Synchronization) in neuroscience; (2) It can integrate the extracted features in a very short time to meet the low latency requirements of real-time control of the robotic arm.

[0059] Specifically, the multilayer perceptron classifier consists of two hidden layers. The first layer maps the 256-dimensional input to 128 dimensions, and the second layer maps the 128-dimensional input to 64 dimensions. Each layer is followed by a ReLU activation function for non-linear mapping. A Dropout layer is added between the two fully connected layers, with a dropout coefficient set to 0.1, to reduce feature redundancy and prevent overfitting. Finally, the probability distribution of six types of motion intentions is output through a Softmax function.

[0060] Step 5: Real-time acquisition of online EEG signals to be tested; extraction of intermediate features and calculation of average energy features by the pre-trained encoder; input of the pre-trained multilayer perceptron classifier for real-time reasoning; decoding and output of motor imagery intention labels.

[0061] Specifically, subjects wore electrode caps to acquire 63 channels of EEG signals in real time. The real-time acquired signals underwent the same preprocessing as in the offline training phase (same as step 1).

[0062] The preprocessed EEG data segments are input into a pre-trained encoder to extract multi-scale spatial features from the intermediate layer. Calculate the average energy characteristics (mean over time).

[0063] After normalization and smoothing, the average energy feature vector is input into a trained multilayer perceptron classifier for fast inference. The softmax layer outputs the probability distribution of six types of motion intentions, and the class with the highest probability is taken as the predicted label for the current time window. .

[0064] A complete online classification control pipeline can be represented as: ; in, Represents raw brain electrical signals. Indicates preprocessing operation. This represents multi-scale spatiotemporal convolution and attention feature extraction. This represents the calculation of average energy characteristic transformation. This indicates that dynamic normalization and smoothing operations are performed on the feature vectors. This represents the classifier output logits score (raw predicted score) divided by Softmax normalization. Predictive labels indicating the final intention of the imagined movement.

[0065] Step 6: Generate control commands based on the spatial motion mapping relationship corresponding to the motion imagination intention tag, and control the external robotic arm to perform the target action.

[0066] The classification result of the motor imagery intention is six categories of motor imagery intention labels, which include left hand, right hand, tongue, both feet, both hands, and rest, respectively mapped to the six spatial directions of left, right, up, down, forward, and backward movement commands executed by the external robotic arm.

[0067] Specifically, in this embodiment, the predicted label The following mapping relationship is established with the six-directional movement of the robotic arm: when When (left hand movement intention) is shown, it is mapped to the "left" direction: the industrial control computer sends a left yaw command CMD_YAW_LEFT, which drives the rotary motor of the robot arm base to reverse, and controls the robot arm to yaw horizontally to the left by a set angle (e.g., 15 degrees). when When (right hand movement intention) is shown, it is mapped to the "right" direction: the industrial control computer issues the right yaw command CMD_YAW_RIGHT, drives the robotic arm base rotary motor to rotate forward, and controls the robotic arm to rotate horizontally to the right by a set angle (e.g., 15 degrees). when When the tongue is moving, it is mapped to the "up" direction: the industrial control computer sends a lifting command CMD_PITCH_UP, which drives the servo motor of the boom to lift upward, and controls the end of the robotic arm to move upward in the vertical Z-axis direction to lift the set height (e.g., 5cm). when When (two feet move in the intention) it is mapped to the "down" direction: the industrial control computer sends a downward pressure command CMD_PITCH_DOWN to drive the arm servo motor to tilt downward, and controls the end of the robotic arm to move downward in the vertical Z-axis direction to reduce the set height (e.g., 5cm). when When (the intention of the two hands) is shown, it is mapped to the "forward" direction: the industrial control computer issues the extension command CMD_ARM_EXTEND, which drives the servo motors of the upper arm and lower arm to move forward in coordination, and controls the end of the robotic arm to extend forward horizontally by a set distance (e.g., 5cm) in the horizontal X-axis direction. when (In the resting state) it is mapped to the "back" direction: the industrial control computer sends the retraction command CMD_ARM_RETRACT to drive the multi-joint servo motors of the upper arm and lower arm to move in opposite directions, and control the end of the robotic arm to retract horizontally a set distance (e.g., 5cm) in the horizontal X-axis direction.

[0068] The above motion is a discrete stepping motion: after each recognition result is output, the robotic arm performs a fixed displacement or rotation angle. The control frequency is consistent with the time window sliding frequency (outputting a control command every 0.5 seconds).

[0069] The industrial control computer generates control commands and sends them to the motion controller of the external robotic arm via a serial port or TCP / IP communication link. The communication protocol uses a custom command frame format, which includes fields such as command type, motion direction, motion step size, and checksum. After parsing the commands, the robotic arm controller drives the servo motors of the corresponding joints to execute the target action. At the same time, the current pose information of the robotic arm can be fed back to the industrial control computer through sensors, forming a closed-loop control.

[0070] In addition, to verify the effectiveness of the method in this embodiment, experiments were conducted on a self-collected motor imagery EEG dataset.

[0071] The dataset contains complete data from one subject (20 runs, 24 trials per run, for a total of 480 trials), 63 valid electrodes (excluding the reference electrode), and a sampling rate of 1000 Hz.

[0072] A 10-fold cross-validation method was used: the 20 runs were divided into 10 groups (2 runs per group) according to the collection time period. One group was selected alternately as the test set (to verify the generalization ability across time periods), and the remaining 9 groups were used as the training set. This was repeated 10 times, and the average accuracy was taken as the final performance indicator.

[0073] EEGNet was selected as the baseline model for comparison and validation. EEGNet is the authoritative baseline network in the field of motor imagery EEG classification. It was trained and tested using the same input data and the same 10-fold cross-validation method. The network structure of EEGNet remained unchanged from its original design. The training parameters were consistent with those in this embodiment. Detailed classification performance for each category is shown in Table 2.

[0074] The confusion matrix of the method in this embodiment under 10-fold cross-validation is as follows: Figure 6 As shown, the average classification accuracy of the six categories of motor imagination intentions reached 56%, which is far higher than the level of random guessing (the accuracy of random six-class classification is 1 / 6 ≈ 16.67%). Detailed classification performance for each category is shown in Table 1.

[0075] Table 1. Classification performance of various motor imagination intention states (method in this embodiment)

[0076] Table 2 Classification performance of EEGNet for various motion imagination intention states

[0077] Under 10-fold cross-validation, EEGNet achieved an average classification accuracy of 42.66% ± 4.38% and an average macro F1 score of 40.90% ± 4.27%. The method presented in this embodiment significantly outperforms the EEGNet baseline model across all metrics (average accuracy 56% vs 42.66%, average macro F1 54.76% vs 40.90%), demonstrating the effectiveness of the dynamic attention-guided multi-scale spatiotemporal feature extraction and contrastive learning co-optimization method employed in this embodiment.

[0078] This embodiment provides a contrastive learning-based EEG intention recognition and robotic arm control method that can effectively improve the recognition accuracy and online reasoning stability of six categories of cross-time periodic motor imagery, thereby achieving efficient real-time control of the robotic arm in six directions.

[0079] Furthermore, this solution achieves high-precision six-class classification without introducing additional large-scale computing modules. The average energy feature calculation is merely an arithmetic mean operation in the time dimension, resulting in extremely low computational overhead. The multilayer perceptron classifier contains only two hidden layers, with a parameter count far smaller than that of an end-to-end fully connected network. The entire model can be deployed on portable edge computing devices without relying on high-performance servers, and the inference latency meets the engineering requirements of millisecond-level response for real-time robotic arm control. Simultaneously, preprocessing relies only on standard tools, and the network structure is built based on conventional convolutional layers, without involving special hardware acceleration operators. It can be seamlessly integrated into existing brain-computer interface software stacks, exhibiting good engineering compatibility and scalability.

[0080] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics of the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.

Claims

1. A method for electroencephalogram intention recognition and robot control based on contrastive learning, characterized in that, Includes the following steps: Step 1: Acquire multi-channel EEG signals from the subject, and after preprocessing, divide them into multiple EEG data segments according to time windows; Step 2: Construct a training set by pairing EEG signal fragments of the same motor imagery category collected from the same subject at different time periods to construct positive sample pairs, and EEG signal fragments of different motor imagery categories to construct negative sample pairs. Step 3: Construct a pre-trained model, which includes a multi-scale spatiotemporal convolution module and an attention mechanism module. The multi-scale spatiotemporal convolution module extracts local temporal features and multi-scale spatiotemporal composite features of the EEG signal. The attention mechanism module calculates channel attention weights for the extracted features and performs element-wise weighting. With the goal of minimizing the InfoNCE loss function, comparative learning pre-training is performed based on the positive and negative sample pairs. The parameters of the multi-scale spatiotemporal convolution module and the attention mechanism module are iteratively updated to obtain the pre-trained encoder. Step 4: Extract intermediate features from the intermediate layer of the pre-trained encoder and calculate the average energy features. Input the average energy features into the multilayer perceptron classifier for training and output the motion imagination intention classification result. Step 5: Real-time acquisition of online EEG signals to be tested; after the pre-trained encoder extracts intermediate features and calculates average energy features, the signals are input into the trained multilayer perceptron classifier for real-time reasoning, and the decoded output is a motor imagery intention label. Step 6: Generate control commands based on the spatial motion mapping relationship corresponding to the motion imagination intention tag, and control the external robotic arm to perform the target action.

2. The EEG intention recognition and robotic arm control method based on contrastive learning according to claim 1, characterized in that, The multi-scale spatiotemporal convolution module, the attention mechanism module, and the contrastive learning pre-training form a closed-loop collaborative optimization architecture. After the multi-scale spatiotemporal convolution module extracts multi-scale spatiotemporal features, the weighted output is fed into the contrastive learning pre-training module via the attention mechanism module. The gradient backpropagation of the contrastive learning pre-training module updates the parameters of both the multi-scale spatiotemporal convolution module and the attention mechanism module. The updated parameters then affect the feature extraction of the multi-scale spatiotemporal convolution module and the weight allocation of the attention mechanism module in the next iteration.

3. The EEG intention recognition and robotic arm control method based on contrastive learning according to claim 1, characterized in that, The multi-channel EEG signal has 63 channels and a sampling rate of 1000Hz; the length of the time window is 1 second in the contrastive learning pre-training stage and 0.5 seconds in the multilayer perceptron classifier classification stage.

4. The EEG intention recognition and robotic arm control method based on contrastive learning according to claim 1, characterized in that, The multi-scale spatiotemporal convolution module uses temporal convolutional layers and multiple sets of spatial convolutional layers with different dilation coefficients to extract local temporal features and multi-scale spatiotemporal composite features of EEG signals in parallel.

5. The EEG intention recognition and robotic arm control method based on contrastive learning according to claim 4, characterized in that, The temporal convolutional layer uses 16 one-dimensional convolutional kernels of size 1×30 to extract local temporal features along the time axis; The spatial convolutional layer consists of four groups of dilated convolutional layers with dilation coefficients of 1, 3, 6 and 12, each containing 16 spatial convolutional kernels. The four convolutional results are spliced ​​along the channel dimension to output multi-scale spatiotemporal composite features.

6. The EEG intention recognition and robotic arm control method based on contrastive learning according to claim 1, characterized in that, The attention mechanism module calculates channel attention weights for the extracted features and performs element-wise weighting, including the following sub-steps: Local temporal features are extracted along the time axis using depthwise separable temporal convolution. After compressing the time dimension by mean pooling, the attention values ​​of each channel are calculated by pointwise convolution. The channel attention weights are obtained by normalization using the Softmax function. The channel attention weights are then multiplied elementwise with the output features of the multi-scale spatiotemporal convolution module to output attention-weighted features.

7. The EEG intention recognition and robotic arm control method based on contrastive learning according to claim 1, characterized in that, In the contrastive learning pre-training, the attention-weighted features are input into two temporally smoothed convolutional layers after average pooling. The temporally smoothed convolutional layers use a 2x channel expansion factor for inter-layer smoothing evolution and embed hierarchical normalization constraints during forward propagation. The output feature map is flattened and L2 normalization mapping is performed to obtain the basic feature vector. The basic feature vectors are input into the projection head, and the network parameters are updated iteratively under the configuration of Adam optimizer and cosine annealing scheduler, with the goal of minimizing the InfoNCE loss function.

8. The EEG intention recognition and robotic arm control method based on contrastive learning according to claim 1, characterized in that, The preprocessing includes: performing bandpass filtering and power frequency notch preprocessing on the multi-channel EEG signal; the frequency range of the bandpass filtering is 0.5–40Hz, and the frequency of the power frequency notch is 50Hz; Calculate the median of the absolute values ​​of EEG data for the current time period, and determine 30 times the median as the amplitude threshold. Data with absolute values ​​less than the amplitude threshold are retained as valid data. Calculate the mean using the retained valid data and standard deviation According to the formula Normalize the data for the current time period, where, To correct for small constants.

9. The EEG intention recognition and robotic arm control method based on contrastive learning according to claim 1, characterized in that, After extracting intermediate features from the intermediate layers of the pre-trained encoder and before calculating the average energy features, a normalization process is also included: Using the global mean of the training data and global variance As a reference benchmark, an exponential decay factor is used. The data statistics are updated in real time, and the intermediate features are dynamically normalized.

10. The EEG intention recognition and robotic arm control method based on contrastive learning according to claim 1, characterized in that, The classification result of the motor imagery intention is six categories of motor imagery intention labels, which include left hand, right hand, tongue, both feet, both hands, and rest, respectively mapped to the six spatial directions of left, right, up, down, forward, and backward movement commands executed by the external robotic arm.

Images