Robot control method and control system for a robot

Abstract

The application provides a robot control method and a robot control system, wherein the method comprises: collecting brain electrical signals of a user in real time; determining a motion intention keyword corresponding to the brain electrical signals according to the brain electrical signals and a pre-trained motion intention decoding model; determining a reference trajectory sequence of the robot according to the motion intention keyword and a pre-trained motion conversion model; and controlling the robot motion according to the reference trajectory sequence. The application can decouple signal quality and motion generation, avoid directly regressing brain electrical signals into spatial trajectories, and thus make the robot motion process no longer limited to the signal quality of brain electrical signals, significantly improve the stability of robot motion, ensure the physical feasibility of generated trajectories, greatly improve the practicability and reliability of the control process, reduce real-time computing load, and improve computing efficiency and scalability.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and more specifically, to a robot control method and a robot control system. Background Technology

[0002] Non-invasive brain-computer interfaces (BCIs) enable human-computer interaction by acquiring brain activity signals (EEGs) and have been widely used in fields such as medical rehabilitation and assistive device control. In recent years, with the rapid development of humanoid robot technology, how to combine BCIs with high-degree-of-freedom humanoid robots has become a research hotspot.

[0003] Existing technologies for non-invasive BCI-based robot control primarily employ direct decoding methods. Specifically, motion parameters (such as hand trajectories or joint angles) are extracted directly from EEG signals, and control commands are generated through regression models to achieve robot control.

[0004] However, this approach is limited by the signal quality of the EEG signal, which is susceptible to noise interference, resulting in significant errors in continuous motion trajectories and unstable robot movements. Furthermore, it can only perform simple actions (such as robotic arm grasping). In addition, the generated commands may violate robot dynamics constraints (such as joint torque limits), leading to hardware damage or loss of control, and a lack of physical adaptability. Simultaneously, users cannot adjust their intentions based on robot feedback, resulting in a low success rate and a missing interactive loop. Summary of the Invention

[0005] The purpose of this application is to address the shortcomings of the prior art by providing a robot control method and a robot control system, thereby solving the problems of lack of physical adaptability, unstable movements, and missing interactive loops in the prior art.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, one embodiment of this application provides a robot control method, the method comprising:

[0008] Real-time acquisition of users' brainwave signals;

[0009] Based on the EEG signal and the pre-trained motion intention decoding model, the motion intention keyword corresponding to the EEG signal is determined, and the motion intention keyword is used to indicate the type of robot movement;

[0010] Based on the motion intent keywords and the pre-trained motion conversion model, the reference trajectory sequence of the robot is determined;

[0011] The robot's movement is controlled based on the reference trajectory sequence.

[0012] Secondly, another embodiment of this application provides a robot control system, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the robot control system is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of any of the methods described in the first aspect above.

[0013] The beneficial effects of this application are as follows: By collecting the user's EEG signals in real time, determining the corresponding motion intention keywords based on the EEG signals and a pre-trained motion intention decoding model, and determining the robot's reference trajectory sequence based on the motion intention keywords and a pre-trained motion conversion model, the robot's movement can be controlled according to the reference trajectory sequence. This enables precise control of the robot's movement directly through EEG signals, establishing an efficient mapping between EEG signals and robot actions, and providing a new paradigm for human-robot collaboration. Simultaneously, generating motion intention keywords from EEG signals and then generating the robot's reference trajectory sequence based on these keywords, thereby controlling the robot's movement through the reference trajectory sequence, decouples signal quality from motion generation. This avoids directly reducing EEG signals to spatial trajectories, ensuring that the robot's movement is no longer limited by the signal quality of the EEG signals. This significantly improves the stability of the robot's movements and ensures the physical feasibility of the generated trajectory.

[0014] Furthermore, by converting motion intent keywords into reference trajectory sequences using a pre-trained motion transformation model, the naturalness and fluency of the movements are ensured while satisfying actual physical constraints, significantly improving the practicality and reliability of the control process. In addition, users can adjust their intentions in real time during robot movement; this interactive feature makes the entire control process more natural and flexible, greatly enhancing the robustness and adaptability of human-robot collaboration. Through this brain-computer interface, end-to-end intelligent control from neural signals to complex robot movements has been successfully achieved. Moreover, by using motion intent keywords as an intermediate representation, the combination and innovation of robot movements are supported, allowing users to trigger complex actions with simple keywords without needing to train a separate model for each action, thus improving the diversity and generalization of robot movements. Furthermore, the determination of motion intent keywords and reference trajectory sequences can be optimized independently, reducing real-time computational load and improving computational efficiency and scalability. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A schematic flowchart of a robot control method provided in an embodiment of this application;

[0017] Figure 2 This is a flowchart illustrating the process of determining the motion intention keyword corresponding to the electroencephalogram signal in the robot control method provided in this application embodiment;

[0018] Figure 3 This is a schematic flowchart illustrating the generation of EEG features corresponding to EEG signals in the robot control method provided in this application embodiment.

[0019] Figure 4 A schematic diagram of a motion intent decoding model in the robot control method provided in this application embodiment;

[0020] Figure 5 This is a flowchart illustrating the process of determining the motion intention keyword corresponding to the electroencephalogram signal in the robot control method provided in this application embodiment;

[0021] Figure 6 A schematic diagram of a frequency segmentation and conversion module in the motion intent decoding model of the robot control method provided in this application embodiment;

[0022] Figure 7 A flowchart illustrating the generation of a multi-band feature sequence in the robot control method provided in this application embodiment;

[0023] Figure 8 A schematic diagram of a motion intent encoding module in the motion intent decoding model of the robot control method provided in this application embodiment;

[0024] Figure 9 A schematic flowchart illustrating the generation of enhanced features in the robot control method provided in this application embodiment;

[0025] Figure 10 This is a schematic diagram of the decoding output module in the motion intent decoding model of the robot control method provided in the embodiments of this application;

[0026] Figure 11 This is a flowchart illustrating the process of generating motion intent keywords corresponding to electroencephalogram signals in the robot control method provided in this application embodiment;

[0027] Figure 12 A schematic diagram of a motion conversion model in the robot control method provided in this application embodiment;

[0028] Figure 13 This is a flowchart illustrating the process of determining a reference trajectory sequence for a robot in the robot control method provided in this application embodiment.

[0029] Figure 14 A schematic flowchart illustrating the training process of the motion conversion model in the robot control method provided in this application embodiment;

[0030] Figure 15 This is a schematic diagram of a robot control system provided in an embodiment of this application. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0032] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0033] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0034] Existing technologies for non-invasive BCI-based robot control primarily employ direct decoding methods. Specifically, motion parameters (such as hand trajectories or joint angles) are extracted directly from EEG signals, and control commands are generated through regression models to achieve robot control.

[0035] However, this approach is limited by the signal quality of the EEG signal, which is susceptible to noise interference, resulting in significant errors in continuous motion trajectories and unstable robot movements. Furthermore, it can only perform simple actions (such as robotic arm grasping). In addition, the generated commands may violate robot dynamics constraints (such as joint torque limits), leading to hardware damage or loss of control, and a lack of physical adaptability. Simultaneously, users cannot adjust their intentions based on robot feedback, resulting in a low success rate and a missing interactive loop.

[0036] This application proposes a robot control method to address the aforementioned problems. It involves real-time acquisition of the user's electroencephalogram (EEG) signals, determining the corresponding motion intention keywords based on the EEG signals and a pre-trained motion intention decoding model, and then determining the robot's reference trajectory sequence based on the motion intention keywords and a pre-trained motion conversion model. The robot's movement is then controlled according to this reference trajectory sequence. This method allows the generation of the robot's reference trajectory sequence from the motion intention keywords, avoiding the direct reduction of EEG signals to spatial trajectories. This means the robot's movement is no longer entirely limited by the signal quality of the EEG signals, improving the robot's motion stability and enabling it to perform a wider range of actions. Furthermore, converting the motion intention keywords into a reference trajectory sequence using the pre-trained motion conversion model ensures the reasonableness of the resulting reference trajectory sequence, thus improving the physical adaptability of the robot's movement. In addition, the user can adjust their intentions in real-time during robot movement, achieving a natural and robust brain-controlled robot interaction process.

[0037] First, the relevant background of the robot control method involved in the robot control method provided in the embodiments of this application will be explained.

[0038] It is understood that the robot control method provided in this application embodiment can be deployed in a robot, for example, in a humanoid robot or a humanoid robot.

[0039] Optionally, users can wear an electroencephalogram (EEG) signal acquisition device and interact with the robot through the EEG signal acquisition device, so that the robot can execute the steps of the robot control method provided in the embodiments of this application and perform corresponding movements.

[0040] For example, the EEG signal acquisition device can be used to acquire multi-channel EEG signals.

[0041] For example, a humanoid robot is a type of robot that mimics the appearance, movement, or behavior patterns of a human. It typically possesses human-like structures such as a head, torso, arms, and legs, and can perform human-like functions such as running, dancing, jumping, swimming, fighting, grasping, and facial expression interaction. Furthermore, each arm is composed of multiple joint mechanisms, enabling it to perform different actions depending on the scenario.

[0042] Optionally, the robot may include a mechanical structure, a drive system, a sensing system, a control system, and a power system.

[0043] The mechanical structure includes at least one robotic arm, end effectors corresponding to each robotic arm, two legs, and two feet. The mechanical structure is used to perform specific actions.

[0044] The drive system includes servo motors, etc., which are used to control the rotation or linear motion of each joint in the robotic arm and provide precise power output.

[0045] Optionally, the sensing system may include a torque sensor, a vision camera, an inertial measurement unit, and a tactile sensor. The sensing system is used to detect environmental information in real time to obtain real-time environmental information in order to control the robot.

[0046] The control system includes a processor and a memory, etc. The memory stores machine-readable instructions that can be executed by the processor. When the robot's control system is running, the processor executes the machine-readable instructions to perform the steps of the robot control method provided in the embodiments of this application, thereby controlling the operation of the robot.

[0047] The robot control method provided in this application will be described in detail below with reference to several embodiments.

[0048] Figure 1 This is a flowchart illustrating a robot control method provided in an embodiment of this application, with reference to... Figure 1 As shown, the executing entity of this method can be any electronic device with processing capabilities, such as the control system of the robot described above. The method includes:

[0049] S101. Real-time acquisition of the user's EEG signals.

[0050] Optionally, the user's electroencephalogram (EEG) signals can be acquired in real time. The user can be someone wearing the aforementioned EEG signal acquisition device, thus enabling the acquisition of EEG signals.

[0051] For example, a user can silently recite the movements that the robot needs to perform, thereby generating brainwave signals. A user can also imagine the movements that the robot needs to perform, thereby generating brainwave signals.

[0052] S102. Based on the EEG signal and the pre-trained motion intention decoding model, determine the motion intention keywords corresponding to the EEG signal.

[0053] Optionally, after acquiring EEG signals, a pre-trained motion intent decoding model can be used to convert the EEG signals into motion intent keywords. These motion intent keywords can be understood as high-level semantic motion intentions. The motion intent keywords are used to indicate the type of movement of the robot.

[0054] Specifically, the motion intent decoding model is used for semantic understanding based on EEG signals. It classifies EEG signals into discrete motion intents, thereby generating motion intent keywords. It can solve the noise problem of EEG signals through semantic abstraction, avoiding the robot motion stability error caused by directly decoding EEG signals into spatial trajectories. At the same time, motion intent keywords can also represent more types of motion, enabling robots to perform richer actions.

[0055] For example, EEG signals can be input into a pre-trained motion intention decoding model. The motion intention decoding model then processes the EEG signals to generate motion intention keywords corresponding to the EEG signals. The motion intention decoding model can be a combination of CNN and Transformer.

[0056] For example, the user can continuously imagine the movement that the robot needs to perform, such as "running", "dancing", "jumping", "swimming", "fighting", etc., where "dance" can specifically include specific dance movements such as "ballet" and "paso doble". The robot can determine the movement intention keyword corresponding to the EEG signal as one of "running", "ballet", "paso doble", "jumping", "swimming" or "fighting" based on the EEG signal and the movement intention decoding model.

[0057] S103. Determine the robot's reference trajectory sequence based on the motion intention keywords and the pre-trained motion conversion model.

[0058] Optionally, after obtaining the motion intent keywords, the motion intent keywords can be converted into continuous human motion trajectories through a pre-trained motion conversion model, thereby realizing text-to-motion modeling and generating a reference trajectory sequence for the robot.

[0059] The robot's reference trajectory sequence refers to the continuous motion trajectory reference sequence of the robot. Specifically, it refers to the smooth and coherent motion state changes of each joint of the robot over time. It can include a series of continuous poses or joint states of all the robot's joints and can be used to describe the complete dynamic process of motion.

[0060] For example, the motion transformation model can be implemented based on Transformer, capable of converting text input into motion trajectories. Specifically, the motion transformation model can be implemented based on any of the following methods: Transformer, VectorQuantized Variational Autoencoder (VQ-VAE), Motion Diffusion, Generative Adversarial Network (GAN), and Temporal Convolutional Network (TCN), thereby enabling the conversion of motion intent keywords into temporally continuous, physically feasible, and reference trajectory sequences that match the motion intent keywords.

[0061] For example, the reference trajectory sequence can be a reference trajectory sequence in the format of a three-dimensional human body model (Skinned Multi-Person Linear Model, or SMPL for short).

[0062] For example, taking "running" as the keyword of exercise intention, the reference trajectory sequence can be a continuous running trajectory, specifically including periodic leg swings and trunk balance.

[0063] S104. Control the robot's movement according to the reference trajectory sequence.

[0064] Optionally, after obtaining the reference trajectory sequence, a reinforcement learning (RL) strategy can be used to calculate the target action based on the reference trajectory sequence and the current robot state, and drive the motor to execute the target action to control the robot's movement.

[0065] Optionally, after obtaining the reference trajectory sequence, the reference trajectory sequence can be adapted to the robot joints to generate the robot joint angle trajectory. Then, through a pre-trained reinforcement learning strategy, the target action can be calculated based on the robot joint angle trajectory and the current robot state, and the motor can be driven to execute the target action to control the robot's movement.

[0066] For example, after obtaining the reference trajectory sequence, the current state of the robot can be observed to obtain the current state of the robot. Then, the target action can be calculated by using the pre-trained reinforcement learning policy, the pre-trained AMP discriminator, and the pre-trained regularization coefficients, and the motor can be driven to execute the target action to control the robot's movement.

[0067] For example, taking a continuous running trajectory as the reference trajectory sequence, after obtaining the reference trajectory sequence, the continuous running trajectory can be adapted to the robot joints through motion redirection to avoid knee joint over-limitation, and the robot can be controlled to run through reinforcement learning strategy to ensure that the robot does not fall while running.

[0068] For example, during actual testing, the user was able to successfully trigger control of the robot approximately 31 times every 3 minutes via brainwave signals.

[0069] In this embodiment, by collecting the user's EEG signals in real time, and based on the EEG signals and a pre-trained motion intention decoding model, motion intention keywords corresponding to the EEG signals are determined. Then, based on the motion intention keywords and a pre-trained motion conversion model, a reference trajectory sequence for the robot is determined. The robot's movement is controlled according to this reference trajectory sequence. This allows for precise control of the robot's movement directly through EEG signals, establishing an efficient mapping between EEG signals and robot actions, providing a new paradigm for human-robot collaboration. Simultaneously, generating motion intention keywords from EEG signals and then generating the robot's reference trajectory sequence from these keywords, thereby controlling the robot's movement through the reference trajectory sequence, decouples signal quality from motion generation. This avoids directly reducing EEG signals to spatial trajectories, ensuring that the robot's movement is no longer limited by the signal quality of the EEG signals. This significantly improves the stability of the robot's movements and ensures the physical feasibility of the generated trajectory.

[0070] Furthermore, by converting motion intent keywords into reference trajectory sequences using a pre-trained motion transformation model, the naturalness and fluency of the movements are ensured while satisfying actual physical constraints, significantly improving the practicality and reliability of the control process. In addition, users can adjust their intentions in real time during robot movement; this interactive feature makes the entire control process more natural and flexible, greatly enhancing the robustness and adaptability of human-robot collaboration. Through this brain-computer interface, end-to-end intelligent control from neural signals to complex robot movements has been successfully achieved. Moreover, by using motion intent keywords as an intermediate representation, the combination and innovation of robot movements are supported, allowing users to trigger complex actions with simple keywords without needing to train a separate model for each action, thus improving the diversity and generalization of robot movements. Furthermore, the determination of motion intent keywords and reference trajectory sequences can be optimized independently, reducing real-time computational load and improving computational efficiency and scalability.

[0071] In one possible implementation, Figure 2 This is a flowchart illustrating the process of determining the motion intention keyword corresponding to the electroencephalogram (EEG) signal in the robot control method provided in this application embodiment, with reference to... Figure 2As shown, in step S102 above, the motion intention keywords corresponding to the EEG signals are determined based on the EEG signals and the pre-trained motion intention decoding model, including:

[0072] S201. Perform frequency domain transformation on the EEG signal to generate the corresponding EEG features.

[0073] Optionally, the EEG signal can be transformed in the frequency domain using frequency domain feature extraction methods to generate corresponding EEG features. The frequency analysis methods can include Fourier transform, wavelet transform, and multivariate autoregression, among others.

[0074] For example, the received EEG signal can be transformed into frequency domain features by performing wavelet transform on the wavelet signal using predefined wavelet parameters, thereby generating the EEG features corresponding to the EEG signal.

[0075] Among them, EEG features can be used to reflect the pattern of energy distribution of brain neural oscillations changing over time. Essentially, it is a mapping of brain motor cortex neural oscillation energy on a two-dimensional time-frequency plane. It can not only distinguish neural rhythms of different functions from the frequency dimension, but also capture the dynamic process related to motor imagination from the time dimension.

[0076] By generating EEG features corresponding to EEG signals through wavelet transform, the EEG features can be directly correlated with known neural oscillation mechanisms, ensuring the physiological interpretability of the obtained EEG features. It can also filter transient artifacts (such as noise caused by blinking), ensuring the noise resistance of the obtained EEG features. Furthermore, wavelet transform uses a wide time window for smoothing in the low-frequency band and a narrow window for accurately capturing transients in the high-frequency band, ensuring that the obtained EEG features can be used for multi-resolution analysis.

[0077] S202. Based on the EEG characteristics and the motor intention decoding model, determine the motor intention keywords corresponding to the EEG signals.

[0078] Optionally, after obtaining the EEG features, the EEG features can be input into the motion intention decoding model. The motion intention decoding model performs motion intention decoding processing on the EEG features to obtain the motion intention keywords corresponding to the EEG signals.

[0079] By performing frequency domain transformation on EEG signals, corresponding EEG features are generated. Based on these features and a motor intent decoding model, the corresponding motor intent keywords are determined. This approach addresses the issues of low signal-to-noise ratio, non-stationarity, and physiological specificity of EEG signals. It also ensures that the obtained EEG features match neuroscience mechanisms and improves the robustness to noise during motor intent decoding. Furthermore, decoding using EEG features clarifies the physiological meaning of the features, facilitating algorithm debugging. Additionally, transforming EEG signals into EEG features reduces feature dimensionality, thereby lowering the computational complexity of the model.

[0080] In one possible implementation, Figure 3 This is a flowchart illustrating the generation of EEG features corresponding to EEG signals in the robot control method provided in this application embodiment, with reference to... Figure 3 As shown, in S201 above, frequency domain transformation is performed on the EEG signal to generate the corresponding EEG features, including:

[0081] S301. The EEG signal is input into a bandpass filter for filtering to generate a filtered signal.

[0082] Optionally, the EEG signal can be input into a pre-trained bandpass filter for filtering, thereby extracting effective neural activity related to motor intention, while suppressing noise and irrelevant physiological signals to obtain a filtered signal.

[0083] The bandpass filter can be a Chebyshev Type II filter with a frequency range of 0.5-50 Hz.

[0084] For example, the input EEG signal can be 128 channels × 1000Hz sampling rate. After processing by the difference equation of a bandpass filter, zero-phase filtering (filtfilt) is used to avoid time shift and ensure time alignment. The coefficients of the difference equation can be determined through training.

[0085] By filtering the EEG signal using a bandpass filter, the low-frequency drift component in the EEG signal can be completely suppressed, the key frequency band of the motor intention in the EEG signal can be preserved, the power frequency noise in the EEG signal can be significantly attenuated, and the electromyographic artifacts in the EEG signal can be significantly weakened. This results in a filtered signal that retains the motion-related μ / β rhythm and suppresses noise.

[0086] S302. Generate the separated signal based on the pre-trained separation matrix and the filtered signal.

[0087] Optionally, the filtered signal can be processed for artifact removal based on a pre-trained separation matrix to separate the neural signal from the artifact components and generate the separated signal. The artifact components can be eye points, electromyography (EMG), and electrocardiography (ECG).

[0088] The separation matrix can be an ICA weight matrix, specifically a separation matrix based on the Infomax algorithm. For example, the separation matrix can be obtained through pre-training.

[0089] For example, the independent components in the filtered signal can be calculated using the separation matrix, and artifact suppression can be performed based on the artifact component indexes marked during the training phase. Then, the artifact-free filtered signal can be obtained through inverse transformation, which is the separated signal.

[0090] By using the pre-trained separation matrix and the filtered signal, a separated signal is generated. The filtered signal can be filtered by component-based physiological features to achieve accurate artifact separation. Furthermore, it can preserve the neural activity related to motor imagery, achieve high-fidelity signal reconstruction, and also has the advantage of lightweight computation.

[0091] S303. Generate EEG features based on the separated signals and the pre-obtained wavelet parameters.

[0092] Optionally, after obtaining the separated signal, the separated signal is transformed into the time-frequency domain using pre-obtained wavelet parameters, thereby extracting neural oscillation features related to motor intention and obtaining EEG features.

[0093] The pre-obtained wavelet parameters can be acquired through training, including the mother wavelet type parameter and the scale parameter. Specifically, the mother wavelet type parameter can be a Morlet wavelet with a center frequency of 6. The scale parameter can include a frequency band coverage parameter, specifically 8-30Hz.

[0094] For example, each channel signal in the separated signal can be subjected to continuous wavelet transform to obtain a complex time-frequency matrix, which includes amplitude and phase information.

[0095] For example, the time-frequency energy distribution is obtained by taking the square of the modulus of the complex time-frequency matrix. The time-frequency energy distribution can be characterized as [time point, number of frequency bands].

[0096] For example, for each frequency band, the average value is taken along the time dimension to obtain the global features of that frequency band, thereby obtaining the global features of all frequency bands, which can be used as EEG features.

[0097] By generating EEG features using the separated signals and pre-obtained wavelet parameters, it is possible to explicitly extract μ / β rhythm changes, match the neural mechanisms of motor intentions, achieve physiological feature focusing, and improve noise robustness by suppressing irrelevant signals through frequency band isolation. At the same time, the single transformation takes less than 10ms, which also has the advantage of fast operation speed.

[0098] In one possible implementation, Figure 4 This is a schematic diagram of a motion intent decoding model in the robot control method provided in this application embodiment. Figure 5 This is a flowchart illustrating the process of determining the motion intention keyword corresponding to the electroencephalogram (EEG) signal in the robot control method provided in this application embodiment, with reference to... Figure 4 as well as Figure 5 As shown, the motion intention decoding model includes: a frequency segmentation and conversion module, a motion intention encoding module, and a decoding output module. In step S202 above, based on EEG characteristics and the motion intention decoding model, the motion intention keywords corresponding to the EEG signal are determined, including:

[0099] S501. Input the EEG features into the frequency segmentation and conversion module, perform segmentation and conversion along the frequency dimension, and generate a multi-band feature sequence.

[0100] Optionally, the EEG features are input into the frequency segmentation and conversion module, the EEG features are segmented along the frequency dimension and converted into discrete tokens, and after spatial feature extraction and location embedding, a multi-band feature sequence is generated.

[0101] For example, taking a 128×20 matrix as an example of a time-frequency slice of EEG features at a certain moment, a multi-band feature sequence with shape [frequency band=20, embedding dimension=512] can be generated through the frequency segmentation and conversion module. Each token in the multi-band feature sequence corresponds to all channel information of one frequency band.

[0102] By inputting EEG features into the frequency segmentation and transformation module for segmentation and transformation along the frequency dimension, multi-band feature sequences are generated. This enables explicit modeling of the spatial-frequency joint features of EEG, significantly improving the robustness of motion intent decoding while avoiding the limitations of traditional manual feature extraction.

[0103] S502. Input the multi-band feature sequence into the motion intent encoding module for cross-band enhanced encoding to generate enhanced features.

[0104] Optionally, after obtaining the multi-band feature sequence, the multi-band feature sequence is input into the motion intent encoding module for cross-band enhancement encoding, encoding the input multi-band feature sequence into high-level motion intent features, resulting in enhanced features. The enhanced features can be understood as an enhanced temporal feature sequence, where each token contains enhanced frequency band-time information, which can be used to indicate global dependencies between frequency bands and local time-frequency patterns.

[0105] By inputting multi-band feature sequences into the motion intent coding module for cross-band enhanced coding, enhanced features are generated. These enhanced features can serve as joint frequency band-time features, preserving the original temporal structure while encoding inter-band relationships. This allows the motion intent decoding process to utilize both intra-band features and cross-band relationships, ultimately improving the decoding accuracy of motion intent.

[0106] S503. The enhanced feature input decoding output module is used to decode the motion intention and generate motion intention keywords corresponding to the EEG signal.

[0107] Optionally, the enhanced features are input into the decoding output module for motion intent decoding, and the enhanced features are converted into a probability distribution of motion keywords, so that the keyword with the highest probability can be selected as the output to generate the motion intent keyword corresponding to the EEG signal.

[0108] For example, the decoding output module can be implemented based on a classifier to focus on key features, thereby achieving efficient decoding and obtaining the motion intention keywords corresponding to the EEG signal.

[0109] By inputting EEG features into a frequency segmentation and conversion module for segmentation and conversion along the frequency dimension, a multi-band feature sequence is generated. This multi-band feature sequence is then input into a motion intention encoding module for cross-band enhanced encoding, generating enhanced features. Finally, the enhanced features are input into a decoding and output module for motion intention decoding, generating motion intention keywords corresponding to the EEG signal. This approach combines frequency domain prior guidance with data-driven modeling, enabling frequency domain features to directly correspond to neurophysiological markers, achieving high-precision motion intention decoding. Furthermore, it exhibits strong robustness and real-time response.

[0110] In one possible implementation, Figure 6 This is a schematic diagram of a frequency segmentation and conversion module in the motion intent decoding model of the robot control method provided in this application embodiment. Figure 7 This is a flowchart illustrating the generation of multi-band feature sequences in the robot control method provided in this application embodiment, with reference to... Figure 6 as well as Figure 7As shown, the frequency segmentation and conversion module includes a spatial encoder and a location embedding module. In step S501 above, the EEG features are input to the frequency segmentation and conversion module for segmentation and conversion along the frequency dimension, generating a multi-band feature sequence, including:

[0111] S701. Divide the EEG features into multiple sub-feature sequences according to frequency bands.

[0112] Optionally, EEG features can be segmented into multiple sub-feature sequences according to frequency bands. Specifically, a continuous frequency axis can be divided into 20 non-overlapping sub-bands, resulting in multiple sub-feature sequences. Each sub-feature sequence corresponds to a frequency band.

[0113] For example, taking a continuous frequency axis of 8-30Hz as an example, 20 non-overlapping sub-bands can be obtained, each band covering a bandwidth of 1.1Hz, and the range covered by each band is taken as a frequency band.

[0114] For example, for each frequency band, channel energy aggregation is performed separately, the average energy of all 128 channels in the frequency band is calculated, and each frequency band feature is mapped to a 512-dimensional space to achieve token embedding, resulting in multiple sub-feature sequences.

[0115] By segmenting EEG features into multiple sub-feature sequences according to frequency bands, each sub-feature sequence can focus on a specific frequency band, preserving frequency locality and avoiding cross-frequency band confusion. At the same time, all channels within the same frequency band can be aggregated, enhancing the signal-to-noise ratio. Furthermore, adding new frequency bands only requires increasing the number of sub-feature sequences without modifying the model structure, and it also has a certain degree of scalability.

[0116] S702. Input each sub-feature sequence into the spatial encoder to extract spatial features and generate the extracted sequences for each frequency band.

[0117] Optionally, each sub-feature sequence is input into a spatial encoder, which extracts spatial features from the EEG signals of all channels within the same frequency band, captures the coordinated activity patterns of different brain regions, and generates the extracted sequences for each frequency band.

[0118] For example, taking a sub-feature sequence as an example, the input of the spatial encoder is all channel data of a single frequency band, and the shape can be [number of channels N=128, time point T].

[0119] For example, the spatial encoder can first perform channel-dimensional convolution processing, using 1D convolution kernels to slide along the channel dimension, calculate local spatial patterns, and output feature maps: [number of channels N, time point T, number of convolution kernels C]. Then, global average pooling is performed on the output feature maps. Specifically, pooling is performed along the channel dimension to generate the initial representation of the frequency band token: [1, T, C].

[0120] For example, the initial representation of the frequency band token is mapped to 512 dimensions through a fully connected layer to achieve linear projection, resulting in an extracted sequence of a frequency band.

[0121] By inputting each sub-feature sequence into a spatial encoder for spatial feature extraction, and generating extracted sequences for each frequency band, irrelevant channel noise in the EEG signal can be eliminated, such as interference from the occipital lobe region on motor intention. At the same time, it can also enhance the contribution of motor cortex-related channels, thereby accurately capturing the coordinated activity patterns of different brain regions.

[0122] S703. Frequency location information is added to the extracted sequences of each frequency band through the location embedding module, and multi-band feature sequences are output.

[0123] Optionally, after obtaining the extracted sequences of each frequency band, frequency position information can be added to the extracted sequences of each frequency band through the position embedding module, while retaining the frequency band order information, thereby distinguishing the physiological significance of different frequency bands, realizing frequency band-specific physiological function encoding, and obtaining multi-frequency band feature sequences.

[0124] In the multi-band feature sequence, each feature corresponds to all channel information of a frequency band.

[0125] For example, the location embedding module may include a learnable location coding matrix, which can be used to add different location codes to the extracted sequences of each frequency band according to the different positions of each frequency band.

[0126] For example, taking a multi-band feature sequence with the shape of [frequency band = 20, embedding dimension = 512] as an example, the multi-band feature sequence can be represented as a 20×512 matrix, where each row vector of the matrix corresponds to the global feature representation of a frequency band.

[0127] By adding frequency location information to the extracted sequences of each frequency band through the location embedding module, a multi-band feature sequence is output, which can preserve the cooperative relationship between frequency bands. This allows for the simultaneous use of intra-band features and cross-band relationships when decoding motion intent.

[0128] In one possible implementation, Figure 8 This is a schematic diagram of a motion intent encoding module in the motion intent decoding model of the robot control method provided in this application embodiment. Figure 9 This is a schematic flowchart illustrating the generation of enhanced features in the robot control method provided in this application embodiment, with reference to... Figure 8 as well as Figure 9As shown, the motion intent encoding module includes: a first encoder and a second encoder; in S502 above, the multi-band feature sequence is input into the motion intent encoding module for cross-band enhanced encoding to generate enhanced features, including:

[0129] S901. Input the multi-band features into the first encoder for local feature extraction to generate initial motion intention features.

[0130] Optionally, the multi-band features are input into the first encoder for local feature extraction, extracting local temporal features, such as short-term patterns of brain waves, and performing coarse-grained intent recognition to generate initial motion intent features.

[0131] For example, the first encoder can be implemented based on multi-head self-attention (MHSA) and a convolution module. MHSA focuses on the coordinated activity of global brain regions, while the convolution module extracts local features of frequency bands (such as the rhythm of α / β / γ bands) to generate initial motion intention features.

[0132] For example, the first encoder may also include: a feed-forward network (FFN), layer normalization, and residual connections.

[0133] S902. Input the initial motion intent features into the second encoder for semantic association modeling to generate enhanced features.

[0134] Optionally, the initial motor intention features are input into a second encoder for semantic association modeling to capture global semantic associations, such as the synergistic relationship between signals from different brain regions, and fine-grained keyword classification is performed to generate enhanced features.

[0135] For example, the second encoder can be implemented based on multi-head self-attention (MHSA) and a convolution module. MHSA establishes high-level semantic associations of motion keywords (e.g., the neural signal differences between "run" and "jump"), and the convolution module refines temporal alignment (e.g., the precise time points of action start / end) to generate enhanced features.

[0136] For example, the second encoder may also include a feed-forward network (FFN), layer normalization, and residual connections.

[0137] By capturing local EEG patterns with the first encoder and integrating global context with the second encoder, efficient multi-scale modeling is possible, achieving a balance between computational efficiency and signal decoding accuracy. Furthermore, since only two encoders are used, overfitting can be avoided, making it suitable for low signal-to-noise ratio scenarios in non-invasive brain-computer interfaces.

[0138] In one possible implementation, Figure 10 This is a schematic diagram of a decoding output module in the motion intent decoding model of the robot control method provided in this application embodiment. Figure 11 This is a flowchart illustrating the generation of motion intention keywords corresponding to EEG signals in the robot control method provided in this application embodiment, with reference to... Figure 10 as well as Figure 11 As shown, the decoding output module includes a pooling layer and a classifier; in S503 above, the enhanced features are input into the decoding output module for motion intent decoding to generate motion intent keywords corresponding to the EEG signal, including:

[0139] S1101. The enhanced features are input into the pooling layer for feature aggregation processing to generate feature vectors.

[0140] Optionally, the enhanced features are input into the pooling layer for temporal dimension downsampling and feature aggregation processing to compress the enhanced features, retain the strongest frequency band response related to motion intent, and generate feature vectors.

[0141] The feature vector is used to characterize the intention encoding of the EEG signal, which integrates multi-band and multi-time step information and includes joint spatial-frequency-temporal features.

[0142] For example, the pooling layer can be a max pooling layer. Taking the enhanced feature shape as [time step=20, feature dimension=512] as an example, the pooling layer can take the maximum value of the enhanced feature along the time dimension, eliminate short-term noise interference such as single-frame artifacts, and highlight the continuous and stable motion image features, generating a feature vector with shape [1, 512].

[0143] S1102. Input the feature vector into the classifier for keyword mapping to generate motion intention keywords corresponding to the EEG signal.

[0144] Optionally, the feature vector is input into a classifier for keyword mapping, the feature vector is mapped to the probability distribution of motion keywords, and the keyword with the highest probability is selected as the output to generate motion intention keywords corresponding to the EEG signal.

[0145] The process of generating motion intent keywords has been explained in detail above. It can be understood that after obtaining the motion intent keywords, the robot's reference trajectory sequence must be determined based on the motion intent keywords and the pre-trained motion conversion model. The following is a detailed explanation.

[0146] In one possible implementation, Figure 12 This is a schematic diagram of a motion conversion model in the robot control method provided in this application embodiment. Figure 13 This is a flowchart illustrating the process of determining the reference trajectory sequence of a robot in the robot control method provided in this application embodiment, with reference to... Figure 12 as well as Figure 13 As shown, the motion conversion model includes a text encoder, a text motion generation module, and a motion decoder. In step S103 above, based on the motion intent keywords and the pre-trained motion conversion model, the robot's reference trajectory sequence is determined, including:

[0147] S1301. Input the motion intent keywords into the text encoder for text conversion processing to generate a text sequence.

[0148] Optionally, the motion intent keywords are input into a pre-trained text encoder for text conversion processing to generate a text sequence. The text encoder can be a pre-trained language model (e.g., T5, BERT).

[0149] S1302. Input the text sequence into the text motion generation module for autoregressive prediction to generate a motion sequence.

[0150] Optionally, the text sequence is input into the text motion generation module for autoregressive generation to obtain the motion sequence. The text motion generation module can be trained based on a Transformer model, and during training, it learns the mapping relationship between the text sequence and the motion sequence.

[0151] For example, the text motion generation module can perform stepwise autoregression on each subsequence in the text sequence to generate a motion sequence.

[0152] S1303. Input the motion sequence into the motion decoder for decoding processing to generate a reference trajectory sequence.

[0153] Optionally, the motion sequence is input into a pre-trained motion decoder for decoding, and the motion sequence is reconstructed into a continuous reference trajectory sequence.

[0154] The motion decoder can be implemented using any of the following methods: Transformer, Vector Quantized Variational Autoencoder (VQ-VAE), Motion Diffusion, Generative Adversarial Network (GAN), and Temporal Convolutional Network (TCN).

[0155] By inputting motion intent keywords into a text encoder for text conversion, a text sequence is generated. This text sequence is then input into a text motion generation module for autoregressive prediction to generate a motion sequence. Finally, by inputting the motion sequence into a motion decoder for decoding, a reference trajectory sequence is generated. This method can convert motion keywords in text form into discrete motion sequences and then decode them into continuous motion trajectories. It can decouple signal quality from motion generation, thereby resisting EEG signal noise and improving the robustness of the reference motion trajectory generation process.

[0156] In one possible implementation, Figure 14 This is a schematic flowchart illustrating the training process of the motion conversion model in the robot control method provided in this application embodiment, with reference to... Figure 14 As shown, the training process of the motion conversion model includes:

[0157] S1401. Obtain the training set of motion data.

[0158] Optionally, a motion data training set can be obtained. This training set can be a large-scale human motion dataset, such as the AMASS dataset.

[0159] S1402. Based on the motion data training set, train the motion segmenter in the initial motion conversion model, and use the motion decoder in the initial motion conversion model at the end of training as the motion decoder in the motion conversion model.

[0160] The initial motion conversion model includes at least a motion segmenter and a text motion generation module. The motion segmenter includes a motion encoder, a motion quantizer, and a motion decoder.

[0161] Optionally, after obtaining the motion data training set, the motion segmenter in the initial motion conversion model can be trained using the motion data training set, and the motion decoder in the initial motion conversion model at the end of training can be used as the motion decoder in the motion conversion model during the inference process.

[0162] For example, the motion segmenter can be a Vector Quantized Variational Autoencoder (VQ-VAE), which includes a motion encoder, a motion quantizer, and a motion decoder.

[0163] For example, the process of training a motion segmenter includes training a motion encoder, a motion quantizer, and a motion decoder.

[0164] Specifically, the motion encoder is used to map the training set of motion data to the latent space and output low-dimensional continuous latent variables.

[0165] Specifically, the motion quantizer is used to discretize low-dimensional continuous latent variables using a codebook to obtain discrete motion sequences.

[0166] Specifically, the motion decoder is used to reconstruct a training set of motion data from discrete motion sequences.

[0167] The loss function includes reconstruction loss and codebook learning loss.

[0168] S1403, Obtain the training set of motion keywords.

[0169] Optionally, a motion keyword training set can be obtained. This motion keyword training set can be the same as the motion data training set.

[0170] S1404. Input the motion data training set into the initial motion decoder in the initial motion conversion model at the end of training to generate motion sequence data.

[0171] Optionally, the motion data training set is input into the initial motion decoder in the initial motion conversion model at the end of training, i.e., the motion decoder used by the motion conversion model during the inference process, to convert the motion data training set and generate motion sequence data corresponding to the motion data training set.

[0172] S1405. Based on motion sequence data and motion keyword training set, train the text motion generation module in the initial motion conversion model, and use the text motion generation module in the initial motion conversion model at the end of training as the text motion generation module in the motion conversion model.

[0173] Optionally, the text motion generation module in the initial motion conversion model can be trained based on motion sequence data and a motion keyword training set, so that the text motion generation module can learn the mapping relationship between text sequences and motion sequences.

[0174] Optionally, the text motion generation module in the initial motion conversion model at the end of training can be used as the text motion generation module in the motion conversion model during the inference process.

[0175] For example, the text encoder input to the motion keyword training set is subjected to text conversion processing to generate a motion keyword text sequence corresponding to the motion keyword training set.

[0176] For example, the text sequence of motion keywords and motion sequence data are input into the text motion generation module of the initial motion conversion model for autoregressive training. Specifically, the training is performed using a causal mask method, which only allows attention to historical information and uses the text sequence and a portion of the motion sequence as prefixes to predict the next motion sequence.

[0177] For example, the loss function can be the negative log-likelihood (NLL).

[0178] By optimizing the motion segmenter during the training phase, a codebook can be constructed, which represents motion data as discrete token sequences for the text motion generation module to learn. During the inference phase, only the motion decoder and the text motion generation module are needed for inference, which can efficiently map natural language to complex motion trajectories, avoid redundant calculations, and improve inference efficiency.

[0179] In one possible implementation, controlling the robot's motion according to the reference trajectory sequence in S104 above includes:

[0180] The reference trajectory sequence is adapted to the robot joints to generate the robot joint angle trajectory. Based on the robot joint angle trajectory and the current robot state, the target action is calculated using the pre-trained robot control strategy, and the motors are driven to execute the target action to control the robot's movement.

[0181] Optionally, a pre-constructed equivalent joint mapping relationship between the human body and the robot can be obtained, and the reference trajectory sequence can be transformed into the robot's base coordinate system to obtain the target trajectory sequence. The coordinate system of the reference trajectory sequence can be a coordinate system based on the pelvic center, and the robot's base coordinate system can be a coordinate system based on the torso center.

[0182] Optionally, based on the target trajectory sequence, the joint angles of each joint of the robot at multiple time steps are calculated by combining the inverse kinematics algorithm and the gradient descent algorithm with preset joint angle limit constraints.

[0183] Optionally, after obtaining the joint angles of each joint of the robot at multiple time steps, the joint torque and joint speed of each joint at multiple time steps can be calculated based on the joint angles of each joint at multiple time steps to determine whether they meet the physical feasibility requirements. If not, the joint angles of each joint at multiple time steps can be recalculated to ensure that the movement of each joint meets the physical feasibility requirements.

[0184] Optionally, after obtaining the joint angles of each joint of the robot at multiple time steps, the smoothness of the movement trajectory of each joint of the robot can be determined by the joint angles of each joint of the robot at multiple time steps. If not, the joint angles of each joint at multiple time steps can be smoothed by interpolation or filtering to ensure that the generated joint trajectory conforms to the robot's dynamic capabilities.

[0185] Optionally, the robot control strategy obtained through pre-training can be used to calculate the target action based on the joint angles of each joint at multiple time steps and the current robot state, and drive the motor to execute the target action in order to control the robot's movement.

[0186] Optionally, the robot control strategy can be implemented based on a reinforcement learning strategy. During the training process of the robot control strategy, an adversarial motion prior (AMP) discriminator can be introduced to punish movements that do not conform to human style and to give higher weights to key joints (such as the spine and legs) to ensure that the core features of the movement remain unchanged.

[0187] Alternatively, the robot control strategy can be implemented based on reinforcement learning and imitation learning.

[0188] For example, human motion can be extracted from AMASS dataset or Xsens motion capture data, and the robot joint space trajectory can be obtained by motion retargeting. A state representation can be defined. For example, an observation space can be defined, which includes: joint angles and angular velocities, torso posture, and several future frames of the reference trajectory.

[0189] For example, the mapping from state to reference action can be learned directly through behavior cloning (BC), and a discriminator can be introduced to distinguish between actions generated by the policy and the reference trajectory. The goal of the policy is to deceive the discriminator, making it unable to distinguish between generated actions and real actions. The reward function can also be set such that the reward approaches 1 when the policy action is close to the reference trajectory.

[0190] For example, in the reinforcement learning policy optimization stage, the reinforcement learning policy can be optimized based on the Proximal Policy Optimization (PPO algorithm) and the Adversarial Motion Prior (AMP).

[0191] Specifically, the reward function can include trajectory tracking reward, physical constraint reward, AMP-style reward, and regularization reward, thereby ensuring that the actions generated by the reinforcement learning policy not only meet the trajectory tracking requirements and physical constraints, but also ensure that the actions are natural, safe, physically feasible, and smooth.

[0192] Specifically, the discriminator with adversarial motion priors can be pre-trained using human motion data to evaluate the "human similarity" of the policy-generated actions. The policy learns natural actions by maximizing the AMP reward.

[0193] For example, robot dynamics can be simulated using multi-joint contact dynamics (MuJoCo) or the Python bullet physics engine (PyBullet) to construct a simulation environment and randomize simulation parameters, thereby improving the generalization ability of the strategy. Specifically, simulation parameters such as the ground friction coefficient, mass distribution, and motor delay can be randomized.

[0194] For example, the training termination condition for the robot control strategy can be that the control strategy can stably track the reference trajectory in the simulation, for example, the success rate is greater than a preset threshold, such as 95%. Alternatively, the AMP discriminator cannot distinguish between the strategy action and the reference trajectory, for example, the discriminator accuracy is ≈ 50%.

[0195] In the process of training robot control strategies, introducing adversarial motion priors can enable the motion to be realized in a human style. Introducing regularized rewards can ensure the safety of the obtained actions. By randomizing simulation parameters, the strategy can be forced to learn to adapt to diverse conditions, thereby improving its robustness in the real world and solving the problem of performance degradation of robot control strategies trained in simulation environments when deployed in the real world.

[0196] This application also provides a robot control system, such as... Figure 15 As shown, Figure 15 This application provides a schematic diagram of a robot control system, including a processor 1501, a memory 1502, and optionally, a bus 1503. The memory 1502 stores machine-readable instructions executable by the processor 1501. When the robot control system is running, the processor 1501 and the memory 1502 communicate via the bus 1503. When the machine-readable instructions are executed by the processor 1501, the steps of the aforementioned robot control method are performed.

[0197] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A robot control method characterized by, The method comprises the following steps: real-time acquisition of the user's electroencephalogram signal, frequency domain transformation of the electroencephalogram signal, generation of the electroencephalogram signal corresponding to the electroencephalogram feature; According to the electroencephalogram feature and the pre-trained motion intention decoding model, the motion intention key corresponding to the electroencephalogram signal is determined; According to the motion intention key and the pre-trained motion conversion model, the motion intention key is converted into a continuous human motion trajectory by the motion conversion model to determine the reference trajectory sequence of the robot; According to the reference trajectory sequence, the motion of the robot is controlled; Wherein, the motion intention decoding model comprises: frequency segmentation conversion module, motion intention encoding module and decoding output module; according to the electroencephalogram feature and the pre-trained motion intention decoding model, the motion intention key corresponding to the electroencephalogram signal is determined, which comprises: The electroencephalogram feature is input into the frequency segmentation conversion module for frequency dimension segmentation and conversion processing to generate a multi-band feature sequence; the multi-band feature sequence is input into the motion intention encoding module for cross-band enhanced coding to generate an enhanced feature; the enhanced feature is input into the decoding output module for motion intention decoding to generate the motion intention key corresponding to the electroencephalogram signal; The frequency segmentation conversion module comprises: a spatial encoder and a position embedding module; the electroencephalogram feature is input into the frequency segmentation conversion module for frequency dimension segmentation and conversion processing to generate a multi-band feature sequence, which comprises: The electroencephalogram feature is divided into a plurality of sub-feature sequences according to the frequency band, and each sub-feature sequence corresponds to a frequency band; each sub-feature sequence is input into the spatial encoder for spatial feature extraction to generate an extracted sequence of each frequency band; the position embedding module is used to add frequency position information to the extracted sequence of each frequency band to output a multi-band feature sequence.

2. The robot control method according to claim 1, characterized by, The electroencephalogram signal is input into a band-pass filter for filtering processing to generate a filtered signal; According to the pre-trained separation matrix and the filtered signal, a separated signal is generated; According to the separated signal and the pre-obtained wavelet parameter, the electroencephalogram feature is generated. Each feature in the multi-band feature sequence corresponds to all channel information of a frequency band.

3. The robot control method according to claim 1, wherein, The motion intention encoding module comprises: a first encoder and a second encoder; 4. The robot control method according to claim 1, characterized by, The multi-band feature sequence is input into the motion intention encoding module for cross-band enhanced coding to generate an enhanced feature, which comprises: The multi-band feature is input into the first encoder for local feature extraction to generate an initial motion intention feature; The initial motion intention feature is input into the second encoder for semantic association modeling to generate an enhanced feature. The decoding output module comprises: a pooling layer and a classifier; 5. The robot control method according to claim 1, wherein, The enhanced feature is input into the decoding output module for motion intention decoding to generate the motion intention key corresponding to the electroencephalogram signal, which comprises: ​ Input the enhanced feature into the pooling layer for feature aggregation processing to generate a feature vector; Input the feature vector into the classifier for keyword mapping to generate a movement intention keyword corresponding to the electroencephalogram signal.

6. The robot control method according to claim 1, wherein, The movement conversion model comprises a text encoder, a text motion generation module, and a movement decoder; The reference trajectory sequence of the robot is determined according to the movement intention keyword and a pre-trained movement conversion model, comprising: The movement intention keyword is input into the text encoder for text conversion processing to generate a text sequence; The text sequence is input into the text motion generation module for autoregressive prediction to generate a movement sequence; The movement sequence is input into the movement decoder for decoding processing to generate the reference trajectory sequence.

7. The robot control method according to claim 1, wherein, The training process of the movement conversion model comprises: Obtaining a movement data training set; Based on the movement data training set, training a movement tokenizer in an initial movement conversion model, and taking the movement decoder in the initial movement conversion model at the end of the training as the movement decoder in the movement conversion model, the initial movement conversion model at least comprising a movement tokenizer and a text motion generation module, the movement tokenizer comprising a movement encoder, a movement quantizer, and a movement decoder; Obtaining a movement keyword training set; Inputting the movement data training set into the initial movement decoder in the initial movement conversion model at the end of the training to generate movement sequence data; Based on the movement sequence data and the movement keyword training set, training the text motion generation module in the initial movement conversion model, and taking the text motion generation module in the initial movement conversion model at the end of the training as the text motion generation module in the movement conversion model.

8. A control system of a robot characterized by comprising: Comprise: A processor and a memory, the memory storing machine readable instructions executable by the processor, when the control system of the robot is running, the processor executes the machine readable instructions to execute the steps of the robot control method as claimed in any one of claims 1 to 7.

Images