Mechanical arm control method and device based on brain-computer interface and edge computing terminal

Abstract

The invention relates to the technical field of brain-computer interfaces, in particular to a mechanical arm control method and device based on a brain-computer interface and an edge computing terminal.The method comprises the steps that an initial electroencephalogram signal of a user is collected; preprocessing the initial electroencephalogram signal to obtain an actual electroencephalogram signal of the user; and extracting at least one of a waveform feature, a power spectral density feature and a time-frequency domain feature of the actual electroencephalogram signal, and inputting the at least one of the waveform feature, the power spectral density feature and the time-frequency domain feature into a pre-constructed hybrid model for decoding so as to output an actual intention of the user and send a control instruction corresponding to the actual intention to control end equipment corresponding to the target mechanical arm. Therefore, the problems that when a BCI system in the related technology converts electroencephalogram signals into control signals, stability and precision are insufficient in the face of a complex environment, signal processing is delayed, mechanical response is slow, the self-adaptive capacity is poor, and signal processing is not stable are solved. And the complexity and the cost of the system are increased, and the generalization and the practical application capability of the BCI technology are limited at the same time.

Description

Technical Field

[0001] The present application relates to the field of brain-computer interface technology, and in particular to a method and device for controlling a robotic arm based on a brain-computer interface, and an edge computing terminal. Background Art

[0002] In the related technology, in the field of brain-computer interface (BCI), it has been realized to convert electroencephalogram (EEG) signals into control signals to achieve control of external devices or systems. This includes key steps such as data acquisition, signal preprocessing, feature extraction, decoding and classification, as well as control and feedback. Specifically, EEG equipment is used to collect electrical signals generated by brain activity, and then the signal quality is improved through preprocessing steps such as filtering, artifact removal and baseline correction. Next, features related to specific cognitive states or intentions are extracted from the preprocessed EEG signals, and these features are mapped to specific control commands using machine learning algorithms. Finally, the decoded intentions are converted into control signals for external devices, and feedback is provided to adjust the control strategy.

[0003] However, the BCI system of related technologies lacks accuracy and stability when decoding EEG signals, especially in complex or changing environments, and many BCI systems have delays in processing and decoding EEG signals, which affects the real-time response capability of the system. In addition, BCI systems often require users to undergo long-term training so that the system can adapt to the user's EEG signal characteristics, which limits the ease of use and popularity of the system. In addition, the EEG signal characteristics between different users vary greatly, resulting in the need for personalized adjustments to the BCI system, which increases the complexity and cost of the system and limits the application potential of BCI technology in a wider range of fields, which needs to be solved urgently. Summary of the invention

[0004] The present application provides a method, device and edge computing terminal for controlling a robotic arm based on a brain-computer interface to solve the problems in the related art that when the BCI system converts electroencephalogram (EEG) signals into control signals, the stability and accuracy are insufficient in the face of complex environments, and there is a delay in signal processing, resulting in slow mechanical response and poor adaptability, which increases the complexity and cost of the system while limiting the generalization and practical application capabilities of BCI technology.

[0005] The first aspect of the present application provides a method for controlling a robotic arm based on a brain-computer interface, comprising the following steps: collecting an initial EEG signal of a user; preprocessing the initial EEG signal to obtain an actual EEG signal of the user; extracting at least one of the waveform characteristics, power spectral density characteristics, and time-frequency domain characteristics of the actual EEG signal, and inputting the at least one into a pre-built hybrid model for decoding to output the actual intention of the user, so as to send a control instruction corresponding to the actual intention to a control end device corresponding to the target robotic arm.

[0006] Through the above technical means, the embodiment of the present application can collect the user's EEG signal and extract at least one of the waveform characteristics, power spectrum density characteristics and time-frequency domain characteristics in the EEG signal to obtain the corresponding control instructions to control the robot arm. As a result, it is achieved that the data processing and preliminary analysis tasks are deployed on the edge computing terminal near the user through edge computing technology, reducing data transmission delays and improving the real-time response capability of the BCI system. Through online transfer learning technology, the BCI system can quickly adapt to the real-time EEG signal characteristics of new users and solve the problem of individual difference adaptability. Through the hybrid model decoding strategy, combined with deep learning and traditional machine learning technology, the decoding accuracy and stability are effectively improved, and the user is fed back in a multi-modal manner based on the actual control results, which effectively improves the human-computer interactivity and intelligence level of this application.

[0007] Optionally, in one embodiment of the present application, the preprocessing of the initial EEG signal to obtain the actual EEG signal of the user includes: removing electrooculographic artifact signals and electromyographic artifact signals in the initial EEG signal to obtain the first actual EEG signal of the user; and / or calculating the average value of all electrodes that collect the initial EEG signal, and processing the initial EEG signal based on the average value to obtain the second actual EEG signal of the user.

[0008] Through the above-mentioned technical means, the embodiment of the present application can remove the electrooculographic artifact signals and electromyographic artifact signals in the initial EEG signal, effectively improving the quality of the initial EEG signal. The embodiment of the present application can improve the signal-to-noise ratio of the EEG signal by calculating the average value of all electrode signals in the initial EEG signal and adopting the common average reference spatial filtering technology in the spatial filtering technology, which can make the characteristics of the EEG signal more obvious and help the subsequent interpretation and analysis of the EEG signal.

[0009] Optionally, in one embodiment of the present application, before inputting at least one of the above into the pre-built hybrid model for decoding, it also includes: collecting historical data and behavior patterns of the user; and updating at least one model parameter of the hybrid model based on the historical data and the behavior patterns.

[0010] Through the above-mentioned technical means, the embodiment of the present application can collect the user's historical data and behavior patterns as training data to train a certain hybrid model to update the model parameters of the hybrid model, which can effectively improve the accuracy of the hybrid model in the present application and thereby enhance the application capability of the hybrid model in actual scenarios.

[0011] Optionally, in one embodiment of the present application, before inputting the at least one into the pre-built hybrid model for decoding, it also includes: generating adjustment parameters of the hybrid model based on the at least one; and adjusting at least one model parameter of the hybrid model using the adjustment parameters.

[0012] Through the above-mentioned technical means, the embodiment of the present application can realize online transfer learning of the hybrid model based on at least one feature in the user's actual EEG signal. When it is detected that the EEG signal characteristics of the user change significantly when performing a specific task, the model parameters are fine-tuned accordingly by calculating the relationship between the feature change amount and the model parameters to improve the model's decoding accuracy of the user's intention.

[0013] Optionally, in one embodiment of the present application, it also includes: obtaining execution feedback information of the target robotic arm; generating multimodal instructions based on the execution feedback information, and using the multimodal instructions to provide the user with at least one of visual, auditory, tactile and olfactory biofeedback prompts.

[0014] Through the above-mentioned technical means, the embodiment of the present application can generate execution feedback information based on the execution status of the target robotic arm, so as to generate multimodal feedback prompts to the user according to the execution feedback information of the target robotic arm, including but not limited to biological feedback such as vision, hearing, touch and smell, thereby effectively improving the human-computer interactivity of the present application and enhancing the user's perception of control and control accuracy.

[0015] The second aspect of the present application provides a brain-computer interface-based robotic arm control device, including: a first acquisition module, used to acquire the user's initial EEG signal; a processing module, used to preprocess the initial EEG signal to obtain the user's actual EEG signal; a control module, used to extract at least one of the waveform characteristics, power spectral density characteristics and time-frequency domain characteristics of the actual EEG signal, and input the at least one into a pre-built hybrid model for decoding to output the user's actual intention, so as to send the control instruction corresponding to the actual intention to the control end device corresponding to the target robotic arm.

[0016] Through the above technical means, the embodiment of the present application can collect the user's EEG signal and extract at least one of the waveform characteristics, power spectrum density characteristics and time-frequency domain characteristics in the EEG signal to obtain the corresponding control instructions to control the robot arm. As a result, it is achieved that the data processing and preliminary analysis tasks are deployed on the edge computing terminal near the user through edge computing technology, reducing data transmission delays and improving the real-time response capability of the BCI system. Through online transfer learning technology, the BCI system can quickly adapt to the real-time EEG signal characteristics of new users and solve the problem of individual difference adaptability. Through the hybrid model decoding strategy, combined with deep learning and traditional machine learning technology, the decoding accuracy and stability are effectively improved, and the user is fed back in a multi-modal manner based on the actual control results, which effectively improves the human-computer interactivity and intelligence level of this application.

[0017] Optionally, in one embodiment of the present application, the processing module includes: a removal unit for removing electrooculographic artifact signals and electromyographic artifact signals in the initial EEG signal to obtain the first actual EEG signal of the user; and / or a calculation unit for calculating the average value of all electrodes for collecting the initial EEG signal, and processing the initial EEG signal based on the average value to obtain the second actual EEG signal of the user.

[0018] Through the above-mentioned technical means, the embodiment of the present application can remove the electrooculographic artifact signals and electromyographic artifact signals in the initial EEG signal, effectively improving the quality of the initial EEG signal. The embodiment of the present application can improve the signal-to-noise ratio of the EEG signal by calculating the average value of all electrode signals in the initial EEG signal and adopting the common average reference spatial filtering technology in the spatial filtering technology, which can make the characteristics of the EEG signal more obvious and help the subsequent interpretation and analysis of the EEG signal.

[0019] Optionally, in one embodiment of the present application, it also includes: a second acquisition module, used to collect the user's historical data and behavior pattern before inputting at least one of the data into the pre-built hybrid model for decoding; an update module, used to update at least one model parameter of the hybrid model based on the historical data and the behavior pattern.

[0020] Through the above-mentioned technical means, the embodiment of the present application can collect the user's historical data and behavior patterns as training data to train a certain hybrid model to update the model parameters of the hybrid model, which can effectively improve the accuracy of the hybrid model in the present application and thereby enhance the application capability of the hybrid model in actual scenarios.

[0021] Optionally, in one embodiment of the present application, it also includes: a generation module, used to generate adjustment parameters of the hybrid model based on the at least one of the above before inputting the at least one of the above into the pre-built hybrid model for decoding; and an adjustment module, used to adjust at least one model parameter of the hybrid model using the adjustment parameters.

[0022] Through the above-mentioned technical means, the embodiment of the present application can realize online transfer learning of the hybrid model based on at least one feature in the user's actual EEG signal. When it is detected that the EEG signal characteristics of the user change significantly when performing a specific task, the model parameters are fine-tuned accordingly by calculating the relationship between the feature change amount and the model parameters to improve the model's decoding accuracy of the user's intention.

[0023] Optionally, in one embodiment of the present application, it also includes: an acquisition module for acquiring execution feedback information of the target robotic arm; a feedback module for generating multimodal instructions based on the execution feedback information, and using the multimodal instructions to provide the user with at least one of visual, auditory, tactile and olfactory biofeedback prompts.

[0024] Through the above-mentioned technical means, the embodiment of the present application can generate execution feedback information based on the execution status of the target robotic arm, so as to generate multimodal feedback prompts to the user according to the execution feedback information of the target robotic arm, including but not limited to biological feedback such as vision, hearing, touch and smell, thereby effectively improving the human-computer interactivity of the present application and enhancing the user's perception of control and control accuracy.

[0025] The third aspect of the present application provides an edge computing terminal, characterized in that it includes: a robotic arm control device based on a brain-computer interface as described in the above embodiment.

[0026] The fourth aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the brain-computer interface-based robotic arm control method as described in the above embodiment.

[0027] The fifth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned brain-computer interface-based robotic arm control method.

[0028] The sixth aspect of the present application provides a computer program product, including a computer program, which, when executed, is used to implement the above-mentioned brain-computer interface-based robotic arm control method.

[0029] The embodiment of the present application can obtain the corresponding control instructions to control the mechanical arm by collecting the user's EEG signal and extracting at least one of the waveform features, power spectrum density features and time-frequency domain features in the EEG signal. Thus, it is realized that through edge computing technology, data processing and preliminary analysis tasks are deployed on the edge computing terminal near the user, data transmission delay is reduced, and the real-time response capability of the BCI system is improved. Through online transfer learning technology, the BCI system can quickly adapt to the real-time EEG signal characteristics of the new user, solve the problem of individual difference adaptability, and effectively improve the decoding accuracy and stability through the hybrid model decoding strategy, combined with deep learning and traditional machine learning technology, and feedback to the user in a multi-modal manner based on the actual control results, effectively improving the human-computer interaction and intelligence level of the present application, thereby increasing the generalization and practical application ability and application scope of the present application. Thus, the BCI system in the related art solves the problem that when converting EEG signals (EEG) into control signals, the stability and accuracy are insufficient when facing complex environments, and there is delay in signal processing, resulting in slow mechanical response and poor adaptive ability, which increases the complexity and cost of the system while limiting the generalization and practical application ability of BCI technology.

[0030] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0032] Figure 1 A flowchart of a method for controlling a robotic arm based on a brain-computer interface according to an embodiment of the present application;

[0033] Figure 2 A schematic diagram of an EEG signal processing process according to an embodiment of the present application;

[0034] Figure 3 A schematic diagram of a hybrid model training and initialization process according to an embodiment of the present application;

[0035] Figure 4 A schematic diagram of a hybrid model online transfer learning and personalized model adjustment process according to an embodiment of the present application;

[0036] Figure 5 A schematic diagram of a robot arm control and real-time feedback process according to an embodiment of the present application;

[0037] Figure 6 A schematic diagram of the structure of a robotic arm control device based on a brain-computer interface according to an embodiment of the present application;

[0038] Figure 7 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application.

[0039] Reference numerals:

[0040] 10-Robotic arm control device based on brain-computer interface: 100-first acquisition module, 200-processing module and 300-control module; 701-memory, 702-processor and 703-communication interface. DETAILED DESCRIPTION

[0041] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0042] The following describes the robot arm control method, device and edge computing terminal based on brain-computer interface of the embodiment of the present application with reference to the accompanying drawings. In view of the fact that the BCI system in the related art mentioned in the above background technology converts the electroencephalogram (EEG) signal into a control signal, the stability and accuracy are insufficient when facing a complex environment, and there is a delay in signal processing, resulting in a slow response of the machine and poor adaptability, which increases the complexity and cost of the system while limiting the generalization and practical application capabilities of the BCI technology. The present application provides a robot arm control method based on brain-computer interface, in which the robot arm can be controlled by collecting the user's electroencephalogram (EEG) signal and extracting at least one of the waveform characteristics, power spectrum density characteristics and time-frequency domain characteristics in the electroencephalogram (EEG) signal to obtain a corresponding control instruction. Thus, it is realized that through edge computing technology, data processing and preliminary analysis tasks are deployed on edge computing terminals near users, data transmission delays are reduced, and the real-time response capability of the BCI system is improved. Through online transfer learning technology, the BCI system can quickly adapt to the real-time EEG signal characteristics of new users, solve the problem of individual difference adaptability, and effectively improve the decoding accuracy and stability through a hybrid model decoding strategy, combined with deep learning and traditional machine learning technology, and feedback is provided to users in a multimodal manner based on actual control results, effectively improving the human-computer interaction and intelligence level of this application, thereby increasing the generalization and practical application capabilities and scope of application of this application. Thus, the problems of insufficient stability and accuracy when the BCI system in the related art converts EEG signals (EEG) into control signals in the face of complex environments are solved, and there is a delay in signal processing, resulting in a slow response of the machine and poor adaptability, which increases the complexity and cost of the system while limiting the generalization and practical application capabilities of BCI technology.

[0043] Before explaining the brain-computer interface-based robotic arm control method in the embodiment of the present application, the equipment, edge computing terminal, robotic arm, computer, etc. involved in the embodiment of the present application are first explained.

[0044] EEG data acquisition device: The embodiment of the present application uses a wearable EEG acquisition device integrated with an EEG sensor, which has multiple electrodes and can accurately collect electrical signals generated by brain activity. The device can transmit real-time data with the edge computing terminal through multiple signal methods such as Bluetooth.

[0045] Edge computing terminal: A portable edge computing device equipped with a high-performance processor and sufficient memory that can run data preprocessing and feature extraction algorithms, as well as some decoding model calculation tasks. The edge computing terminal interacts with the robotic arm control terminal via Wi-Fi.

[0046] Robotic arm: In the embodiment of the present application, an industrial robotic arm with multiple degrees of freedom is selected, and its joints are driven by high-precision motors, which can achieve flexible and precise movement. The robotic arm is equipped with joint position sensors and end force sensors for real-time feedback of the movement state and force conditions of the robotic arm.

[0047] Control-side computer: Install the control-side software, which is responsible for communicating with the edge computing terminal, performing hybrid model decoding, generating robot control signals, processing multimodal feedback, and implementing user interaction functions. The control-side computer communicates with the robot control hardware via a wired Ethernet connection.

[0048] Specifically, Figure 1 A flowchart of a method for controlling a robotic arm based on a brain-computer interface provided in an embodiment of the present application.

[0049] like Figure 1 As shown, the robot arm control method based on brain-computer interface includes the following steps:

[0050] In step S101, the user's initial EEG signal is collected.

[0051] It is understandable to those skilled in the art that, through certain signal processing, EEG signals can be converted into control signals, thereby achieving control over external devices or systems.

[0052] In some embodiments, the present application can collect the user's initial EEG signal, and convert it into a control signal for the robotic arm by performing certain processing on the initial EEG signal, thereby controlling the robotic arm to perform the control action corresponding to the user's EEG signal.

[0053] For example, the present application can use an edge computing terminal with an integrated EEG sensor to collect the user's EEG signals in real time. The integrated EEG sensor can be, but is not limited to, set on an EEG acquisition device, which can have multiple electrodes so as to accurately collect the electrical signals generated by brain activity, and the device can also perform real-time data transmission with the edge computing terminal through a variety of connection methods, such as Bluetooth transmission, wireless signal transmission, etc. It can be expressed in the form of a formula, but is not limited to:

[0054] S(t)=f(EEG)

[0055] Where S(t) represents the EEG signal collected at time t, and f represents the collection function.

[0056] For example, taking the collection of the user's initial EEG signal by a wearable EEG acquisition device integrated with an EEG sensor as an example, multiple subjects can be recruited and asked to wear the EEG data acquisition device, sit in a comfortable chair, and face the robotic arm and display screen. Before starting the experiment, the subjects can be allowed to relax and rest for a few minutes to obtain the user's stable baseline EEG signal.

[0057] Next, the data collection program is started to collect the user's EEG signal in real time. The collection frequency can be set to, but not limited to, 250 Hz, and the collection is continued for a certain time, such as 10 minutes, to obtain sufficient sample data. Also, during the collection process, the subject can be kept in a stable state, and the movement of the head and body can be minimized to avoid excessive artifacts.

[0058] The embodiment of the present application can collect the user's initial EEG signal and use the initial EEG signal as data support in subsequent processing.

[0059] Step S102: pre-processing the initial EEG signal to obtain the actual EEG signal of the user.

[0060] During the actual execution process, the collected initial EEG signals may contain certain interference signals. If the initial EEG signals are used directly to control the robotic arm, the control accuracy of the robotic arm will be greatly reduced. Based on this, the present application can perform certain preprocessing on the initial EEG signals to obtain the actual EEG signals of the user that can be used.

[0061] For example, the present application can use an edge computing terminal to perform certain preprocessing on the initial EEG signal, such as denoising, filtering, etc., so as to remove some unnecessary interference signals in the initial EEG signal to obtain the user's actual EEG signal.

[0062] The embodiment of the present application can pre-process the initial EEG signal so that the actual EEG signal obtained can meet the requirements of subsequent processing, and help reduce the time and cost of subsequent signal processing and improve the accuracy of robotic arm control.

[0063] Optionally, in one embodiment of the present application, Figure 2 As shown, the initial EEG signal is preprocessed to obtain the actual EEG signal of the user, including steps S201 to S202. Including:

[0064] Step S201, removing the electrooculographic artifact signal and the electromyographic artifact signal in the initial EEG signal to obtain a first actual EEG signal of the user; and / or

[0065] Step S203, calculating the average value of all electrodes that collect the initial EEG signal, and processing the initial EEG signal based on the average value to obtain a second actual EEG signal of the user.

[0066] Based on the relevant descriptions of other embodiments, it can be understood that the present application can perform certain preprocessing on the initial EEG signal to obtain the actual EEG signal of the user that can be used in practical applications.

[0067] In the actual implementation process, the present application can first remove the electrooculographic artifact signal and electromyographic artifact signal in the initial EEG signal to obtain the user's first actual EEG signal. Among them, the electrooculographic artifact signal refers to the change in potential difference caused by the movement of the eyeball; the electromyographic artifact signal refers to the change in potential of muscle cells during muscle activity. Since EEG signals are usually collected on the scalp, muscle activity in the head and neck is the main source of EEG and EMG interference.

[0068] For example, after the edge computing terminal receives the collected EEG signal, the present application can immediately apply independent component analysis (ICA) or blind source separation (BSS) technology to remove electrooculographic artifacts and electromyographic artifacts in the EEG signal.

[0069] Furthermore, the embodiment of the present application can also process the initial EEG signal by collecting the average value of all electrodes of the initial EEG signal to obtain a second actual EEG signal of the user.

[0070] For example, the present application can improve the signal-to-noise ratio of the signal by calculating the average value of all electrode signals in the initial EEG signal and using the common average reference (CAR) spatial filtering technology in the spatial filtering technology. The specific process can be expressed as follows:

[0071] First, calculate the average value of all electrodes. The formula can be, but is not limited to, expressed as:

[0072]

[0073] Among them, X i represents the signal of the i-th electrode, and N is the total number of electrodes.

[0074] After calculating the average value of all electrode signals, the embodiment of the present application can use this average value as a new reference point.

[0075] Next, subtract the mean from each electrode signal:

[0076] X new,i =X i -Average,

[0077] For each electrode i, its new signal X new,i is the original signal X i Subtract the average value of all electrodes. This process will be repeated for all electrodes, and finally the EEG signal after average reference processing is obtained, that is, the second actual EEG signal.

[0078] It should be noted that the actual EEG signal in the embodiment of the present application can be a first EEG signal from which electrooculogram artifact signals and electromyography artifact signals have been removed, or a second EEG signal obtained by calculating the average value of all electrode signals and then using the common average reference spatial filtering technology in the spatial filtering technology to improve the signal-to-noise ratio. It can also be an EEG signal obtained by removing electrooculogram artifact signals and electromyography artifact signals and then calculating the average value of all electrode signals and then using the common average reference spatial filtering technology in the spatial filtering technology to improve the signal-to-noise ratio.

[0079] The embodiment of the present application can remove the electrooculographic artifact signals and electromyographic artifact signals in the initial EEG signal, effectively improving the quality of the initial EEG signal. The embodiment of the present application can improve the signal-to-noise ratio of the EEG signal by calculating the average value of all electrode signals in the initial EEG signal and adopting the common average reference spatial filtering technology in the spatial filtering technology, which can make the characteristics of the EEG signal more obvious and facilitate the subsequent interpretation and analysis of the EEG signal.

[0080] Step S103, extract at least one of the waveform characteristics, power spectrum density characteristics and time-frequency domain characteristics of the actual EEG signal, and input at least one of them into a pre-built hybrid model for decoding to output the user's actual intention, so as to send the control instruction corresponding to the actual intention to the control end device corresponding to the target robotic arm.

[0081] It is understandable that the pre-built hybrid model here can be understood as a pre-built hybrid model for decoding the actual EEG signal to obtain the actual intention of the user. The target robotic arm here can be understood as the robotic arm that the user wants to control in the EEG signal.

[0082] As a possible implementation method, the embodiment of the present application can extract certain EEG features from the actual EEG signal and input them into a certain hybrid model for decoding, thereby obtaining the user's actual intention, generating certain control instructions based on the actual intention and sending them to the control end device corresponding to the target robotic arm, thereby using the control end device to control the target robotic arm to perform corresponding actions.

[0083] Among them, the features extracted from the EEG signal in the embodiment of the present application include but are not limited to at least one of the waveform features, power spectrum density features and time-frequency domain features of the EEG signal. After extracting at least one of the waveform features, power spectrum density features and time-frequency domain features of the EEG signal, at least one feature can be input into a certain hybrid model to obtain the user's actual intention and output it.

[0084] By generating certain control instructions based on the actual intention and sending them to the control end of the target robotic arm, the robotic arm control end can control the target robotic arm according to the user's actual intention, thereby completing the generation of the user's actual intention through the user's EEG signals and ultimately completing the control of the target robotic arm.

[0085] Among them, when extracting the waveform features, power spectrum density features and time-frequency domain features of the user's EEG signal in the embodiment of the present application, the extraction can be performed by but is not limited to the following methods:

[0086] Use time domain analysis to extract waveform features: such as peaks, troughs, amplitudes, etc. By setting a certain threshold, the peak and trough points in the signal are detected, and the amplitude difference between adjacent peaks and troughs is calculated as the amplitude feature.

[0087] Use frequency domain analysis to extract power spectral density (PSD) features: Use fast Fourier transform (FFT) to convert time domain signals into frequency domain signals, calculate the power spectral density of different frequency bands, and select the PSD value within a specific frequency range (such as the alpha and beta bands of 8-30Hz) as part of the feature vector.

[0088] Use time-frequency domain analysis to extract wavelet transform coefficients: select a suitable wavelet basis function (such as Daubechies wavelet), perform wavelet decomposition on the EEG signal, obtain wavelet coefficients at different scales, and select coefficients of some key scales as time-frequency domain features.

[0089] And, the hybrid model in the embodiment of the present application is mainly but not limited to two parts: a deep learning model CNN and a traditional machine learning model SVM. By combining the advantages of the deep learning model CNN and the traditional machine learning model SVM, the decoding ability of the EEG signal can be effectively improved.

[0090] F(x)=αF1(x)+(1-α)F2(x)

[0091] Among them, F(x) represents the output of the hybrid model, F1(x) and F2(x) represent the outputs of two different models respectively, and α is the weight coefficient.

[0092] It should be noted that the decoding result output by the hybrid model in the embodiment of the present application includes but is not limited to the fusion result of the decoding results of the deep learning model CNN and the traditional machine learning model SVM, which can effectively improve the accuracy and stability of the decoding result. It can be expressed in the form of a formula but is not limited to:

[0093] F final =βF CNN +(1-β)F SVM

[0094] Among them, F fina1 Represents the final decoded output, F CNN and F SVM They represent the outputs of the CNN and SVM models respectively, and β is the weight coefficient.

[0095] The embodiment of the present application can input at least one of the waveform features, power spectral density features and time-frequency domain features in the user's actual EEG signal into a certain hybrid model to obtain the user's actual intention, and can accurately identify the target robotic arm and its control action from the user's EEG signal, thereby completing the control of the target robotic arm.

[0096] Optionally, in one embodiment of the present application, Figure 3 As shown, before at least one of them is input into the pre-built hybrid model for decoding, it also includes: step S301-step S302.

[0097] Step S301, collecting historical data and behavior patterns of users;

[0098] Step S302: updating at least one model parameter of the hybrid model according to historical data and behavior patterns.

[0099] In some embodiments, before inputting at least one of the waveform features, power spectrum density features, and time-frequency domain features in the user's actual EEG signal into a certain hybrid model for decoding, the hybrid model needs to be trained and initialized. During the training process, at least one model parameter of the hybrid model can be determined and updated based on the user's historical data and behavior patterns, so that the hybrid model can be trained using these model parameters.

[0100] For example, the present application can collect EEG signal data of a certain number (e.g., 50) different subjects and their corresponding robotic arm control intention data (e.g., grabbing, placing, moving, etc.) for training the CNN and SVM models in the hybrid model.

[0101] For the CNN model, the embodiment of the present application can construct a network structure including multiple convolutional layers, pooling layers and fully connected layers. Among them, the size and number of convolution kernels of the convolutional layer are optimized and selected according to the experimental data. For example, a convolution kernel of size 3x3 is used, and different numbers of convolution kernels (such as 32, 64, 128, etc.) are set for each layer to gradually extract more abstract features. The pooling layer can be, but is not limited to, a maximum pooling operation with a step size of 2, thereby reducing the data dimension and simplifying data calculation. The fully connected layer can set the corresponding number of output nodes according to the number of control intent categories (such as 5 types of actions).

[0102] Furthermore, the embodiment of the present application can also use the stochastic gradient descent (SGD) algorithm to train the CNN model, and use the cross entropy loss function as the optimization objective function, so that the learning rate is adjusted according to the accuracy of the validation set during the training process, and the training can be stopped when the accuracy no longer improves.

[0103] For the SVM model, the embodiment of the present application may, but is not limited to, select a radial basis function (RBF) as a kernel function, and determine the optimal kernel function parameters (such as values) and penalty parameters by a cross-validation method. Then, the training data collected in advance is input into the SVM model for training to obtain a classification hyperplane, so that samples of different categories can be separated as much as possible in the feature space.

[0104] Finally, according to the trained CNN and SVM models, the embodiment of the present application can initialize the weight coefficients α and β in the hybrid model. Initially, both α and β can be set to 0.5, and can be dynamically adjusted according to the model performance in the subsequent experimental process.

[0105] For example, by leveraging historical data and behavioral patterns of users, BCI systems can learn and predict the intentions of new users.

[0106] W t+1 =W t +ηΔW L (W t ,x t ,y t )

[0107] Among them, W t represents the model parameters at time t, η represents the learning rate, L represents the loss function, x t and t denote the input and label at time t respectively.

[0108] L(w,x,y)=L source (w,x,y)+γL target (w,x,y)

[0109] Where L(w,x,y) represents the total loss function, L source and L target They represent the loss functions of the source task and the target task respectively, and γ represents the weight for balancing the losses of the two tasks.

[0110] The embodiment of the present application can collect users' historical data and behavior patterns as training data to train a certain hybrid model to update the model parameters of the hybrid model, which can effectively improve the accuracy of the hybrid model in the embodiment of the present application and thereby enhance the application capability of the hybrid model in actual scenarios.

[0111] Optionally, in one embodiment of the present application, Figure 4 As shown, before at least one of them is input into the pre-built hybrid model for decoding, it also includes: step S401-step S402.

[0112] Step S401, generating adjustment parameters of the hybrid model according to at least one of the above;

[0113] Step S402: using the adjustment parameter to adjust at least one model parameter of the hybrid model.

[0114] In other embodiments, before inputting at least one of the waveform features, power spectral density features, and time-frequency domain features in the user's actual EEG signal into a certain hybrid model for decoding, the present application can also generate adjustment parameters of the hybrid model based on at least one of the waveform features, power spectral density features, and time-frequency domain features, thereby using the adjustment parameters to adjust at least one model parameter of the hybrid model, thereby realizing functions such as online transfer learning and personalized model adjustment of the hybrid model.

[0115] The embodiment of the present application can realize online transfer learning and personalized model adjustment of the hybrid model, that is, dynamically adjust the model parameters according to the real-time EEG signal characteristics of the new user to adapt to individual differences. The process can be expressed in the form of a formula, but is not limited to:

[0116] W new =W old +ΔW

[0117] Among them, W new represents the adjusted model parameters, W old represents the original model parameters, and ΔW represents the parameter adjustment amount.

[0118] For example, when a new user starts an experiment, the hybrid model can first use the existing historical data (including the EEG signals and control behavior data of previous users) for online transfer learning. That is, based on the initial EEG signal characteristics of the new user, by calculating the similarity with different users in the historical data, the most similar user data is selected as the source task data, and the new user's current data is used as the target task data.

[0119] Next, according to the online transfer learning formula, the learning rate is initially set to 0.01 and can be dynamically adjusted according to the training effect. L represents the loss function (the total loss function combining the source task and target task losses is used, and γ is initially set to 0.5 to balance the weights of the two task losses). The model parameters are updated so that the model can quickly adapt to the EEG signal characteristics of new users.

[0120] During implementation, the embodiment of the present application can continuously monitor the real-time EEG signal characteristics of new users, dynamically adjust model parameters according to their changes, and calculate the parameter adjustment amount according to the personalized model adjustment formula.

[0121] The embodiment of the present application can realize online transfer learning of the hybrid model based on at least one feature in the user's actual EEG signal. When it is detected that the EEG signal feature of the user changes significantly when performing a specific task, the model parameters are fine-tuned accordingly by calculating the relationship between the feature change and the model parameters to improve the hybrid model's decoding accuracy of the user's intention.

[0122] Optionally, in one embodiment of the present application, Figure 5 As shown, steps S501-S502 are also included.

[0123] Step S501, obtaining execution feedback information of the target robotic arm;

[0124] Step S502, generating a multimodal instruction according to the execution feedback information, and using the multimodal instruction to provide the user with at least one of visual, auditory, tactile and olfactory biofeedback prompts.

[0125] In certain embodiments, after the target robotic arm is controlled based on the user's EEG signal, in order to determine whether the target robotic arm has executed the control instruction corresponding to the user's EEG signal, the present application can also obtain the execution feedback information of the target robotic arm and generate multimodal instructions, so as to use the multimodal instructions to provide the user with at least one biofeedback prompt among vision, hearing, touch and smell.

[0126] For example, in terms of vision, the present application can provide feedback to the user by displaying the movement trajectory and current status of the robotic arm in real time on the display screen, as well as prompt information on successful or failed grasping; in terms of hearing, the present application can issue a prompt sound when the robotic arm starts to move, and issue an alarm sound when an abnormal situation (such as a collision) occurs, giving the user feedback through prompt sounds and alarm sounds; in terms of touch, the present application can inform the user of the force condition of the robotic arm through the vibration feedback of the control handle, such as generating appropriate vibrations when grasping objects, and changing the vibration intensity when the grasping force is too large or too small, thereby enhancing the user's perception of the control of the robotic arm and the accuracy of operation; in terms of smell, the present application can release an unpleasant odor when a control error occurs in the robotic arm, thereby increasing the subject's physiological arousal level and correcting its control error.

[0127] The embodiment of the present application can generate execution feedback information based on the execution status of the target robotic arm, so as to generate multimodal feedback prompts to the user according to the execution feedback information of the target robotic arm, including but not limited to biological feedback such as vision, hearing, touch and smell, thereby effectively improving the human-computer interactivity of the present application and enhancing the user's perception of control and control accuracy.

[0128] The present application is explained in detail below with reference to a specific embodiment.

[0129] (1) The subjects watch the prompt information on the display screen and imagine the corresponding robot arm movements in their minds (such as grabbing the object in front), which generates certain electroencephalogram (EEG) signals.

[0130] (2) The edge computing terminal preprocesses and extracts features from the collected real-time EEG signals and transmits the feature data to the control computer.

[0131] (3) The hybrid model in the control computer decodes the input feature data. The feature data is input into the CNN and SVM models respectively to obtain their respective outputs, and then fused according to the preset weight coefficients to obtain the final decoding result, that is, the user's control intention. For example, if the grasping action probability output by the CNN model is 0.7 and the grasping action probability output by the SVM model is 0.8, when α = 0.4 and β = 0.6, the final grasping action probability is 0.4 × 0.7 + 0.6 × 0.8 = 0.76.

[0132] (4) Based on the decoding result, the control end computer generates the corresponding robot arm control signal, controls the joint movement of the robot arm through the motor drive circuit, and realizes the action execution of the robot arm. For example, if the decoding result is a grasping action, the end effector of the robot arm is controlled to open, move to the target object position, and then close to grasp the object.

[0133] (5) During the movement of the robot arm, feedback information from the joint position sensor and the end force sensor is obtained in real time. The actual motion trajectory and force of the robot arm are compared with the expected motion trajectory and action force of the decoding result, and the error signal is calculated. If the error exceeds a certain threshold (such as the position error exceeds 5 mm or the force error exceeds 1 N), the online optimization mechanism of the model is triggered.

[0134] (6) The control-end computer uses an adaptive learning rate adjustment strategy to fine-tune the parameters of the CNN and SVM parts of the hybrid model based on the error signal. For example, if the position error is large, increase the learning rate and adjust the model quickly; if the force error is small, reduce the learning rate and make fine adjustments. At the same time, based on the new sample data (current EEG feature data and robotic arm feedback data), use the back-propagation algorithm to update the model parameters to improve the model's decoding accuracy of the user's actual intention and the control accuracy of the robotic arm.

[0135] (7) Provide multimodal feedback to the subject while the robotic arm is performing actions. Visually, the motion trajectory and current status of the robotic arm are displayed in real time on the display screen, as well as prompt information on success or failure of grasping; auditorily, a prompt sound is emitted when the robotic arm starts to move, and an alarm sound is emitted when an abnormal situation (such as a collision) occurs; tactilely, the vibration feedback of the control handle informs the user of the force applied to the robotic arm, such as generating appropriate vibration when grasping an object, and changing the vibration intensity when the grasping force is too large or too small, thereby enhancing the user's perception of the control of the robotic arm and the accuracy of the operation; olfactorily, when the robotic arm makes a control error, an unpleasant odor is released to increase the subject's physiological arousal level and correct its control error.

[0136] According to the robot arm control method based on brain-computer interface proposed in the embodiment of the present application, the robot arm can be controlled by collecting the user's EEG signal and extracting at least one of the waveform characteristics, power spectrum density characteristics and time-frequency domain characteristics in the EEG signal to obtain the corresponding control instruction. Thus, it is achieved that the data processing and preliminary analysis tasks are deployed on the edge computing terminal near the user through edge computing technology, reducing data transmission delays and improving the real-time response capability of the BCI system. Through online transfer learning technology, the BCI system can quickly adapt to the real-time EEG signal characteristics of new users to solve the problem of individual difference adaptability. Through the hybrid model decoding strategy, combined with deep learning and traditional machine learning technology, the decoding accuracy and stability are effectively improved, and the user is fed back in a multimodal manner based on the actual control results, which effectively improves the human-computer interactivity and intelligence level of the present application, thereby increasing the generalization and practical application capabilities and application scope of the present application. This solves the problem that the BCI system in the related technology lacks stability and accuracy when converting electroencephalogram (EEG) signals into control signals in complex environments, and that there is a delay in signal processing, resulting in slow mechanical response and poor adaptability, which increases the complexity and cost of the system while limiting the generalization and practical application capabilities of BCI technology.

[0137] Next, a brain-computer interface-based robotic arm control device proposed according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0138] Figure 6 It is a structural schematic diagram of a robotic arm control device based on a brain-computer interface according to an embodiment of the present application.

[0139] like Figure 6 As shown, the brain-computer interface-based robotic arm control device 10 includes: a first acquisition module 100 , a processing module 200 and a control module 300 .

[0140] The first acquisition module 100 is used to acquire the initial EEG signal of the user;

[0141] The processing module 200 is used to pre-process the initial EEG signal to obtain the actual EEG signal of the user;

[0142] The control module 300 is used to extract at least one of the waveform characteristics, power spectrum density characteristics and time-frequency domain characteristics of the actual EEG signal, and input at least one of them into a pre-built hybrid model for decoding to output the user's actual intention, so as to send the control instruction corresponding to the actual intention to the control end device corresponding to the target robotic arm.

[0143] Optionally, in one embodiment of the present application, the processing module 200 includes: a removal unit and a calculation unit.

[0144] The removing unit is used to remove the electrooculographic artifact signal and the electromyographic artifact signal in the initial electroencephalogram signal to obtain the first actual electroencephalogram signal of the user; and / or

[0145] The calculation unit is used to calculate the average value of all electrodes that collect the initial EEG signal, and process the initial EEG signal based on the average value to obtain a second actual EEG signal of the user.

[0146] Optionally, in one embodiment of the present application, it also includes: a second acquisition module and an update module.

[0147] wherein the second collection module is used to collect historical data and behavior patterns of users before inputting at least one of them into a pre-built hybrid model for decoding;

[0148] An updating module is used to update at least one model parameter of the hybrid model according to historical data and behavior patterns.

[0149] Optionally, in one embodiment of the present application, it also includes: a generation module and an adjustment module.

[0150] wherein the generating module is used to generate adjustment parameters of the hybrid model according to at least one of the at least one of the inputs to the pre-built hybrid model for decoding;

[0151] The adjustment module is used to adjust at least one model parameter of the hybrid model using the adjustment parameter.

[0152] Optionally, in one embodiment of the present application, it also includes: an acquisition module and a feedback module.

[0153] Wherein, the acquisition module is used to obtain the execution feedback information of the target robotic arm;

[0154] The feedback module is used to generate multimodal instructions according to the execution feedback information, and use the multimodal instructions to provide the user with at least one of visual, auditory, tactile and olfactory biofeedback prompts.

[0155] It should be noted that the aforementioned explanation of the embodiment of the brain-computer interface-based robotic arm control method is also applicable to the brain-computer interface-based robotic arm control device of this embodiment, and will not be repeated here.

[0156] According to the robot arm control device based on brain-computer interface proposed in the embodiment of the present application, the robot arm can be controlled by collecting the user's EEG signal and extracting at least one of the waveform characteristics, power spectrum density characteristics and time-frequency domain characteristics in the EEG signal to obtain the corresponding control instructions. Thus, it is achieved that the data processing and preliminary analysis tasks are deployed on the edge computing terminal near the user through edge computing technology, reducing data transmission delays and improving the real-time response capability of the BCI system. Through online transfer learning technology, the BCI system can quickly adapt to the real-time EEG signal characteristics of new users to solve the problem of individual difference adaptability. Through the hybrid model decoding strategy, combined with deep learning and traditional machine learning technology, the decoding accuracy and stability are effectively improved, and the user is fed back in a multimodal manner based on the actual control results, which effectively improves the human-computer interactivity and intelligence level of the present application, thereby increasing the generalization and practical application capabilities and application scope of the present application. This solves the problem that the BCI system in the related technology lacks stability and accuracy when converting electroencephalogram (EEG) signals into control signals in complex environments, and that there is a delay in signal processing, resulting in slow mechanical response and poor adaptability, which increases the complexity and cost of the system while limiting the generalization and practical application capabilities of BCI technology.

[0157] An embodiment of the present application also provides an edge computing terminal, characterized in that it includes: a robotic arm control device based on a brain-computer interface as described in the above embodiment.

[0158] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0159] A memory 701 , a processor 702 , and a computer program stored in the memory 701 and executable on the processor 702 .

[0160] When the processor 702 executes the program, the brain-computer interface-based robotic arm control method provided in the above embodiment is implemented.

[0161] Furthermore, the electronic device further comprises:

[0162] The communication interface 703 is used for communication between the memory 701 and the processor 702 .

[0163] The memory 701 is used to store computer programs that can be executed on the processor 702 .

[0164] The memory 701 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0165] If the memory 701, the processor 702 and the communication interface 703 are implemented independently, the communication interface 703, the memory 701 and the processor 702 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0166] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0167] The processor 702 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0168] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned brain-computer interface-based robotic arm control method is implemented.

[0169] An embodiment of the present application also provides a computer program product, including a computer program, which can run computer instructions. When the computer instructions are executed by a processor, the brain-computer interface-based robotic arm control method provided in the embodiment of the present application is implemented.

[0170] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0171] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0172] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0173] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0174] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0175] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0176] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0177] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for controlling a robotic arm based on a brain-computer interface, characterized in that: The following steps are involved: Collecting the user's initial EEG signal; Preprocessing the initial EEG signal to obtain an actual EEG signal of the user; Extract at least one of the waveform features, power spectral density features and time-frequency domain features of the actual EEG signal, and input the at least one of them into a pre-built hybrid model for decoding to output the actual intention of the user, so as to send the control instruction corresponding to the actual intention to the control end device corresponding to the target robotic arm.

2. The method according to claim 1, characterized in that The preprocessing of the initial EEG signal to obtain the actual EEG signal of the user includes: removing electrooculographic artifact signals and electromyographic artifact signals from the initial electroencephalogram signal to obtain a first actual electroencephalogram signal of the user; and / or The average value of all electrodes that collect the initial EEG signal is calculated, and the initial EEG signal is processed based on the average value to obtain a second actual EEG signal of the user.

3. The method according to claim 1, characterized in that Before inputting the at least one of the above into the pre-built hybrid model for decoding, the method further comprises: Collecting historical data and behavior patterns of the user; At least one model parameter of the hybrid model is updated according to the historical data and the behavior pattern.

4. The method according to claim 1, characterized in that: Before inputting the at least one of the above into the pre-built hybrid model for decoding, the method further comprises: generating adjustment parameters of the hybrid model according to the at least one of the above; At least one model parameter of the hybrid model is adjusted using the adjustment parameter.

5. The method according to claim 1, characterized in that: Also includes: Obtaining execution feedback information of the target robotic arm; A multimodal instruction is generated according to the execution feedback information, and the multimodal instruction is used to provide the user with at least one of visual, auditory, tactile and olfactory biofeedback prompts.

6. A robot arm control device based on brain-computer interface, characterized in that: include: A collection module, used to collect the user's initial EEG signal; A processing module, used for preprocessing the initial EEG signal to obtain the actual EEG signal of the user; A control module is used to extract at least one of the waveform characteristics, power spectrum density characteristics and time-frequency domain characteristics of the actual EEG signal, and input the at least one of them into a pre-built hybrid model for decoding, so as to output the actual intention of the user, and send the control instruction corresponding to the actual intention to the control end device corresponding to the target robotic arm.

7. An edge computing terminal, characterized in that: include: A robotic arm control device based on a brain-computer interface as described in claim 6.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the brain-computer interface-based robotic arm control method as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the brain-computer interface-based robotic arm control method as described in any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed, it is used to implement the brain-computer interface-based robotic arm control method as described in any one of claims 1-5.

Images