Man-machine interaction system and method based on electroencephalogram and electromyographic signals

Abstract

The invention relates to the technical field of electroencephalogram and electromyographic signal interaction, and discloses an electroencephalogram and electromyographic signal-based man-machine interaction system and method, and the system comprises an electroencephalogram and electromyographic signal collection device which is used for collecting an electroencephalogram signal and an electromyographic signal, carrying out the signal preprocessing, and outputting a preprocessed mixed signal; the host is used for carrying out composite noise suppression on the preprocessed mixed signal to obtain an optimized mixed signal and storing the optimized mixed signal; the PC terminal is used for carrying out anomaly detection on the optimized mixed signal, generating an electroencephalogram and myoelectricity detection report containing an anomaly detection result of the target user, and outputting and displaying the electroencephalogram and myoelectricity detection report; and the mobile phone end is used for receiving the electroencephalogram and myoelectricity detection information of the target user in a wireless communication mode, controlling the on-off state of the electroencephalogram and myoelectricity acquisition process and performing abnormal marking on the optimized mixed signal. According to the method, the accuracy and stability of electroencephalogram and electromyogram signal acquisition can be improved, noise can be effectively removed, and the high-precision man-machine interaction requirement can be comprehensively met.

Description

Technical Field

[0001] The present invention relates to the technical field of brain-myoelectric signal interaction, and in particular to a human-computer interaction system and method based on brain-myoelectric signal. Background Art

[0002] Existing EMG and EMG signal acquisition and interaction systems have numerous shortcomings. For one thing, the accuracy of EMG and EMG signal acquisition is limited; EMG signals are weak and susceptible to interference; and the stability of lead wire connections during EMG signal acquisition is poor, affecting data accuracy. Furthermore, the human-computer interaction method for EMG signals needs improvement, and real-time anomaly detection and output display are inadequate, making it difficult to meet the requirements of high-precision human-computer interaction.

[0003] Therefore, it is urgent to provide a technical solution to solve the above problems. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a human-computer interaction system and method based on brain and myoelectric signals.

[0005] In a first aspect, the present invention provides a human-computer interaction system based on brain and myoelectric signals, the technical solution of the system is as follows:

[0006] A human-computer interaction system based on brain and myoelectric signals includes: a brain and myoelectric acquisition device, a host, a PC terminal and a mobile phone terminal;

[0007] The electroencephalogram and myoelectricity acquisition device is used to synchronously acquire the original electroencephalogram (EEG) signals and the original myoelectricity (EMG) signals of the target user, perform signal preprocessing, and output the preprocessed mixed signals;

[0008] The host is used to perform composite noise suppression on the pre-processed mixed signal to obtain an optimized mixed signal and store it;

[0009] The PC is used to perform abnormality detection on the optimized mixed signal, generate an electroencephalogram and myoelectricity detection report of the target user including the abnormality detection result, and output the report for display;

[0010] The mobile phone terminal is used to receive the brain and myoelectricity detection information of the target user, control the on and off state of the brain and myoelectricity acquisition process, and mark the optimized mixed signal as abnormal through wireless communication.

[0011] The beneficial effects of the human-computer interaction system based on brain and myoelectric signals of the present invention are as follows:

[0012] The system of the present invention can solve the problems of limited accuracy in brain and myoelectric signal acquisition and imperfect signal processing algorithms, improve the accuracy and stability of signal acquisition, effectively remove noise, comprehensively extract features, and meet the needs of high-precision human-computer interaction.

[0013] On the basis of the above solution, the human-computer interaction system based on brain and myoelectric signals of the present invention can be further improved as follows.

[0014] In an optional manner, the brain and myoelectricity acquisition device includes: a multi-channel high-precision sensor array and a lead wire stabilization structure; the multi-channel high-precision sensor array is integrated with a differential amplifier circuit and an embedded pre-filter;

[0015] The multi-channel high-precision sensor array is used to synchronously collect the original EEG signal and the original EMG signal in different channels, and perform common-mode noise suppression and gain adjustment on the original EEG signal and the original EMG signal through the differential amplifier circuit, and then perform frequency band pre-filtering through the embedded pre-filter to output the pre-processed mixed signal;

[0016] The lead wire stabilizing structure is used to physically reduce noise on the sensor electrodes of the multi-channel high-precision sensor array through an elastic contact electrode fixing component.

[0017] In an optional manner, the host includes: a multi-stage adaptive filtering module and a hybrid feature extraction module;

[0018] The multi-stage adaptive filtering module is used to perform composite noise suppression on the pre-processed mixed signal through a cascaded power frequency notch filter, a wavelet packet denoising unit and an independent component analysis unit to obtain the optimized mixed signal;

[0019] The hybrid feature extraction module is used to perform a joint time-frequency domain analysis on the optimized hybrid signal, extract the rhythm energy features corresponding to the EEG signal and the AR model coefficient features corresponding to the EMG signal, and generate a fusion feature vector.

[0020] In an optional manner, the PC terminal includes: an anomaly detection unit, a deep neural network unit and a support vector machine unit;

[0021] The anomaly detection unit is used to perform anomaly detection on the optimized mixed signal, generate an electroencephalogram and myoelectricity detection report of the target user including the anomaly detection result, and output the report for display;

[0022] The deep neural network unit is configured as a fully connected network having an input layer, two hidden layers, and an output layer, wherein the number of neurons in the input layer is consistent with the dimension of the fused feature vector, and is configured to perform nonlinear mapping on the fused feature vector received by the input layer based on two hidden layers using a ReLU activation function, and output high-dimensional abstract features through the output layer;

[0023] The support vector machine unit is configured as a multi-classifier based on a radial basis kernel function, which is used to receive the high-dimensional abstract features and output a classification probability distribution, and generate an interaction intention instruction according to the classification probability distribution and a preset threshold.

[0024] In an optional manner, the electroencephalogram (EBM) detection information includes: EBM detection time, detection status, lead wire connection status, and wireless communication status.

[0025] In an optional manner, the mobile phone is used to receive the interaction intention instruction through wireless transmission, and generate a control signal according to the interaction intention instruction to drive an external device; wherein, the wireless transmission method adopts a dual-mode redundant transmission mechanism; the dual-mode redundant transmission mechanism includes: alternating transmission channels of Bluetooth 5.0 and WiFi6, and automatically switches to the backup channel when it is detected that the packet loss rate of the current transmission channel exceeds a threshold.

[0026] In an optional manner, the mobile phone terminal includes: a data packet checking unit, a signal encoding unit and a driving interface unit;

[0027] The data packet verification unit is used to verify the integrity of the interaction intention instruction using a cyclic redundancy check and retransmission request protocol;

[0028] The signal encoding unit is used to convert the interaction intention instruction into a PWM waveform signal or a standard serial port instruction;

[0029] The driving interface unit is used to determine a hardware enable signal according to the type of the external device, and superimpose the hardware enable signal with the PWM waveform signal or the standard serial port instruction to generate the control signal to drive the external device.

[0030] In an optional manner, the mobile phone is further used to:

[0031] The motion trajectory data is collected by the motion sensor, and the voice data collected by the voice input module is obtained. The interaction intention instruction, the motion trajectory data and the voice data are aligned in time and space to generate a multi-channel control signal to drive the external device.

[0032] In an optional manner, the mobile phone is further used to:

[0033] Before generating the multi-channel control signal, a logical conflict detection is performed on the interaction intention instruction, the motion trajectory data and the voice data. If a conflict is detected, a secondary confirmation process of the user is triggered.

[0034] In a second aspect, the present invention provides a human-computer interaction method based on brain and myoelectric signals, using a human-computer interaction system based on brain and myoelectric signals as described in the present invention. The technical solution of this method is as follows:

[0035] The EEG and myoelectric acquisition device is used to synchronously acquire the original EEG signals and original EMG signals of the target user, perform signal preprocessing, and output the preprocessed mixed signals;

[0036] The host is used to perform composite noise suppression on the pre-processed mixed signal to obtain an optimized mixed signal and store it;

[0037] The PC terminal is used to perform anomaly detection on the optimized mixed signal, generate an EMG detection report of the target user including the anomaly detection result, and output the report for display;

[0038] The mobile phone is used to receive the target user's electroencephalogram and myoelectricity detection information, control the on / off state of the electroencephalogram and myoelectricity acquisition process, and mark the optimized mixed signal as abnormal through wireless communication.

[0039] The beneficial effects of the human-computer interaction method based on brain and myoelectric signals of the present invention are as follows:

[0040] The method of the present invention can solve the problems of limited accuracy in brain and myoelectric signal acquisition and imperfect signal processing algorithms, improve the accuracy and stability of signal acquisition, effectively remove noise, comprehensively extract features, and meet the needs of high-precision human-computer interaction.

[0041] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the drawings. In the drawings:

[0043] Figure 1 This is a structural block diagram of an embodiment of a human-computer interaction system based on brain and myoelectric signals of the present invention;

[0044] Figure 2 This is the structural block diagram of the brain and myoelectricity acquisition device;

[0045] Figure 3 This is the structural block diagram of the host;

[0046] Figure 4 This is the structural diagram of the PC side;

[0047] Figure 5 This is the structural diagram of the mobile phone terminal;

[0048] Figure 6 The figure is a flow chart of an embodiment of a human-computer interaction method based on brain and myoelectric signals of the present invention. DETAILED DESCRIPTION

[0049] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0050] Figure 1 FIG. 1 shows a schematic diagram of a human-computer interaction system based on brain and myoelectric signals according to an embodiment of the present invention. Figure 1 As shown, the system includes: a brain and myoelectricity acquisition device 1, a host 2, a PC terminal 3 and a mobile phone terminal 4;

[0051] The electroencephalogram and myoelectricity acquisition device 1 is used to synchronously acquire the original electroencephalogram (EEG) signals and original myoelectricity (EMG) signals of a target user, perform signal preprocessing, and output the preprocessed mixed signals.

[0052] Raw EEG signals refer to unprocessed neural electrical activity signals collected directly via scalp electrodes, with a frequency range of 0.5-100 Hz and an amplitude of 1-100 μV, reflecting changes in the electrical potential of neuronal groups in the cerebral cortex. Raw EMG signals refer to unprocessed muscle fiber electrical activity signals collected directly via surface electrodes, with a frequency range of 20-500 Hz and an amplitude of 0.1-5 mV, characterizing the strength and timing of muscle contraction. Preprocessed mixed signals are fused signals obtained by differentially amplifying and band-prefiltering the raw EEG / EMG signals, retaining effective physiological components (0.5-100 Hz for EEG and 20-500 Hz for EMG) and maintaining a signal-to-noise ratio >20 dB.

[0053] The host 2 is used to perform composite noise suppression on the pre-processed mixed signal to obtain an optimized mixed signal and store it.

[0054] The host computer 2 is connected to the EEG and myoelectric acquisition device 1 via a lead cable, comprising 14 electrodes (8 EEG leads, 2 EMG leads, and 4 reference leads). All 14 electrodes are connected via an HDMI interface. The optimized mixed signal is the output signal after the preprocessed mixed signal undergoes power frequency notching, wavelet packet denoising, and ICA processing. The optimized mixed signal has a power frequency interference suppression ratio >40dB and an artifact elimination rate >95%.

[0055] The PC terminal 3 is used to perform abnormality detection on the optimized mixed signal, generate a brain electromyography detection report of the target user including the abnormality detection result, and output it for display.

[0056] Anomaly detection refers to the detection of abnormal waveforms within the optimized mixed signal. The EMG test report includes the target user's basic information (medical record number, name, gender, ID number, phone number, and test start and end times). The PC terminal 3 provides buttons for canceling, saving, and saving and printing the EMG test report. Once generated, the EMG test report is saved as an image, which can be downloaded.

[0057] It should be noted that PC terminal 3 can manage brain and myoelectric test reports. Specific functions include but are not limited to: ① The page pop-up window is displayed in the center. ② The list displays patient information (list, medical record number, name, gender, ID number, telephone number, report time), and the operation column prints, browses, and downloads. ③ Query conditions, medical record number, name, ID number, telephone number, report time (start and end). ④ The list adds a paging function. ⑤ Printing, browsing, and pictures must have the report time and doctor's signature filled in.

[0058] The mobile phone terminal 4 is used to receive the brain and myoelectricity detection information of the target user, control the on and off state of the brain and myoelectricity acquisition process, and mark the optimized mixed signal as abnormal through wireless communication.

[0059] The default wireless communication method is Bluetooth, but this can be adjusted based on actual conditions and is not limited here. EMG testing information includes: EMG testing time (duration), testing status (testing, paused, or stopped), lead wire connection status (normal or disconnected), and wireless communication status (whether the Bluetooth connection is normal).

[0060] It should be noted that a start / pause button is provided on the mobile phone terminal 4 for switching the detection status. Specifically: when the wireless communication status is normal, if the detection status is paused, the target user can switch the detection status to detecting by clicking the start / pause button; if the detection status is detecting, the target user can switch the detection status to pause by clicking the start / pause button. In addition, during the EMG detection process, if the target user finds abnormal information, the mark button in the App of the mobile phone terminal 4 can be used to quickly mark the abnormality, helping the target user to capture and record important abnormal situations in a timely manner, facilitating further inspection and evaluation by medical personnel, thereby improving the overall accuracy and effectiveness of the detection.

[0061] In an alternative approach, such as Figure 2 As shown, the brain and myoelectricity acquisition device 1 includes: a multi-channel high-precision sensor array 11 and a lead wire stabilization structure 12; the multi-channel high-precision sensor array 11 integrates a differential amplifier circuit 111 and an embedded pre-filter 112.

[0062] The multi-channel high-precision sensor array 11 is used to synchronously collect the original EEG signals and the original EMG signals in different channels, and perform common-mode noise suppression and gain adjustment on the original EEG signals and the original EMG signals through the differential amplifier circuit 111, and then perform frequency band pre-filtering through the embedded pre-filter 112 to output the pre-processed mixed signal.

[0063] The multi-channel high-precision sensor array 11 includes an EEG signal channel and an EMG signal channel. The EEG signal channel uses Ag / AgCl electrodes with an input impedance of >100MΩ and collects raw EEG signals S in the 0.5-100Hz frequency band. EEG (t). The electromyographic signal acquisition channel uses stainless steel electrodes with an input impedance of <10kΩ, and collects the original electromyographic signal S in the 20-500Hz frequency band. EMG (t).

[0064] The differential amplifier circuit 111 is used to: EEG (t) and the original electromyographic signal S EMG (t) Gain is applied to perform differential amplification processing, and the original EEG signal S EEG (t) Amplify 1000 times, common mode rejection ratio ≥ 120dB, output amplified EEG signal V EEG (t); original electromyographic signal S EMG (t) Amplify 100 times, common mode rejection ratio ≥ 90dB, output amplified electromyographic signal V EMG (t).

[0065] Among them, the embedded pre-filter is used to: amplify the EEG signal V EEG (t) Perform bandpass filtering (0.5Hz high pass + 100Hz low pass) and output the filtered EEG signal F EEG (t); amplify the electromyographic signal V EMG (t) Perform bandpass filtering (20Hz high pass + 500Hz low pass) and output the filtered EMG signal F EEG (t).

[0066] Among them, the preprocessed mixed signal refers to the filtered EEG signal F EEG (t) and the filtered EMG signal F EEG The preprocessed mixed signal X(t) generated by merging the signals (t) is an N×T matrix, where N represents the number of channels of the multi-channel high-precision sensor array 11, and T represents the number of sampling points (the number of discrete signal samples of a single channel within a time window). It should be noted that the multi-channel high-precision sensor array 11 in this embodiment includes 8 EEG channels and 2 EMG channels by default. The 8 EEG channels are connected to the scalp of the target user, and the 2 EMG channels are connected to the arms or legs of the target user.

[0067] The lead wire stabilizing structure 12 is used to physically reduce noise on the sensor electrodes of the multi-channel high-precision sensor array 11 through an elastic contact electrode fixing component.

[0068] Among them, the lead wire stabilization structure 12 refers to the elastic silicone electrode fixing component, which stabilizes the electrode-skin contact impedance to below 5kΩ (traditional devices are >20kΩ). The physical noise reduction process of the lead wire stabilization structure 12 is as follows: ① Electrode fixation: The Ag / AgCl EEG electrode is embedded in the silicone fixing ring and a constant pressure of 5N is applied to fit the scalp; the stainless steel EMG electrode is fixed to the muscle belly through an elastic strap, with a contact pressure of 3N±0.5N. ② Impedance stabilization: A 10Hz / 1mA AC detection current is injected to monitor the electrode-skin impedance in real time; if the impedance is >5kΩ, the micro-vibration motor (amplitude 0.1mm) is automatically started to reduce the contact resistance. ③ Motion artifact suppression: The elastic material absorbs mechanical vibration energy with a frequency >2Hz, reducing the amplitude of motion artifacts by 60%.

[0069] In an alternative approach, such as Figure 3 As shown, the host 2 includes: a multi-stage adaptive filtering module 21 and a hybrid feature extraction module 22.

[0070] The multi-stage adaptive filtering module 21 is used to perform composite noise suppression on the pre-processed mixed signal through a cascaded power frequency notch filter, a wavelet packet denoising unit and an independent component analysis unit to obtain the optimized mixed signal.

[0071] The power frequency notch filter applies a 50Hz notch filter (Q=30) to the preprocessed mixed signal X(t), eliminating mains interference and outputting a first mixed signal X1(t). The wavelet packet denoising unit performs a four-layer wavelet packet decomposition on the first mixed signal X1(t), reconstructs the signal, and outputs a second mixed signal X2(t). The independent component analysis unit performs the FastICA algorithm on the second mixed signal X2(t), separating the physiological signal from the artifact component, and outputting the optimized mixed signal Y(t).

[0072] The hybrid feature extraction module 22 is used to perform a joint time-frequency domain analysis on the optimized hybrid signal, extract the rhythm energy features corresponding to the EEG signal and the AR model coefficient features corresponding to the EMG signal, and generate a fusion feature vector.

[0073] The rhythm energy feature refers to the wavelet energy value of the EEG signal in a specific frequency band (8-13 Hz for alpha rhythm and 13-30 Hz for beta rhythm). The AR model coefficient feature refers to the coefficient vector of the fourth-order autoregressive model of the EMG signal, obtained by minimizing the prediction error. The fusion feature vector is an 8-dimensional feature vector formed by concatenating the rhythm energy feature (2D), the AR coefficient (4D), and the EMG coupling feature (2D).

[0074] Among them, the rhythm energy feature includes the EEG α rhythm energy feature and the EEG β rhythm energy feature. Specifically, according to the signal channel number, the optimized EEG signal Y is separated from the optimized mixed signal Y(t) to obtain EEG (t) and optimized electromyographic signal Y EMG (t); for optimizing EEG signal Y EEG (t) Perform Morlet wavelet transform to calculate the energy features of α rhythm (8-13Hz) and β rhythm (13-30Hz); construct an autoregressive (AR) model and solve to obtain the AR model coefficient features (the default is the fourth-order coefficients a1, a2, a3, a4); calculate the mutual information between the EEG α rhythm energy feature and the optimized EMG signal in the time-frequency domain to obtain the first coupling feature, and calculate the mutual information between the EEG β rhythm energy feature and the optimized EMG signal in the time-frequency domain to obtain the second coupling feature; splice the EEG α rhythm energy feature, the EEG β rhythm energy feature, the AR model coefficient feature, the first coupling feature and the second coupling feature to output the fusion feature vector F.

[0075] In an alternative approach, such as Figure 4 As shown, the PC terminal 3 includes: an anomaly detection unit 31, a deep neural network unit 32 and a support vector machine unit 33;

[0076] The anomaly detection unit 31 is used to perform anomaly detection on the optimized mixed signal, generate an EMG detection report of the target user including the anomaly detection result, and output it for display;

[0077] The deep neural network unit 32 is configured as a fully connected network having an input layer, two hidden layers and an output layer. The number of neurons in the input layer is consistent with the dimension of the fused feature vector. It is used to perform nonlinear mapping on the fused feature vector received by the input layer based on two hidden layers using a ReLU activation function, and output high-dimensional abstract features through the output layer.

[0078] The deep neural network unit 32 includes an input layer, a first hidden layer, a second hidden layer, and an output layer. The input layer receives the fused feature vector F, performs linear mapping on the fused feature vector F to obtain a first feature, and outputs it to the first hidden layer. The first hidden layer performs weight calculation on the first feature and activates it using the ReLU activation function to obtain a second feature, and outputs it to the second hidden layer. The second hidden layer performs weight calculation on the second feature and activates it using the ReLU activation function to obtain a third feature, and outputs it to the output layer. The output layer performs linear mapping on the third feature and outputs a high-dimensional abstract feature O.

[0079] The support vector machine unit 33 is configured as a multi-classifier based on a radial basis kernel function, which is used to receive the high-dimensional abstract features and output a classification probability distribution, and generate the interaction intention instruction according to the classification probability distribution and a preset threshold.

[0080] Among them: ① The high-dimensional abstract feature O is mapped to the reproducing kernel Hilbert space K(O,O) through the radial basis kernel function (RBF) i )=exp(-γ‖OO i ‖ 2 ), O i represents the i-th support vector, γ=0.05;②Calculate the multi-classification decision function M represents the number of support vectors, α i represents the Lagrange multiplier, y i ∈[-1,1] is the category label; ③ Use Platt scaling to convert g(O) into a probability distribution: max c P(y=c|O) represents the probability distribution of category c A and B represent the maximum likelihood estimation parameters; ④ If max c If P(y=c|O)>0.9, the corresponding intention instruction is output.

[0081] In an optional manner, the mobile phone terminal 4 is used to receive the interaction intention instruction via wireless transmission, and generate a control signal to drive the external device according to the interaction intention instruction.

[0082] The control signal consists of a PWM waveform (duty cycle 5%-95%) or a standard serial port command superimposed on an enable signal (3.3V / 5V level). The external device refers to the controlled terminal device (such as an intelligent prosthesis, wheelchair, or household appliance), and the interface is compatible with RS-485 / CAN bus or PWM drive.

[0083] Among them, the wireless transmission method adopts a dual-mode redundant transmission mechanism. The dual-mode redundant transmission mechanism includes: alternating transmission channels of Bluetooth 5.0 and WiFi6, and automatically switching to the backup channel when it is detected that the packet loss rate of the current transmission channel exceeds the threshold. The threshold corresponding to the packet loss rate defaults to 5%, and can also be adjusted according to actual conditions. Specifically: the packet loss rate of the current channel (Bluetooth / WiFi) is calculated every 50ms. If the packet loss rate is greater than 5% and lasts for three cycles, the channel is determined to be failed and switched to the backup channel in the next 10ms time slot. In addition, the last 5 data packets are cached before switching, and are given priority for retransmission after the new channel is established.

[0084] In an alternative approach, such as Figure 5 As shown, the mobile phone terminal 4 includes: a data packet checking unit 41, a signal encoding unit 42 and a driving interface unit 43.

[0085] The data packet verification unit 41 is used to verify the integrity of the interaction intention instruction using a cyclic redundancy check and retransmission request protocol.

[0086] The CRC-32 is used to calculate the interaction intention instruction check code. If the check fails, an ARQ retransmission request is sent; if the check passes, the signal encoding unit 42 is called.

[0087] The signal encoding unit 42 is used to convert the interaction intention instruction into a PWM waveform signal or a standard serial port instruction.

[0088] Among them, if the external device is a motor type, a PWM waveform signal is generated; if the external device is a communication device, a standard serial port command (such as RS-485 serial port command) is generated.

[0089] The driving interface unit 43 is configured to determine a hardware enable signal according to the type of the external device, and superimpose the hardware enable signal with the PWM waveform signal or the standard serial port instruction to generate the control signal to drive the external device.

[0090] The hardware enable signal is an electrical signal used to control the execution authority of instructions of an external device.

[0091] In an optional manner, the mobile phone terminal 4 is further used to:

[0092] The motion trajectory data is collected by the motion sensor, and the voice data collected by the voice input module is obtained. The interaction intention instruction, the motion trajectory data and the voice data are aligned in time and space to generate a multi-channel control signal to drive the external device.

[0093] Specifically, motion trajectory data (100Hz sampling), voice data (16kHz sampling), and interaction intention instructions (10Hz update) are input into the cache queue; dynamic time warping (DTW) aligns the three signals with a delay compensation window of ±50ms; and they are fused to generate multi-channel control signals.

[0094] In an optional manner, the mobile phone terminal 4 is further used to:

[0095] Before generating the multi-channel control signal, a logical conflict detection is performed on the interaction intention instruction, the motion trajectory data and the voice data. If a conflict is detected, a secondary confirmation process of the user is triggered.

[0096] The conflict determination rules are as follows: If the interaction intent command = "forward" and the motion sensor detects a "backward swing gesture," it's considered a directional conflict; if the voice command = "stop" and the EMG command = "accelerate," it's considered a speed conflict. The conflict confidence level is calculated as the number of conflicting features divided by the total number of features. If the conflict confidence level is greater than or equal to 0.6, the user's secondary confirmation process is triggered. A voice prompt will pop up on the phone: "Conflict detected, please confirm the operation." If there is no response within 5 seconds, the command is canceled.

[0097] In an optional manner, the mobile phone terminal 4 is further used to:

[0098] The target user's brain and myoelectric signals are displayed in real time.

[0099] Specifically, in the mobile terminal 4: ① Waveform rendering: EEG signals are drawn with a time domain waveform at a refresh rate of 50Hz (amplitude range ±100μV, time window 2 seconds); EMG signals are superimposed with a spectrum waterfall diagram (frequency range 0-500Hz, color code indicates power density). ② Feature annotation: Mark the α / β rhythm energy peak position in the waveform diagram (red mark >15μV 2 ); AR coefficient is displayed in real time as a bar graph (update rate 10Hz). ③ Alarm prompt: If the signal quality index (SQI) is less than 0.8, a yellow warning box will be displayed and the prompt "Detecting signal interference" will be displayed.

[0100] The technical solution of this embodiment can solve the problems of limited accuracy in brain and myoelectric signal acquisition and imperfect signal processing algorithms, improve the accuracy and stability of signal acquisition, effectively remove noise, comprehensively extract features, and meet the needs of high-precision human-computer interaction.

[0101] Figure 6 The flowchart of an embodiment of a human-computer interaction method based on brain myoelectric signals provided by the present invention is shown, and a human-computer interaction system based on brain myoelectric signals provided by the present invention is used. Figure 6 As shown, the following steps are included:

[0102] S1, the brain and myoelectricity acquisition device 1 is used to synchronously acquire the original brain and myoelectricity signals of the target user and perform signal preprocessing, and output the preprocessed mixed signal;

[0103] S2, host 2 is used to perform composite noise suppression on the pre-processed mixed signal to obtain an optimized mixed signal and store it;

[0104] S3, PC terminal 3 is used to perform abnormality detection on the optimized mixed signal, generate an electroencephalogram and myoelectricity detection report of the target user including the abnormality detection result, and output it for display;

[0105] S4, mobile phone terminal 4 is used to receive the target user's brain and myoelectric detection information, control the on and off state of the brain and myoelectric acquisition process, and mark the optimized mixed signal as abnormal through wireless communication.

[0106] The technical solution of this embodiment can solve the problems of limited accuracy in brain and myoelectric signal acquisition and imperfect signal processing algorithms, improve the accuracy and stability of signal acquisition, effectively remove noise, comprehensively extract features, and meet the needs of high-precision human-computer interaction.

[0107] The above description is merely a preferred embodiment of the present invention and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present invention is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the present invention.

[0108] It should be noted that the terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects and to define a specific order or precedence. Where appropriate, the order used for similar objects may be interchanged, such that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0109] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A human-computer interaction system based on brain and myoelectric signals, characterized in that: include: Brain and myoelectricity acquisition device (1), host (2), PC terminal (3) and mobile phone terminal (4); The electroencephalogram and myoelectricity acquisition device (1) is used to synchronously acquire the original electroencephalogram signal and the original myoelectricity signal of the target user, perform signal preprocessing, and output the preprocessed mixed signal; The host (2) is used to perform composite noise suppression on the pre-processed mixed signal to obtain an optimized mixed signal and store it; The PC terminal (3) is used to perform abnormality detection on the optimized mixed signal, generate a brain electromyography detection report of the target user including the abnormality detection result, and output it for display; The mobile phone terminal (4) is used to receive the target user's electroencephalogram and myoelectricity detection information, control the on / off state of the electroencephalogram and myoelectricity acquisition process, and mark the optimized mixed signal as abnormal through wireless communication.

2. The human-computer interaction system based on brain and myoelectric signals according to claim 1, characterized in that: The brain and myoelectricity acquisition device (1) comprises: a multi-channel high-precision sensor array (11) and a lead wire stabilization structure (12); a differential amplifier circuit (111) and an embedded pre-filter (112) are integrated in the multi-channel high-precision sensor array (11); The multi-channel high-precision sensor array (11) is used to synchronously collect the original EEG signal and the original EMG signal in different channels, and perform common-mode noise suppression and gain adjustment on the original EEG signal and the original EMG signal through the differential amplifier circuit (111), and then perform frequency band pre-filtering through the embedded pre-filter (112) to output the pre-processed mixed signal; The lead wire stabilizing structure (12) is used to physically reduce noise on sensor electrodes of the multi-channel high-precision sensor array (11) through an elastic contact electrode fixing component.

3. The human-computer interaction system based on brain and myoelectric signals according to claim 1, characterized in that: The host (2) comprises: a multi-stage adaptive filtering module (21) and a hybrid feature extraction module (22); The multi-stage adaptive filtering module (21) is used to perform composite noise suppression on the pre-processed mixed signal through a cascaded power frequency notch filter, a wavelet packet denoising unit and an independent component analysis unit to obtain the optimized mixed signal; The hybrid feature extraction module (22) is used to perform a time-frequency domain joint analysis on the optimized hybrid signal, extract the rhythm energy features corresponding to the EEG signal and the AR model coefficient features corresponding to the EMG signal, and generate a fusion feature vector.

4. The human-computer interaction system based on brain and myoelectric signals according to claim 3, characterized in that: The PC terminal (3) includes: an anomaly detection unit (31), a deep neural network unit (32) and a support vector machine unit (33); The abnormality detection unit (31) is used to perform abnormality detection on the optimized mixed signal, generate a brain electromyography detection report of the target user including the abnormality detection result, and output it for display; The deep neural network unit (32) is configured as a fully connected network having an input layer, two hidden layers and an output layer, wherein the number of neurons in the input layer is consistent with the dimension of the fused feature vector, and is used to perform nonlinear mapping on the fused feature vector received by the input layer based on the two hidden layers using a ReLU activation function, and output high-dimensional abstract features through the output layer; The support vector machine unit (33) is configured as a multi-classifier based on a radial basis kernel function, for receiving the high-dimensional abstract features and outputting a classification probability distribution, and generating an interaction intention instruction according to the classification probability distribution and a preset threshold.

5. The human-computer interaction system based on brain and myoelectric signals according to claim 4, characterized in that: The EMG detection information includes: EMG detection time, detection status, lead wire connection status, and wireless communication status.

6. The human-computer interaction system based on brain and myoelectric signals according to claim 5, characterized in that: The mobile phone terminal (4) is used to receive the interaction intention instruction through wireless transmission, and generate a control signal according to the interaction intention instruction to drive the external device; wherein, the wireless transmission method adopts a dual-mode redundant transmission mechanism; the dual-mode redundant transmission mechanism includes: Bluetooth 5.0 and WiFi6 alternating transmission channels, and automatically switches to the backup channel when it is detected that the packet loss rate of the current transmission channel exceeds a threshold.

7. The human-computer interaction system based on brain and myoelectric signals according to claim 6, characterized in that: The mobile phone terminal (4) comprises: a data packet checking unit (41), a signal encoding unit (42) and a driving interface unit (43); The data packet verification unit (41) is used to verify the integrity of the interaction intention instruction using a cyclic redundancy check and retransmission request protocol; The signal encoding unit (42) is used to convert the interaction intention instruction into a PWM waveform signal or a standard serial port instruction; The driving interface unit (43) is used to determine a hardware enable signal according to the type of the external device, and superimpose the hardware enable signal with the PWM waveform signal or the standard serial port instruction to generate the control signal to drive the external device.

8. The human-computer interaction system based on brain and myoelectric signals according to claim 7, characterized in that: The mobile phone terminal (4) is also used for: The motion trajectory data is collected by the motion sensor, and the voice data collected by the voice input module is obtained. The interaction intention instruction, the motion trajectory data and the voice data are aligned in time and space to generate a multi-channel control signal to drive the external device.

9. The human-computer interaction system based on brain and myoelectric signals according to claim 8, characterized in that: The mobile phone terminal (4) is also used for: Before generating the multi-channel control signal, a logical conflict detection is performed on the interaction intention instruction, the motion trajectory data and the voice data. If a conflict is detected, a secondary confirmation process of the user is triggered.

10. A human-computer interaction method based on brain and myoelectric signals, using the human-computer interaction system based on brain and myoelectric signals according to any one of claims 1 to 9, characterized in that: include: The brain and myoelectricity acquisition device (1) is used for synchronously acquiring the original brain and myoelectricity signals of the target user and performing signal preprocessing, and outputting the preprocessed mixed signal; The host (2) is used to perform composite noise suppression on the pre-processed mixed signal to obtain an optimized mixed signal and store it; The PC end (3) is used to perform abnormality detection on the optimized mixed signal, generate a brain electromyography detection report of the target user including the abnormality detection result, and output it for display; The mobile phone terminal (4) is used to receive the target user's electroencephalogram and myoelectricity detection information, control the on / off state of the electroencephalogram and myoelectricity acquisition process, and mark the optimized mixed signal as abnormal through wireless communication.

Images