A human-computer interaction system and method based on brain and myoelectric signals

Abstract

The application relates to the technical field of brain and myoelectric signal interaction, and discloses a man-machine interaction system and method based on brain and myoelectric signals, which comprises the following: a brain and myoelectric signal acquisition device, which is used for acquiring brain and myoelectric signals, performing signal preprocessing, and outputting a preprocessed mixed signal; a host computer, which is used for performing complex noise suppression on the preprocessed mixed signal, obtaining an optimized mixed signal, and storing the optimized mixed signal; a PC terminal, which is used for performing abnormality detection on the optimized mixed signal, generating a brain and myoelectric detection report of a target user containing an abnormality detection result, and outputting and displaying the brain and myoelectric detection report; and a mobile phone terminal, which is used for receiving brain and myoelectric detection information of the target user, controlling the start and stop states of a brain and myoelectric acquisition process, and performing abnormality marking on the optimized mixed signal in a wireless communication mode. The application can improve the accuracy and stability of brain and myoelectric signal acquisition, can effectively remove noise, and can fully meet the demand of high-precision man-machine interaction.

Description

Technical Field

[0001] This invention relates to the field of brain electromyography (BEM) signal interaction technology, and in particular to a human-computer interaction system and method based on BEM signals. Background Technology

[0002] Existing brain-motor signal acquisition and interaction systems have several shortcomings. On the one hand, the accuracy of EEG and EMG signal acquisition is limited; EEG signals are weak and easily interfered with, and the connection stability of EMG signals during lead acquisition is poor, affecting data accuracy. On the other hand, the human-computer interaction methods for brain-motor signals need improvement; real-time anomaly detection and output display are not possible, making it difficult to meet the needs of high-precision human-computer interaction.

[0003] Therefore, there is an urgent need to provide a technical solution to address the above problems. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a human-computer interaction system and method based on electromyography (EMG) signals.

[0005] In a first aspect, the present invention provides a human-computer interaction system based on electromyography (EMG) signals, the technical solution of which is as follows:

[0006] A human-computer interaction system based on electroencephalogram (EEG) signals includes: an EEG acquisition device, a host computer, a PC, and a mobile phone.

[0007] The electroencephalogram (EEG) and electromyogram (EMG) acquisition device is used to simultaneously acquire the target user's raw EEG and raw EMG signals, perform signal preprocessing, and output the preprocessed mixed signal.

[0008] The host is used to perform composite noise suppression on the preprocessed mixed signal to obtain an optimized mixed signal and store it.

[0009] The PC is used to perform anomaly detection on the optimized mixed signal, generate an EMG report for the target user containing the anomaly detection results, and output and display it.

[0010] The mobile device is used to receive the target user's electroencephalogram (EEG) detection information via wireless communication, control the start and stop status of the EEG acquisition process, and mark anomalies in the optimized mixed signal.

[0011] The beneficial effects of the human-computer interaction system based on electromyography signals of the present invention are as follows:

[0012] The system of this invention can solve the problems of limited accuracy in brain electromyography signal acquisition and imperfect signal processing algorithms, improve the accuracy and stability of signal acquisition, effectively remove noise, comprehensively extract features, and meet the needs of high-precision human-computer interaction.

[0013] Based on the above scheme, the human-computer interaction system based on brain electromyography signals of the present invention can be further improved as follows.

[0014] In one alternative embodiment, the electroencephalogram (EEG) acquisition device includes: a multi-channel high-precision sensor array and a lead-line stabilization structure; the multi-channel high-precision sensor array integrates a differential amplifier circuit and an embedded pre-filter;

[0015] The multi-channel high-precision sensor array is used to synchronously acquire the original EEG signal and the original EMG signal in separate channels, and the original EEG signal and the original EMG signal are subjected to common-mode noise suppression and gain adjustment by the differential amplifier circuit, and then the frequency band pre-filter is performed by the embedded pre-filter to output the pre-processed mixed signal.

[0016] The lead-line stabilization structure is used to physically reduce noise on the sensor electrodes of the multi-channel high-precision sensor array through the elastic contact electrode fixing assembly.

[0017] In one alternative approach, the host computer includes: a multi-level adaptive filtering module and a hybrid feature extraction module;

[0018] The multi-stage adaptive filtering module is used to suppress composite noise in the preprocessed mixed signal through a cascaded power frequency notch filter, wavelet packet denoising unit, and independent component analysis unit to obtain the optimized mixed signal.

[0019] The hybrid feature extraction module is used to perform time-frequency domain joint analysis on the optimized hybrid signal, extract the rhythm energy features corresponding to the EEG signal and the AR model coefficient features corresponding to the EMG signal, and generate a fused feature vector.

[0020] In one alternative embodiment, the PC terminal includes: an anomaly detection unit, a deep neural network unit, and a support vector machine unit;

[0021] The anomaly detection unit is used to detect anomalies in the optimized mixed signal, generate an electroencephalogram (EEG) report for the target user that includes the anomaly detection results, and output and display it.

[0022] The deep neural network unit is configured as a fully connected network with an input layer, two hidden layers and an output layer. The number of neurons in the input layer is consistent with the dimension of the fused feature vector. It is used to perform nonlinear mapping on the fused feature vector received by the input layer based on the two hidden layers using the ReLU activation function, and output high-dimensional abstract features through the output layer.

[0023] The support vector machine unit is configured as a multi-classifier based on the radial basis function kernel function, used to receive the high-dimensional abstract features and output the classification probability distribution, and generate interactive intent commands based on the classification probability distribution and a preset threshold.

[0024] In one alternative approach, the electroencephalogram (EEG) detection information includes: EEG detection time, detection status, lead connection status, and wireless communication status.

[0025] In one optional approach, the mobile device is used to receive the interaction intent command via wireless transmission and generate control signals based on the interaction intent command to drive external devices; wherein the wireless transmission method adopts a dual-mode redundant transmission mechanism; the dual-mode redundant transmission mechanism includes: alternating transmission channels of Bluetooth 5.0 and WiFi 6, and automatically switching to a backup channel when the packet loss rate of the current transmission channel exceeds a threshold.

[0026] In one alternative embodiment, the mobile terminal includes: a data packet verification unit, a signal encoding unit, and a driver interface unit;

[0027] The data packet verification unit is used to verify the integrity of the interactive intent instruction using a cyclic redundancy check and retransmission request protocol.

[0028] The signal encoding unit is used to convert the interactive intent command into a PWM waveform signal or a standard serial port command.

[0029] The driver interface unit is used to determine the hardware enable signal according to the type of the external device, and superimpose the hardware enable signal with the PWM waveform signal or the standard serial port instruction to generate the control signal to drive the external device.

[0030] In one alternative approach, the mobile device is also used for:

[0031] Motion trajectory data is collected using a motion sensor, and voice data is acquired from a voice input module. The interactive intent command, the motion trajectory data, and the voice data are spatiotemporally aligned to generate multi-channel control signals to drive external devices.

[0032] In one alternative approach, the mobile device is also used for:

[0033] Before generating the multi-channel control signal, logical conflict detection is performed on the interactive intent command, the motion trajectory data and the voice data. If a conflict is detected, a secondary confirmation process for the user is triggered.

[0034] Secondly, this invention provides a human-computer interaction method based on electromyography (EMG) signals, employing a human-computer interaction system based on EMG signals as described in this invention. The technical solution of this method is as follows:

[0035] The electroencephalogram (EEG) and electromyogram (EMG) acquisition device is used to simultaneously acquire the raw EEG and EMG signals of the target user, perform signal preprocessing, and output the preprocessed mixed signal.

[0036] The host is used to perform composite noise suppression on the preprocessed mixed signal to obtain an optimized mixed signal and store it.

[0037] The PC terminal is used to perform anomaly detection on the optimized mixed signal, generate an EMG report for the target user containing the anomaly detection results, and output and display it.

[0038] The mobile device is used to receive the electroencephalogram (EEG) detection information of the target user via wireless communication, control the start and stop status of the EEG acquisition process, and mark anomalies in the optimized mixed signal.

[0039] The beneficial effects of the human-computer interaction method based on electromyography signals of the present invention are as follows:

[0040] The method of this invention can solve the problems of limited accuracy in brain electromyography signal acquisition and imperfect signal processing algorithms, improve the accuracy and stability of signal acquisition, effectively remove noise, comprehensively extract features, and meet the needs of high-precision human-computer interaction.

[0041] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0042] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0043] Figure 1 This is a structural block diagram of an embodiment of the human-computer interaction system based on electromyography (EMG) signals of the present invention.

[0044] Figure 2 This is a structural block diagram of a brain electromyography (EMG) acquisition device.

[0045] Figure 3 This is a block diagram of the host computer's structure.

[0046] Figure 4 This is a structural diagram of the PC version;

[0047] Figure 5 This is a structural diagram of the mobile version;

[0048] Figure 6 This is a flowchart illustrating an embodiment of a human-computer interaction method based on electromyography (EMG) signals according to the present invention. Detailed Implementation

[0049] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0050] Figure 1 A schematic diagram of an embodiment of a human-computer interaction system based on electromyography (EMG) signals provided by the present invention is shown. Figure 1 As shown, the system includes: a brain-muscle electroacupuncture acquisition device 1, a host 2, a PC terminal 3, and a mobile phone terminal 4;

[0051] The electroencephalogram (EEG) and electromyogram (EMG) acquisition device 1 is used to simultaneously acquire the target user's original EEG and EMG signals and perform signal preprocessing, outputting the preprocessed mixed signal.

[0052] The raw EEG signal refers to the unprocessed neural electrical activity signal directly acquired through scalp electrodes, with a frequency range of 0.5-100Hz and an amplitude of 1-100μV, reflecting changes in the potential of neuronal groups in the cerebral cortex. The raw EMG signal refers to the unprocessed muscle fiber electrical activity signal directly acquired through body surface electrodes, with a frequency range of 20-500Hz and an amplitude of 0.1-5mV, characterizing the intensity and temporal characteristics of muscle contraction. The preprocessed mixed signal refers to the fused signal of the raw EEG / EMG signals after differential amplification and frequency band pre-filtering, retaining effective physiological components (EEG 0.5-100Hz, EMG 20-500Hz), with a signal-to-noise ratio >20dB.

[0053] The host 2 is used to perform composite noise suppression on the preprocessed mixed signal to obtain an optimized mixed signal and store it.

[0054] The host unit 2 is connected to the EEG / EMG acquisition device 1 via lead wires, including 14 electrodes (8 EEG, 2 EMG, and 4 reference). All 14 electrodes are connected via HDMI interfaces. Optimized mixed signal refers to the output signal after preprocessing the mixed signal through power frequency notch filtering, wavelet packet denoising, and ICA processing, achieving a power frequency interference suppression ratio >40dB and artifact elimination rate >95%.

[0055] The PC terminal 3 is used to perform anomaly detection on the optimized mixed signal, generate an electroencephalogram (EEG) report for the target user containing the anomaly detection results, and output and display it.

[0056] Anomaly detection refers to the detection of abnormal waveforms in the optimized mixed signal. The EMG (electroencephalogram) test report includes basic information about the target user (medical record number, name, gender, ID number, phone number, and start and end times of the test). The PC version 3 includes buttons for canceling, saving, and saving and printing the EMG test report. The generated EMG test report is saved as an image, which can be downloaded.

[0057] It should be noted that the PC version 3 can manage EMG (electroencephalography) test reports, with specific functions including but not limited to: ① A pop-up window displayed in the center of the page. ② A list displaying patient information (list including medical record number, name, gender, ID number, phone number, and report time), with operation columns for printing, browsing, and downloading. ③ Search criteria: medical record number, name, ID number, phone number, and report time (start and end). ④ Added pagination functionality to the list. ⑤ Printing, browsing, and image viewing require the report time and the doctor's signature.

[0058] The mobile terminal 4 is used to receive the electroencephalogram (EEG) detection information of the target user via wireless communication, control the start and stop status of the EEG acquisition process, and mark anomalies in the optimized mixed signal.

[0059] The default wireless communication method is Bluetooth, but it can be adjusted according to the actual situation; no restrictions are set here. EMG detection information includes: EMG detection time (duration), detection status (in progress, paused, or stopped), lead connection status (normal or disconnected), and wireless communication status (whether the Bluetooth connection is normal).

[0060] It should be noted that the mobile app 4 has a start / pause button for switching the detection status. Specifically: when the wireless communication is normal, if the detection status is paused, the target user can switch to detection in progress by clicking the start / pause button; if the detection status is in progress, the target user can switch to paused by clicking the start / pause button. Furthermore, during EMG detection, if the target user detects abnormal information, they can quickly mark the abnormality using the marking button in the mobile app 4. This helps the target user to promptly capture and record important abnormalities, facilitating further examination and evaluation by medical personnel, thereby improving the overall accuracy and effectiveness of the detection.

[0061] In one alternative approach, such as Figure 2 As shown, the electroencephalogram (EEG) acquisition device 1 includes: a multi-channel high-precision sensor array 11 and a lead-line stabilization structure 12; the multi-channel high-precision sensor array 11 integrates a differential amplifier circuit 111 and an embedded pre-filter 112.

[0062] The multi-channel high-precision sensor array 11 is used to synchronously acquire the original EEG signal and the original EMG signal through separate channels. The differential amplifier circuit 111 performs common-mode noise suppression and gain adjustment on the original EEG signal and the original EMG signal, and then performs frequency band pre-filtering through the embedded pre-filter 112 to output the pre-processed mixed signal.

[0063] The multi-channel high-precision sensor array 11 includes an electroencephalogram (EEG) signal channel and an electromyography (EMG) signal channel. The EEG signal channel uses Ag / AgCl electrodes with an input impedance >100MΩ, acquiring raw EEG signals S in the 0.5-100Hz frequency band. EEG (t). The electromyography (EMG) signal acquisition channel uses stainless steel electrodes with an input impedance of <10kΩ to acquire raw EMG signals in the 20-500Hz frequency band. EMG (t).

[0064] The differential amplifier circuit 111 is used to: process the raw EEG signal S EEG (t) and the original electromyographic signal S EMG (t) Differential amplification processing is performed by applying gain separately, and the original EEG signal S EEG (t) Amplified 1000 times, common-mode rejection ratio ≥120dB, output brain electroencephalogram signal V EEG (t); raw electromyographic signal S EMG (t) Amplified 100 times, common-mode rejection ratio ≥90dB, output amplified electromyographic signal V EMG (t).

[0065] The embedded pre-filter is used to: amplify the electroencephalogram (EEG) signal V. EEG (t) Perform bandpass filtering (0.5Hz high-pass + 100Hz low-pass), output filtered EEG signal F EEG (t); for amplified electromyographic signals V EMG (t) Perform bandpass filtering (20Hz high-pass + 500Hz low-pass), output filtered electromyographic signal F EEG (t).

[0066] Among them, the preprocessed mixed signal refers to the filtered EEG signal F processed according to the channel number. EEG (t) and filtered electromyographic signal F EEG The preprocessed mixed signal X(t) generated by merging (t) is an N×T matrix, where N represents the number of channels in the multi-channel high-precision sensor array 11, and T represents the number of sampling points (the number of discrete signal samples of a single channel within a time window). It should be noted that the multi-channel high-precision sensor array 11 in this embodiment includes 8 EEG channels and 2 EMG channels by default. The 8 EEG channels are connected to the target user's scalp, and the 2 EMG channels are connected to the target user's arm or leg.

[0067] The lead-line stabilization structure 12 is used to physically reduce noise on the sensor electrodes of the multi-channel high-precision sensor array 11 through the elastic contact electrode fixing assembly.

[0068] The lead-line stabilization structure 12 refers to the elastic silicone electrode fixing assembly, which stabilizes the electrode-skin contact impedance below 5kΩ (traditional devices >20kΩ). The physical noise reduction process of the lead-line stabilization structure 12 is as follows: ① Electrode fixing: The Ag / AgCl EEG electrode is embedded in the silicone fixing ring, and a constant pressure of 5N is applied to adhere to the scalp; the stainless steel electromyography electrode is fixed to the muscle belly with an elastic band, with a contact pressure of 3N±0.5N. ② Impedance stabilization: A 10Hz / 1mA AC detection current is injected to monitor the electrode-skin impedance in real time; if the impedance is >5kΩ, a micro vibration motor (amplitude 0.1mm) is automatically activated to reduce the contact resistance. ③ Motion artifact suppression: The elastic material absorbs mechanical vibration energy with a frequency >2Hz, reducing the amplitude of motion artifacts by 60%.

[0069] In one alternative approach, such as Figure 3 As shown, the host 2 includes a multi-level adaptive filtering module 21 and a hybrid feature extraction module 22.

[0070] The multi-stage adaptive filtering module 21 is used to suppress composite noise in the preprocessed mixed signal through a cascaded power frequency notch filter, wavelet packet denoising unit and independent component analysis unit, so as to obtain the optimized mixed signal.

[0071] The system includes a power frequency notch filter (Q=30) that applies a 50Hz notch filter to the preprocessed mixed signal X(t) to eliminate mains interference and output the first mixed signal X1(t). A wavelet packet denoising unit performs a 4-level wavelet packet decomposition on the first mixed signal X1(t), reconstructs the signal, and outputs the second mixed signal X2(t). An independent component analysis unit performs the FastICA algorithm on the second mixed signal X2(t) to separate physiological signals from artifact components and outputs an optimized mixed signal Y(t).

[0072] The hybrid feature extraction module 22 is used to perform time-frequency domain joint analysis on the optimized hybrid signal, extract the rhythm energy features corresponding to the EEG signal and the AR model coefficient features corresponding to the EMG signal, and generate a fusion feature vector.

[0073] Among them, rhythm energy features refer to the wavelet energy values ​​of EEG signals in specific frequency bands (α rhythm 8-13Hz, β rhythm 13-30Hz). AR model coefficient features refer to the coefficient vector of the fourth-order autoregressive model of EMG signals, which is obtained by minimizing the prediction error. The fusion feature vector is an 8-dimensional feature vector formed by splicing rhythm energy features (2D), AR coefficients (4D), and EMG coupling features (2D).

[0074] The rhythm energy features include EEG α-rhythm energy features and EEG β-rhythm energy features. Specifically, based on the signal channel number, the optimized EEG signal Y is separated from the optimized mixed signal Y(t). EEG (t) and optimized electromyographic signal Y EMG (t); for optimizing EEG signal Y EEG (t) Perform Morlet wavelet transform to calculate the energy features of α rhythm (8-13Hz) and β rhythm (13-30Hz); construct an autoregressive (AR) model and solve for the AR model coefficient features (default fourth-order coefficients a1, a2, a3, a4); calculate the mutual information between the EEG α rhythm energy features and the optimized EMG signal in the time-frequency domain to obtain the first coupling feature, and calculate the mutual information between the EEG β rhythm energy features and the optimized EMG signal in the time-frequency domain to obtain the second coupling feature; concatenate the EEG α rhythm energy features, EEG β rhythm energy features, AR model coefficient features, the first coupling feature, and the second coupling feature to output the fused feature vector F.

[0075] In one alternative approach, such as Figure 4 As shown, the PC terminal 3 includes: an anomaly detection unit 31, a deep neural network unit 32, and a support vector machine unit 33;

[0076] The anomaly detection unit 31 is used to perform anomaly detection on the optimized mixed signal, generate an electroencephalogram (EEG) report for the target user containing the anomaly detection results, and output and display it.

[0077] The deep neural network unit 32 is configured as a fully connected network with an input layer, two hidden layers and an output layer. The number of neurons in the input layer is consistent with the dimension of the fused feature vector. It is used to perform nonlinear mapping on the fused feature vector received by the input layer based on the two hidden layers using the ReLU activation function, and output high-dimensional abstract features through the output layer.

[0078] The deep neural network unit 32 includes an input layer, a first hidden layer, a second hidden layer, and an output layer. The input layer receives a fused feature vector F, performs a linear mapping on F to obtain a first feature, and outputs it to the first hidden layer. The first hidden layer calculates the weights of the first feature and activates it using the ReLU activation function to obtain a second feature, which is then output to the second hidden layer. The second hidden layer calculates the weights of the second feature and activates it using the ReLU activation function to obtain a third feature, which is then output to the output layer. The output layer performs a linear mapping on the third feature and outputs a high-dimensional abstract feature O.

[0079] The support vector machine unit 33 is configured as a multi-classifier based on the radial basis function kernel function, used to receive the high-dimensional abstract features and output the classification probability distribution, and generate the interactive intent instruction according to the classification probability distribution and a preset threshold.

[0080] Wherein: ① The high-dimensional abstract feature O is mapped to the reproducing kernel Hilbert space K(O,O) through the radial basis function (RBF). i )=exp(-γ‖OO i || 2 ), O i Let γ represent the i-th support vector, where γ = 0.05; ② Calculate the multi-class decision function. M represents the number of support vectors, α i Denotes Lagrange multipliers, y i ∈[-1,1] represents the category label; ③ Platt scaling is used to convert g(O) into a probability distribution: max c P(y=c|O) represents the probability distribution of category c, and A and B represent the maximum likelihood estimation parameters; ④ If max c If P(y=c|O)>0.9, then the corresponding intention instruction will be output.

[0081] In one alternative approach, the mobile terminal 4 is used to receive interactive intent commands via wireless transmission and generate control signals based on the interactive intent commands to drive external devices.

[0082] The control signal consists of a PWM waveform (duty cycle 5%-95%) or a standard serial port command superimposed with an enable signal (3.3V / 5V level). External devices refer to the controlled terminal devices (such as intelligent prostheses, wheelchairs, and home appliances), with interfaces compatible with RS-485 / CAN bus or PWM drivers.

[0083] The wireless transmission method employs a dual-mode redundancy transmission mechanism. This mechanism includes alternating transmission channels between Bluetooth 5.0 and WiFi 6. When the packet loss rate of the current transmission channel exceeds a threshold, it automatically switches to the backup channel. The default threshold for the packet loss rate is 5%, but it can be adjusted according to actual conditions. Specifically, the packet loss rate of the current channel (Bluetooth / WiFi) is calculated every 50ms. If the packet loss rate is greater than 5% and persists for three consecutive cycles, the channel is deemed to have failed, and the system switches to the backup channel in the next 10ms time slot. Furthermore, the last 5 data packets are buffered before switching and retransmitted preferentially after the new channel is established.

[0084] In one alternative approach, such as Figure 5 As shown, the mobile terminal 4 includes: a data packet verification unit 41, a signal encoding unit 42, and a driving interface unit 43.

[0085] The data packet verification unit 41 is used to verify the integrity of the interactive intent instruction using a cyclic redundancy check and retransmission request protocol.

[0086] Specifically, the CRC-32 algorithm is used to calculate the interaction intent command check code. If the check fails, an ARQ retransmission request is sent. If the check passes, the signal encoding unit 42 is invoked.

[0087] The signal encoding unit 42 is used to convert the interactive intent command into a PWM waveform signal or a standard serial port command.

[0088] If the external device is a motor, a PWM waveform signal is generated; if the external device is a communication device, a standard serial port command (such as an RS-485 serial port command) is generated.

[0089] The drive interface unit 43 is used to determine the hardware enable signal according to the type of the external device, and superimpose the hardware enable signal with the PWM waveform signal or the standard serial port instruction to generate the control signal to drive the external device.

[0090] Among them, the hardware enable signal is an electrical signal used to control the execution permission of external device instructions.

[0091] In an alternative embodiment, the mobile terminal 4 is further used for:

[0092] Motion trajectory data is collected using a motion sensor, and voice data is acquired from a voice input module. The interactive intent command, the motion trajectory data, and the voice data are spatiotemporally aligned to generate multi-channel control signals to drive external devices.

[0093] Specifically, motion trajectory data (100Hz sampling), voice data (16kHz sampling), and interactive intent commands (10Hz update) are input into a buffer queue; dynamic time warping (DTW) aligns the three signals with a delay compensation window of ±50ms; and multi-channel control signals are generated by fusion.

[0094] In an alternative embodiment, the mobile terminal 4 is further used for:

[0095] Before generating the multi-channel control signal, logical conflict detection is performed on the interactive intent command, the motion trajectory data and the voice data. If a conflict is detected, a secondary confirmation process for the user is triggered.

[0096] The conflict determination rules are as follows: if the interaction intention command is "forward" and the motion sensor detects a "backward swing gesture," it is determined to be a directional conflict; if the voice command is "stop" and the EEG command is "accelerate," it is determined to be a speed conflict. Conflict confidence is calculated as the number of conflicting features divided by the total number of features. If the conflict confidence is greater than or equal to 0.6, a secondary confirmation process is triggered. A voice prompt appears on the mobile device saying "Conflict detected, please confirm the operation." If there is no response within 5 seconds, the command is canceled.

[0097] In an alternative embodiment, the mobile terminal 4 is further used for:

[0098] The electromyography (EMG) signals of the target user are displayed in real time.

[0099] Specifically, in the mobile version 4: ① Waveform rendering: EEG signals are plotted as time-domain waveforms at a refresh rate of 50Hz (amplitude range ±100μV, time window 2 seconds); EMG signals are superimposed on a spectral waterfall plot (frequency range 0-500Hz, color scales indicate power density). ② Feature annotation: The peak positions of α / β rhythm energy are marked in the waveform diagram (red markers >15μV). 2 ); AR coefficient is displayed in real time as a bar chart (update rate 10Hz). ③ Alarm prompt: If the signal quality index (SQI) < 0.8, a yellow warning box is displayed and the message "Signal interference detected" is displayed.

[0100] The technical solution of this embodiment can solve the problems of limited accuracy in electromyography (EMG) signal acquisition and imperfect signal processing algorithms, improve the accuracy and stability of signal acquisition, effectively remove noise, comprehensively extract features, and meet the needs of high-precision human-computer interaction.

[0101] Figure 6 This diagram illustrates a flowchart of an embodiment of a human-computer interaction method based on electromyography (EMG) signals provided by the present invention, employing a human-computer interaction system based on EMG signals as provided by the present invention. Figure 6 As shown, it includes the following steps:

[0102] S1, Electroencephalogram and Electromyogram (EMG) acquisition device 1 is used to simultaneously acquire the target user's raw EEG signal and raw EMG signal and perform signal preprocessing, and output the preprocessed mixed signal;

[0103] S2, host 2 is used to perform composite noise suppression on the preprocessed mixed signal to obtain an optimized mixed signal and store it;

[0104] S3 and PC 3 are used to perform anomaly detection on the optimized mixed signal, generate an EMG detection report for the target user containing the anomaly detection results, and output and display it.

[0105] S4, the mobile terminal 4 is used to receive the electroencephalogram (EEG) detection information of the target user via wireless communication, control the start and stop status of the EEG acquisition process, and mark the optimized mixed signal as abnormal.

[0106] The technical solution of this embodiment can solve the problems of limited accuracy in electromyography (EMG) signal acquisition and imperfect signal processing algorithms, improve the accuracy and stability of signal acquisition, effectively remove noise, comprehensively extract features, and meet the needs of high-precision human-computer interaction.

[0107] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0108] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0109] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A human-computer interaction system based on electromyography (EMG) signals, characterized in that, include: Electroencephalogram (EEG) acquisition device (1), host (2), PC terminal (3) and mobile terminal (4); The electroencephalogram (EEG) acquisition device (1) is used to simultaneously acquire the target user's original EEG signal and original EMG signal and perform signal preprocessing, and output the preprocessed mixed signal; The host (2) is used to perform composite noise suppression on the preprocessed mixed signal to obtain an optimized mixed signal and store it; The PC terminal (3) is used to perform anomaly detection on the optimized mixed signal, generate an electroencephalogram (EEG) report containing the anomaly detection results for the target user, and output and display it. The mobile terminal (4) is used to receive the electroencephalogram (EEG) detection information of the target user, control the start and stop status of the EEG acquisition process, and mark the optimized mixed signal as abnormal through wireless communication. The host (2) includes: a multi-level adaptive filtering module (21) and a hybrid feature extraction module (22); The multi-level adaptive filtering module (21) is used to suppress composite noise in the preprocessed mixed signal by cascading power frequency notch filter, wavelet packet denoising unit and independent component analysis unit to obtain the optimized mixed signal; The hybrid feature extraction module (22) is used to perform time-frequency domain joint analysis on the optimized hybrid signal, extract the rhythm energy features corresponding to the EEG signal and the AR model coefficient features corresponding to the EMG signal, and generate a fusion feature vector; The PC terminal (3) includes: an anomaly detection unit (31), a deep neural network unit (32), and a support vector machine unit (33). The anomaly detection unit (31) is used to perform anomaly detection on the optimized mixed signal, generate an electroencephalogram (EEG) report containing the anomaly detection results for the target user, and output and display it. The deep neural network unit (32) includes: an input layer, a first hidden layer, a second hidden layer, and an output layer; the input layer receives the fused feature vector, performs a linear mapping on the fused feature vector to obtain a first feature, and outputs it to the first hidden layer; the first hidden layer calculates the weights of the first feature and activates it using the ReLU activation function to obtain a second feature, and outputs it to the second hidden layer; the second hidden layer calculates the weights of the second feature and activates it using the ReLU activation function to obtain a third feature, and outputs it to the output layer; the output layer performs a linear mapping on the third feature and outputs a high-dimensional abstract feature. The support vector machine unit (33) is configured as a multi-classifier based on the radial basis function kernel function, used to receive the high-dimensional abstract features and output a classification probability distribution, and generate interactive intent instructions based on the classification probability distribution and a preset threshold, specifically: The high-dimensional abstract features are mapped to the regenerating kernel Hilbert space through the radial basis kernel function. , This represents the i-th support vector. =0.05; Calculate the multi-class decision function ; Indicates the number of support vectors. Represents the Lagrange multipliers. Category labels; Using Platt scaling Convert to probability distribution: ; Let A and B represent the probability distribution of category c, and let B represent the maximum likelihood estimation parameters. like If so, the corresponding interactive intent command will be output.

2. The human-computer interaction system based on electromyography (EMG) signals according to claim 1, characterized in that, The electroencephalogram (EEG) acquisition device (1) includes: a multi-channel high-precision sensor array (11) and a lead wire stabilization structure (12); the multi-channel high-precision sensor array (11) integrates a differential amplifier circuit (111) and an embedded pre-filter (112). The multi-channel high-precision sensor array (11) is used to synchronously acquire the original EEG signal and the original EMG signal through the channel, and to perform common-mode noise suppression and gain adjustment on the original EEG signal and the original EMG signal through the differential amplifier circuit (111), and then perform frequency band pre-filtering through the embedded pre-filter (112) to output the pre-processed mixed signal; The lead-line stabilization structure (12) is used to physically reduce noise on the sensor electrodes of the multi-channel high-precision sensor array (11) through the elastic contact electrode fixing assembly.

3. The human-computer interaction system based on electromyography (EMG) signals according to claim 1, characterized in that, The electroencephalogram (EEG) detection information includes: EEG detection time, detection status, lead connection status, and wireless communication status.

4. The human-computer interaction system based on electromyography (EMG) signals according to claim 3, characterized in that, The mobile terminal (4) is used to receive the interaction intent instruction via wireless transmission and generate control signals according to the interaction intent instruction to drive external devices; wherein, the wireless transmission method adopts a dual-mode redundant transmission mechanism; the dual-mode redundant transmission mechanism includes: alternating transmission channels of Bluetooth 5.0 and WiFi 6, and automatically switching to the backup channel when the packet loss rate of the current transmission channel exceeds the threshold.

5. The human-computer interaction system based on electromyography (EMG) signals according to claim 4, characterized in that, The mobile terminal (4) includes: a data packet verification unit (41), a signal encoding unit (42), and a driver interface unit (43). The data packet verification unit (41) is used to perform integrity verification on the interactive intent instruction using the Cyclic Redundancy Check and Retransmission Request Protocol. The signal encoding unit (42) is used to convert the interactive intent instruction into a PWM waveform signal or a standard serial port instruction; The drive interface unit (43) is used to determine the hardware enable signal according to the type of the external device, and superimpose the hardware enable signal with the PWM waveform signal or the standard serial port instruction to generate the control signal to drive the external device.

6. The human-computer interaction system based on electromyography (EMG) signals according to claim 5, characterized in that, The mobile terminal (4) is also used for: Motion trajectory data is collected using a motion sensor, and voice data is acquired from a voice input module. The interactive intent command, the motion trajectory data, and the voice data are spatiotemporally aligned to generate multi-channel control signals to drive external devices.

7. The human-computer interaction system based on electromyography (EMG) signals according to claim 6, characterized in that, The mobile terminal (4) is also used for: Before generating the multi-channel control signal, logical conflict detection is performed on the interactive intent command, the motion trajectory data and the voice data. If a conflict is detected, a secondary confirmation process for the user is triggered.

8. A human-computer interaction method based on electroencephalogram (EEG) signals, employing the human-computer interaction system based on EEG signals as described in any one of claims 1 to 7, characterized in that, include: The electroencephalogram (EEG) acquisition device (1) is used to simultaneously acquire the raw EEG signal and raw EMG signal of the target user and perform signal preprocessing, and output the preprocessed mixed signal; The host (2) is used to perform composite noise suppression on the preprocessed mixed signal, obtain an optimized mixed signal, and store it; The PC terminal (3) is used to perform anomaly detection on the optimized mixed signal, generate an electroencephalogram (EEG) report containing the anomaly detection results for the target user, and output and display it. The mobile terminal (4) is used to receive the electroencephalogram (EEG) detection information of the target user, control the start and stop status of the EEG acquisition process, and mark the optimized mixed signal as abnormal via wireless communication.

Images