Human-computer interaction control system based on brain-computer interface system

Abstract

The invention provides a human-computer interaction control system based on a brain-computer interface system. The human-computer interaction control system comprises a data acquisition module, a data processing module, a feedback monitoring module and a comprehensive output module, wherein the data acquisition module is used for acquiring an electroencephalogram signal of a user; the data processing module is used for performing preprocessing according to the received electroencephalogram signal, further performing feature extraction processing on the electroencephalogram signal to obtain feature information of the electroencephalogram signal, and performing feature recognition processing according to the obtained feature information to obtain a user intention recognition result; the feedback monitoring module is used for collecting a face image of a user and performing user state analysis according to the face image of the user to obtain a user state analysis result; and the comprehensive output module is used for generating corresponding output instructions according to the received user intention recognition result and the user state analysis result, wherein the output instructions comprise an interaction control instruction and an intervention instruction. According to the invention, the reliability and stability of man-machine interaction control can be improved.

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interface technology, and in particular to a human-computer interaction control system based on a brain-computer interface system. Background Art

[0002] Brain-computer interface (BCI) refers to a new type of human-computer interaction technology that directly connects the human or animal brain to external devices to achieve direct control of external devices.

[0003] At present, most of the research on brain-computer interface is carried out in the direction of EEG signal processing, focusing on improving the accuracy of signal processing algorithms and innovating the design of operation interfaces to improve control efficiency and user-friendliness. However, current solutions usually ignore the importance of the controller's state (the controller's state will directly affect the output results of interactive control instructions), resulting in insufficient interactive control experience and stability. Summary of the invention

[0004] In response to the above-mentioned technical problems that the current brain-computer interface interaction system has insufficient interactive control experience and stability, the present invention aims to provide a human-computer interaction control system based on the brain-computer interface system.

[0005] The purpose of the present invention is achieved by the following technical solutions:

[0006] The present invention shows a human-computer interaction control system based on a brain-computer interface system, including a data acquisition module, a data processing module, a feedback monitoring module and a comprehensive output module; wherein,

[0007] The data acquisition module is used to collect the user's EEG signals and transmit the collected EEG signals to the data processing module;

[0008] The data processing module is used to perform preprocessing according to the received EEG signal to obtain the preprocessed EEG signal, and further perform feature extraction processing on the EEG signal to obtain feature information of the EEG signal, and perform feature recognition processing according to the obtained feature information to obtain the user intention recognition result; the user intention recognition result is transmitted to the comprehensive output module;

[0009] The feedback monitoring module is used to collect the user's facial image, and perform user status analysis based on the user's facial image to obtain the user status analysis result, and transmit the user status analysis result to the comprehensive output module;

[0010] The comprehensive output module is used to generate corresponding output instructions based on the received user intention recognition results and user status analysis results, wherein the output instructions include interactive control instructions and intervention instructions.

[0011] Preferably, the system further comprises an interaction module;

[0012] The interaction module is used to play the feedback information of the controlled object so that the user can complete the BCI control of the controlled object according to the feedback information of the controlled object.

[0013] Preferably, the data acquisition module includes an EEG recording unit and a signal transmission unit; wherein,

[0014] The EEG recording unit is used to collect the user's EEG signals through a wearable electrode cap;

[0015] The signal transmission unit is used to transmit the collected EEG signals to the data processing module in real time.

[0016] Preferably, the data processing module includes a preprocessing unit, a feature extraction unit and an intention recognition unit; wherein,

[0017] The preprocessing unit is used to perform standardization and filtering on the acquired EEG signal to obtain a preprocessed EEG signal;

[0018] The feature extraction unit is used to perform feature extraction processing based on the preprocessed EEG signal to extract feature information such as time domain features, frequency domain features or time-frequency domain features of the EEG signal;

[0019] The intention recognition unit is used to classify and process the characteristic information of the EEG signal based on the trained classifier, obtain the user intention recognition result according to the output result of the classifier, and transmit the obtained user intention recognition result to the comprehensive output module.

[0020] Preferably, the feedback monitoring module includes an image acquisition unit and an image processing unit; wherein,

[0021] The image acquisition unit is used to acquire the user's face image in real time based on the camera, and transmit the acquired user's face image to the image processing unit;

[0022] The image processing unit is used to extract state features based on the acquired user face image to obtain real-time user state feature information; and perform state recognition processing on the user state feature information based on the trained analysis model to obtain user state analysis results, and transmit the user state analysis results to the comprehensive output module.

[0023] Preferably, the image processing unit includes an image preprocessing unit and a state analysis unit;

[0024] The image preprocessing unit is used to perform filtering and region extraction processing on the acquired user face image to obtain a preprocessed user face image;

[0025] The state analysis unit is used to extract user state features based on the preprocessed user face image to obtain real-time user state feature information, and perform user state analysis based on the user state features to obtain user state analysis results; wherein the user state feature information includes user expression features and / or user eye features.

[0026] Preferably, the state analysis unit specifically includes:

[0027] Extracting the user's eye region according to the preprocessed user face image, detecting the user's current eye opening and closing state according to the eye region, and obtaining the eye opening and closing state feature;

[0028] The user's continuous eye-closing time is accumulated and counted. When the user's continuous eye-closing time exceeds the preset standard time, the user status analysis result is output as closed eyes; otherwise, when the continuous eye-closing time does not exceed the preset standard time, the user status analysis result is output as normal.

[0029] Based on the trained expression recognition model, the preprocessed user face image is subjected to expression recognition to obtain the corresponding expression recognition result. When the expression recognition result is classified as abnormal, the user status analysis result is output as abnormal; otherwise, when the expression recognition result is classified as normal, the status analysis result is output as normal.

[0030] Preferably, the integrated output module includes an instruction output unit and an intervention unit;

[0031] The intervention unit is used to output corresponding intervention control instructions according to the user status analysis results when the user status analysis results are abnormal; wherein the intervention control instructions include disconnecting brain-computer control or playing prompt information;

[0032] The instruction output unit is used to obtain corresponding interactive control instructions according to the user intention recognition result when the user status analysis result is normal, and transmit the interactive control instructions to the host computer.

[0033] Preferably, the intervention unit specifically includes:

[0034] Detect the current user status analysis result. When the user status analysis result is eyes closed and not abnormal, output an intervention control instruction to make the player play music to achieve the influence on the user's current mood.

[0035] The beneficial effects of the present invention are as follows: the brain-computer interface human-computer interaction control system proposed in the present invention can, in addition to collecting and identifying EEG signals, further collect the user's facial status information, analyze the user's current status based on the facial image, and influence the EEG interaction control according to the user status analysis results, thereby implementing intervention adjustments to the user status or adjusting the control content to avoid the occurrence of abnormal situations, helping to achieve abnormal protection for the brain-computer interaction controller, and improving the reliability and stability of the human-computer interaction control. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0037] Figure 1 This is a framework diagram of a human-computer interaction control system based on a brain-computer interface system shown in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention is further described in conjunction with the following application scenarios.

[0039] See also Figure 1 , which shows a human-computer interaction control system based on a brain-computer interface system, including a data acquisition module, a data processing module, a feedback monitoring module and a comprehensive output module; wherein,

[0040] The data acquisition module is used to collect the user's EEG signals and transmit the collected EEG signals to the data processing module;

[0041] The data processing module is used to perform preprocessing according to the received EEG signal to obtain the preprocessed EEG signal, and further perform feature extraction processing on the EEG signal to obtain feature information of the EEG signal, and perform feature recognition processing according to the obtained feature information to obtain the user intention recognition result; the user intention recognition result is transmitted to the comprehensive output module;

[0042] The feedback monitoring module is used to collect the user's facial image, and perform user status analysis based on the user's facial image to obtain the user status analysis result, and transmit the user status analysis result to the comprehensive output module;

[0043] The comprehensive output module is used to generate corresponding output instructions based on the received user intention recognition results and user status analysis results, wherein the output instructions include interactive control instructions and intervention instructions.

[0044] The above-mentioned embodiment of the present invention proposes a brain-computer interface human-computer interaction control system, which can not only collect and identify EEG signals, but also further collect the user's facial state information, analyze the user's current state according to the facial image, and influence the EEG interaction control according to the user state analysis result, thereby implementing intervention adjustment of the user state or adjustment of the control content to avoid the occurrence of abnormal situations, which is helpful to realize abnormal protection of the brain-computer interaction controller and improve the reliability and stability of the human-computer interaction control.

[0045] The system also includes an interaction module;

[0046] The interaction module is used to play the feedback information of the controlled object so that the user can complete the BCI control of the controlled object according to the feedback information of the controlled object.

[0047] In an application scenario of drone control, the interactive module plays the current flight picture information of the drone in real time, so that the controller can control the flight of the drone in real time according to the flight picture.

[0048] In another unmanned driving application scenario, the interactive module plays the road condition picture of the smart car in real time, so that the controller can realize the unmanned driving remote control of the smart car according to the road condition picture.

[0049] The human-machine interaction control system of the present invention can be functionally configured according to different application scenarios to adapt to the use of different unmanned equipment remote control scenarios.

[0050] Preferably, the data acquisition module includes an EEG recording unit and a signal transmission unit; wherein,

[0051] The EEG recording unit is used to collect the user's EEG signals through a wearable electrode cap;

[0052] The signal transmission unit is used to transmit the collected EEG signals to the data processing module in real time.

[0053] The EEG recording unit uses a non-invasive electrode cap to achieve real-time acquisition of the controller's EEG signals. Based on the EEG recording technology (EEG), it records the low-amplitude electrophysiological characteristics of the cerebral cortex layer. The recorded EEG signals can reflect the interaction and information transmission between neurons, thereby extracting the corresponding EEG feature information based on the characteristics of different bands. The collected EEG signals are transmitted to the data processing module for further processing.

[0054] Among them, the data processing module can be built based on a data processing terminal or server. Based on powerful digital computing capabilities, it can realize fast and accurate processing of EEG signals, and output corresponding interactive control instructions to achieve real-time control of unmanned equipment.

[0055] Preferably, the data processing module includes a preprocessing unit, a feature extraction unit and an intention recognition unit; wherein,

[0056] The preprocessing unit is used to perform standardization and filtering on the acquired EEG signal to obtain a preprocessed EEG signal;

[0057] The feature extraction unit is used to perform feature extraction processing based on the preprocessed EEG signal to extract feature information such as time domain features, frequency domain features or time-frequency domain features of the EEG signal;

[0058] The intention recognition unit is used to classify and process the characteristic information of the EEG signal based on the trained classifier, obtain the user intention recognition result according to the output result of the classifier, and transmit the obtained user intention recognition result to the comprehensive output module.

[0059] For the received EEG signals, the EEG signals are first standardized and filtered by the preprocessing unit, including time domain alignment and artifact removal, so as to improve the quality of the EEG signals. Further feature extraction is performed on the EEG signals, wherein based on the basic form of the EEG signals, the corresponding time domain features, frequency domain features or time-frequency domain features can be extracted, and based on the extracted feature information or fused feature information, a classification model can be further used to identify the intention of the feature information, thereby obtaining the intention recognition result. Among them, the classification model can be trained based on the deep learning method of the neural network. By training the classification model based on the training set composed of historical data, the classification model can realize the intelligent recognition of the user's intention, and as the capacity of the training set increases, the accuracy of the classification model can be further improved; through the classification model, different EEG features can be distinguished, thereby realizing accurate recognition of the user's intention.

[0060] In one application scenario, for the processing and specific identification method of EEG signals, an EEG recognition model that has been trained in the prior art can also be used to implement the user intention recognition task based on EEG signals, and the present invention does not make specific limitations here.

[0061] Preferably, the feedback monitoring module includes an image acquisition unit and an image processing unit; wherein,

[0062] The image acquisition unit is used to acquire the user's face image in real time based on the camera, and transmit the acquired user's face image to the image processing unit;

[0063] The image processing unit is used to extract state features based on the acquired user face image to obtain real-time user state feature information; and perform state recognition processing on the user state feature information based on the trained analysis model to obtain user state analysis results, and transmit the user state analysis results to the comprehensive output module.

[0064] The status monitoring of the controller of the human-computer interaction system is realized through the user status analysis method based on the user's facial image, in which the facial image of the user when performing human-computer interaction control is collected in real time based on the image acquisition unit, and the user's facial image is extracted and the status is analyzed based on the collected facial image and the image recognition algorithm, so as to realize the intelligent recognition of the user status based on the machine vision recognition technology and obtain the user status analysis result. By identifying the user status, the current human-computer interaction control process can be further intervened or controlled according to the user status, which helps to improve the stability and reliability of the human-computer interaction system, and is also conducive to protecting the user and taking corresponding countermeasures to the abnormal status of the user in a timely manner.

[0065] Among them, considering that when the user performs EEG control, he is usually in an indoor environment facing the human-computer interaction screen, therefore, in the actual user face image acquisition process, due to the influence of factors such as the screen light emitted by the human-computer interaction screen, it is easy to cause abnormal light reflection on the user's face, thereby affecting the clarity of the facial features in the user's face image. Based on the above situation, the present invention is also specially provided with an image preprocessing unit to filter and enhance the acquired user face image to improve the quality of the user face image.

[0066] Preferably, the image processing unit includes an image preprocessing unit and a state analysis unit;

[0067] The image preprocessing unit is used to perform filtering and region extraction processing on the acquired user face image to obtain a preprocessed user face image;

[0068] The state analysis unit is used to extract user state features based on the preprocessed user face image to obtain real-time user state feature information, and perform user state analysis based on the user state features to obtain user state analysis results; wherein the user state feature information includes user expression features and / or user eye features.

[0069] Among them, through the image preprocessing unit, the obtained user face image is first filtered to improve the image quality, which can effectively eliminate the influence of abnormal lighting on the user's face due to the indoor environment or the light of the human-computer interaction screen, thereby improving the clarity of the face area in the user face image, thereby improving the image quality, which helps to indirectly improve the accuracy of subsequent user status analysis such as eye feature extraction or expression recognition based on the user face image.

[0070] Preferably, the image preprocessing unit performs filtering processing on the acquired user face image, specifically including:

[0071] According to the acquired user face image, the hue component value H(x, y), saturation component value S(x, y) and brightness component value V(x, y) of each pixel in the image are extracted based on the HSV color space;

[0072] Among them, the overall brightness factor of the current image is obtained according to the brightness component value of each pixel Where Nu represents the total number of pixels in the image, and (x, y) represents the pixel in the image;

[0073] Further, according to the acquired user face image, the red component value R(x, y), green component value G(x, y), blue component value B(x, y) and gray value TH(x, y) of each pixel in the image are extracted based on the RGB color space;

[0074] According to the gray value of each pixel in the image, the Sobel operator is used to detect the edge pixels in the image, the edge pixels PB in the image are marked, and the marked edge pixels are expanded at a scale of 1 pixel, and the expanded coverage area is marked as the edge feature area AB, and the non-edge feature area is marked as the general feature area AC;

[0075] For the general feature area, the pixel points in the general feature area are mapped to the feature set transformation:

[0076]

[0077] Get the mapping feature set of pixel points Among them, ΔR(x, y), ΔG(x, y), ΔB(x, y) represent the mapped R, G, B component values ​​respectively; RBT, GBT, BBT represent the standard R, G, B component values ​​respectively; YBT represents the standard brightness factor;

[0078] According to the mapping feature set of each pixel in the general feature area, the pixel points that meet the following conditions are marked as anchor pixel points PM:

[0079]

[0080] Among them, CD1 represents the conditional judgment function, and argmin() represents taking the corresponding parameter that meets the minimum value;

[0081] According to the obtained anchor pixel PM, the brightness component value V(PM) corresponding to the anchor pixel is extracted;

[0082] A first brightness adjustment process is performed on each pixel point in the general feature area, wherein the first brightness adjustment process function used is:

[0083]

[0084] Wherein, V′(x, y) represents the brightness component value of the pixel point (x, y) after the first brightness adjustment, (x, y)∈AC; ω1 represents a preset weight factor, where ω1>0.5, ΔBT represents a preset mapping threshold, where ΔBT∈[10,40]; KX is a reference parameter, ΔX(x, y) is a reference parameter, which refers to the maximum value among ΔR(x, y), ΔG(x, y), and ΔB(x, y);

[0085] After completing the first brightness adjustment process of the general feature area, updating the current user face image;

[0086] A second brightness adjustment process is further performed on each pixel point in the edge feature area, wherein the second brightness adjustment process function used is:

[0087] V″(x,y)=V(x,y)+ω2×V′(a,b)

[0088] Wherein, V″(x, y) represents the brightness component value of the pixel point (x, y) after the second brightness adjustment, (x, y)∈AB, ω2 represents a preset weight factor, wherein ω2∈[0.4,0.6], and V′(a, b) is a reference parameter, which refers to the brightness component value of the pixel point (a, b) in the general feature area closest to the pixel point (x, y);

[0089] Reconstruction is performed based on the brightness component values ​​of the pixels after brightness adjustment to obtain a filtered user face image.

[0090] Further based on the template matching or edge detection method, the face area extraction process is performed based on the user face image after filtering to obtain the preprocessed user face image.

[0091] Among them, the standard R, G, B component values ​​RBT, GBT, BBT are extracted according to the collected standard user face image when the system performs initial user settings, wherein the setting of the standard R, G, B component values ​​is completed according to the average R, G, B component values ​​of the facial area of ​​the collected standard user face image (or a standard pixel point manually specified); the standard brightness factor YBT is extracted according to the average brightness component value corresponding to the standard face image.

[0092] The above-mentioned embodiment of the present invention, aimed at the situation that the clarity of the user face area is easily affected by the light mapping of the human-computer interaction screen in the user face image, a technical scheme for preprocessing the user face image is specially proposed, wherein, based on the acquired user face image, firstly, based on the brightness characteristics of each pixel in the user face image, the overall brightness factor of the current image is extracted to reflect the overall brightness level of the current user face image; based on the grayscale characteristics of the pixel points, the image is divided into an edge feature area and a general feature area, and based on the characteristics of the face image, the general feature area can be used as a benchmark for whether it is affected by light mapping, and firstly, the general feature area is adjusted and processed, wherein based on the RGB characteristics of the pixel points, through the proposed mapping feature set, the mapping features can be extracted based on the relatively stable characteristics of the RGB characteristics of the face area (considering that in the flesh-colored area of ​​the face, if it is obviously affected by the mapping light, this part of the mapping light features can be accurately extracted by color decomposition), and based on the mapping feature set of the pixel points, The method comprises the following steps: performing an analysis, and accurately obtaining pixel points that are not affected by the mapping light (referring to the area that is only affected by the conventional lighting but not particularly affected by the mapping light) as anchor points through the proposed conditional judgment function; performing brightness adjustment processing on the entire general feature area based on the obtained anchor points, which can effectively eliminate the influence of local mapping light on the user's face, thereby restoring the real user's facial features; after completing the processing on the general feature area, further processing is performed on the edge feature area in the feature set, wherein when processing the edge feature area, the adjusted general feature area is used as the basis, so as to adaptively adjust the brightness of the edge feature area, thereby performing overall brightness adjustment and standardized adjustment on the part of the facial feature set in the image, which helps to further improve the performance level of facial features while eliminating the influence of abnormal mapping light, helps to improve the subsequent image quality, and helps to indirectly improve the accuracy of subsequent user state analysis such as eye feature extraction or expression recognition based on the user's facial image.

[0093] Preferably, the state analysis unit specifically includes:

[0094] Extracting the user's eye region according to the preprocessed user face image, detecting the user's current eye opening and closing state according to the eye region, and obtaining the eye opening and closing state feature;

[0095] The length of time the user has closed their eyes continuously is accumulated and counted. When the length of time the user has closed their eyes continuously exceeds the preset standard length of time, the user status analysis result is output as closed eyes; otherwise, when the length of time the user has closed their eyes continuously does not exceed the preset standard length of time, the user status analysis result is output as normal.

[0096] In one application scenario, when the user is physiologically afraid or tired, they are likely to close their eyes tightly. At this time, the user's control is likely to be distorted, which affects the effect of human-computer interaction control and even creates safety hazards. In this regard, considering that the user's eye characteristics are an important indicator, the user's eye condition is specially monitored. When the user is detected to have closed eyes, the intervention unit reminds the user of the status, including playing soothing music or issuing a prompt sound, etc., to calm the user's emotions, assist the user to re-enter the normal state and continue to complete the human-computer interaction control.

[0097] In one application scenario, the eye area is extracted based on the user's facial image, and the opening and closing status of the user's eyes is analyzed based on the eye area. This can be completed based on the AI ​​image recognition engine that has been developed in the prior art (such as Baidu's AI image recognition engine), and the opening and closing status of the user's eyes can be quickly and accurately identified, laying the foundation for subsequent statistics on the length of time the eyes are closed based on the closed eye status and further user status analysis.

[0098] Preferably, the state analysis unit specifically includes:

[0099] Based on the trained expression recognition model, the preprocessed user face image is subjected to expression recognition to obtain the corresponding expression recognition result. When the expression recognition result is classified as abnormal, the user status analysis result is output as abnormal; otherwise, when the expression recognition result is classified as normal, the status analysis result is output as normal.

[0100] In another application scenario, based on the acquired user facial image, the user's expression can be recognized based on image processing technology, and the current user state can be judged based on the expression recognition result. When the user is in an abnormal emotion (such as fright, frustration, etc.), the abnormal user emotion is immediately intervened or blocked by EEG control.

[0101] Preferably, the integrated output module includes an instruction output unit and an intervention unit;

[0102] The intervention unit is used to output corresponding intervention control instructions according to the user status analysis results when the user status analysis results are abnormal; wherein the intervention control instructions include disconnecting brain-computer control or playing prompt information;

[0103] The instruction output unit is used to obtain corresponding interactive control instructions according to the user intention recognition result when the user status analysis result is normal, and transmit the interactive control instructions to the host computer.

[0104] Among them, when the comprehensive output module outputs the final control command, it first refers to the current corresponding user state analysis result to ensure that the EEG control command made by the user is under normal circumstances, and then the corresponding output is performed. If the current user state analysis result finds that the current user state is abnormal, according to the actual situation of the user state, the user's current state is intervened to remind or help the user to return to a normal state, or the control interface is directly disconnected to protect the user and the device.

[0105] Preferably, the intervention unit specifically includes:

[0106] Detect the current user status analysis result. When the user status analysis result is eyes closed and not abnormal, output an intervention control instruction to make the player play music to achieve the influence on the user's current mood.

[0107] It should be noted that each functional unit / module in each embodiment of the present invention may be integrated into one processing unit / module, or each unit / module may exist physically separately, or two or more units / modules may be integrated into one unit / module. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of software functional unit / module.

[0108] Through the description of the above implementation mode, it can be clearly understood by those skilled in the art that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code or any appropriate combination thereof. For hardware implementation, the processor can be implemented in one or more of the following units: application specific integrated circuit (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), field programmable gate array (FPGA), processor, controller, microcontroller, microprocessor, other electronic units designed to implement the functions described herein or their combination. For software implementation, part or all of the process of the embodiment can be completed by instructing the relevant hardware through a computer program. When implemented, the above program can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication medium includes any medium that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that can be accessed by a computer. Computer-readable media can include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should analyze that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and scope of the technical solution of the present invention.

Claims

1. A human-computer interaction control system based on a brain-computer interface system, characterized in that: It includes data acquisition module, data processing module, feedback monitoring module and comprehensive output module; among which, The data acquisition module is used to collect the user's EEG signals and transmit the collected EEG signals to the data processing module; The data processing module is used to perform preprocessing according to the received EEG signal to obtain the preprocessed EEG signal, and further perform feature extraction processing on the EEG signal to obtain feature information of the EEG signal, and perform feature recognition processing according to the obtained feature information to obtain the user intention recognition result; the user intention recognition result is transmitted to the comprehensive output module; The feedback monitoring module is used to collect the user's facial image, and perform user status analysis based on the user's facial image to obtain the user status analysis result, and transmit the user status analysis result to the comprehensive output module; The comprehensive output module is used to generate corresponding output instructions based on the received user intention recognition results and user status analysis results, wherein the output instructions include interactive control instructions and intervention instructions.

2. A human-computer interaction control system based on a brain-computer interface system according to claim 1, characterized in that: It also includes interactive modules; The interaction module is used to play the feedback information of the controlled object so that the user can complete the BCI control of the controlled object according to the feedback information of the controlled object.

3. A human-computer interaction control system based on a brain-computer interface system according to claim 1, characterized in that: The data acquisition module includes an EEG recording unit and a signal transmission unit; wherein, The EEG recording unit is used to collect the user's EEG signals through a wearable electrode cap; The signal transmission unit is used to transmit the collected EEG signals to the data processing module in real time.

4. The human-computer interaction control system based on the brain-computer interface system according to claim 1 is characterized in that: The data processing module includes a preprocessing unit, a feature extraction unit and an intention recognition unit; wherein, The preprocessing unit is used to perform standardization and filtering on the acquired EEG signal to obtain a preprocessed EEG signal; The feature extraction unit is used to perform feature extraction processing on the preprocessed EEG signal to extract feature information of the time domain feature, frequency domain feature or time-frequency domain feature of the EEG signal; The intention recognition unit is used to classify and process the characteristic information of the EEG signal based on the trained classifier, obtain the user intention recognition result according to the output result of the classifier, and transmit the obtained user intention recognition result to the comprehensive output module.

5. The human-computer interaction control system based on the brain-computer interface system according to claim 1 is characterized in that: The feedback monitoring module includes an image acquisition unit and an image processing unit; wherein, The image acquisition unit is used to acquire the user's face image in real time based on the camera, and transmit the acquired user's face image to the image processing unit; The image processing unit is used to extract state features based on the acquired user face image to obtain real-time user state feature information; and perform state recognition processing on the user state feature information based on the trained analysis model to obtain user state analysis results, and transmit the user state analysis results to the comprehensive output module.

6. A human-computer interaction control system based on a brain-computer interface system according to claim 5, characterized in that: The image processing unit includes an image preprocessing unit and a state analysis unit; The image preprocessing unit is used to perform filtering and region extraction processing on the acquired user face image to obtain a preprocessed user face image; The state analysis unit is used to extract user state features based on the preprocessed user face image to obtain real-time user state feature information, and perform user state analysis based on the user state features to obtain user state analysis results; The user status feature information includes user expression features and / or user eye features.

7. A human-computer interaction control system based on a brain-computer interface system according to claim 6, characterized in that: The state analysis unit specifically includes: Extracting the user's eye region according to the preprocessed user face image, detecting the user's current eye opening and closing state according to the eye region, and obtaining the eye opening and closing state feature; The user's continuous eye-closing time is accumulated and counted. When the user's continuous eye-closing time exceeds the preset standard time, the user status analysis result is output as closed eyes; otherwise, when the continuous eye-closing time does not exceed the preset standard time, the user status analysis result is output as normal. Based on the trained expression recognition model, the preprocessed user face image is subjected to expression recognition to obtain the corresponding expression recognition result. When the expression recognition result is classified as abnormal, the user status analysis result is output as abnormal; otherwise, when the expression recognition result is classified as normal, the status analysis result is output as normal.

8. The human-computer interaction control system based on the brain-computer interface system according to claim 5, characterized in that: The integrated output module includes a command output unit and an intervention unit; The intervention unit is used to output corresponding intervention control instructions according to the user status analysis results when the user status analysis results are abnormal; wherein the intervention control instructions include disconnecting brain-computer control or playing prompt information; The instruction output unit is used to obtain corresponding interactive control instructions according to the user intention recognition result when the user status analysis result is normal, and transmit the interactive control instructions to the host computer.

9. A human-computer interaction control system based on a brain-computer interface system according to claim 8, characterized in that: The intervention units specifically include: Detect the current user status analysis result. When the user status analysis result is eyes closed and not abnormal, output an intervention control instruction to make the player play music to achieve the influence on the user's current mood.

Images