Intention recognition method, device, equipment and medium based on brain-computer signal

Abstract

The embodiment of the present invention provides a method, device, equipment and medium for intention recognition based on brain-computer signals, which belongs to the technical field of intention classification. The method includes: determining the number of targets corresponding to the initial visual stimulation object of the control state task corresponding to the target user and the number of targets corresponding to the initial non-visual stimulation object corresponding to the target user; determining the target classification model corresponding to the target user according to the number of targets and the number of targets; determining the initial coding sequence corresponding to the initial visual stimulation object, and displaying the visual stimulation information corresponding to the initial visual stimulation object in the target interface according to the initial coding sequence; real-time acquisition of the target EEG data of the target user gazing at the target interface; performing control state task recognition on the target EEG data according to the target classification model to obtain the first predicted visual stimulation object corresponding to the target EEG data; determining the target intention corresponding to the target user according to the first predicted visual stimulation object.

Description

Technical Field

[0001] The present invention relates to the technical field of intention classification, and in particular to a method, device, equipment and medium for intention recognition based on brain-computer signals. Background Art

[0002] In the EEG-based visual fixation target prediction algorithm, the user's EEG data is mainly collected periodically to infer the visual stimulus object that the user is fixating on in each time period. However, the performance requirements in different application scenarios, such as helping patients with motor impairment to communicate or control environmental equipment, and reducing the false alarm rate (i.e., reducing the false positive rate) are particularly important. This means that it is necessary to ensure that the non-fixation state is not mistakenly judged as the fixation state to avoid misleading or risking the patient. In entertainment-oriented scenarios, improving the hit rate (i.e., increasing the true positive rate) is even more critical. The user's fixation state needs to be identified quickly and accurately to provide a smooth and natural interactive experience and enhance immersion.

[0003] However, the user's cooperation often fluctuates, especially when used for a long time or when attention is distracted, and they may not be able to focus on a specific visual stimulus object in each cycle. This may result in the inability to accurately capture the user's gaze state in some cycles, thereby reducing the overall prediction accuracy. Summary of the invention

[0004] The main purpose of the embodiments of the present invention is to provide a method, device, equipment and medium for intention recognition based on brain-computer signals, aiming to solve the problem in related technologies that the user's cooperation degree often fluctuates, especially when used for a long time or the attention is distracted, and it may be impossible to focus on a specific visual stimulation object in each cycle, which leads to the inability to accurately capture the user's gaze state in certain cycles, thereby reducing the overall prediction accuracy.

[0005] In a first aspect, an embodiment of the present invention provides a method for identifying intention based on brain-computer signals, comprising:

[0006] Determining the number of targets corresponding to the initial visual stimulation objects of the control state task corresponding to the target user and the number of targets corresponding to the initial non-visual stimulation objects corresponding to the target user;

[0007] Determine a target classification model corresponding to the target user according to the target quantity and the number of targets;

[0008] Determining an initial coding sequence corresponding to the initial visual stimulation object, and displaying visual stimulation information corresponding to the initial visual stimulation object in a target interface according to the initial coding sequence;

[0009] Collecting target EEG data of the target user gazing at the target interface in real time;

[0010] Performing control state task identification on the target EEG data according to the target classification model to obtain a first predicted visual stimulation object corresponding to the target EEG data;

[0011] A target intention corresponding to the target user is determined according to the first predicted visual stimulation object.

[0012] In a second aspect, an embodiment of the present invention provides an intention recognition device based on brain-computer signals, comprising:

[0013] A data determination module, used to determine the number of targets corresponding to the initial visual stimulation objects of the control state task corresponding to the target user and the number of targets corresponding to the initial non-visual stimulation objects corresponding to the target user;

[0014] A model determination module, used to determine the target classification model corresponding to the target user according to the target quantity and the number of targets;

[0015] A visual determination module, used to determine an initial coding sequence corresponding to the initial visual stimulation object, and display visual stimulation information corresponding to the initial visual stimulation object in a target interface according to the initial coding sequence;

[0016] A data acquisition module, used for real-time acquisition of target EEG data corresponding to the target user gazing at the target interface;

[0017] A data classification module, used for performing control state task identification on the target EEG data according to the target classification model to obtain a first predicted visual stimulation object corresponding to the target EEG data;

[0018] An intention determination module is used to determine the target intention corresponding to the target user according to the first predicted visual stimulation object.

[0019] In the third aspect, an embodiment of the present invention further provides a terminal device, comprising a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the steps of any one of the methods for intention recognition based on brain-computer signals provided in the specification of the present invention are realized.

[0020] In a fourth aspect, an embodiment of the present invention further provides a storage medium for computer-readable storage, characterized in that the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any one of the methods for intention recognition based on brain-computer signals provided in the specification of the present invention.

[0021] The embodiment of the present invention provides a method, device, equipment and medium for intention recognition based on brain-computer signals. The method includes: by determining the target number of the initial visual stimulation object and the target number of the initial non-visual stimulation object of the control state task corresponding to the target user, customized visual stimulation information can be provided according to the personalized needs and preferences of the user. This personalized interaction method can significantly improve the user's satisfaction and user experience. Then, the target classification model corresponding to the target user is determined according to the target number and the number of targets; the initial coding sequence corresponding to the initial visual stimulation object is determined, and the visual stimulation information corresponding to the initial visual stimulation object is displayed in the target interface according to the initial coding sequence; and then by real-time acquisition of the target EEG data when the target user is looking at the target interface, the user's operation can be quickly responded to, and the real-time and fluency of the interaction can be improved. By using the target classification model to perform control state task recognition on the EEG data, the first predicted visual stimulation object of the target user can be quickly predicted, thereby capturing the target intention of the target user in real time. This fast intention prediction can significantly reduce the delay of user operation and improve the user experience. By continuously acquiring and analyzing the user's target EEG data, the target classification model can be continuously optimized to improve its recognition accuracy and robustness of the user's intention. This continuous optimization enables the system to better adapt to the needs of different users and different scenarios. It also solves the problem in related technologies that the user's cooperation often fluctuates, especially when used for a long time or with distracted attention, and may not be able to focus on a specific visual stimulus object in each cycle, which leads to the inability to accurately capture the user's gaze state in certain cycles, thereby reducing the overall prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 A flowchart of a method for identifying intention based on brain-computer signals provided by an embodiment of the present invention;

[0024] Figure 2 A schematic diagram of seven targets corresponding to a 28-bit near-perfect sequence encoding provided for implementing this embodiment;

[0025] Figure 3 A schematic diagram of the module structure of a device for identifying intentions based on brain-computer signals provided by an embodiment of the present invention;

[0026] Figure 4 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0029] It should be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0030] The embodiment of the present invention provides a method, apparatus, device and medium for identifying intention based on brain-computer signals. The method for identifying intention based on brain-computer signals can be applied to a terminal device, which can be an electronic device such as a tablet computer, a laptop computer, a desktop computer, a personal digital assistant and a wearable device. The terminal device can be a server or a server cluster.

[0031] Brain computer interface (BCI) can express the user's intention without relying on surrounding muscles and nerves, and can control external devices without physical movement. Visual evoked potential (VEP) is an evoked EEG signal, which is mainly collected in the parietal and occipital areas of the human brain and induced by visual stimulation. VEP-based BCI is the main category of non-invasive passive BCI. When the subject looks at the stimulus, VEP can be easily detected, so VEP-based BCI is suitable and stable enough as a communication system. BCI is divided into synchronous and asynchronous types according to the interaction between the user and the device. The synchronous system requires the user to follow the steps of the stimulation system, shift the gaze to the target stimulus in advance and wait for the stimulation to start. During the fixed interval visual stimulation process, body movements should be avoided to prevent artifacts. After the stimulation ends, the system gives feedback to the user. In this scenario, human operation is restricted by the system framework. In the synchronous system, each signal acquisition cycle has a clear time marker, and there is a strict time alignment between data and stimulation. An asynchronous system means that data collection and processing does not rely on fixed time intervals, but is performed according to the user's intentions or actions. In an asynchronous BCI system based on VEP, visual stimulation continues to appear, and the algorithm determines that the user is in an idle state or a control state, and feeds back the recognition results of the control state to the user as soon as possible. When the algorithm predicts the visual stimulation target that the user is looking at, it generally collects EEG data in cycles. The EEG data collected in each cycle predicts the visual stimulation object that the user is looking at in that cycle. If the cycle does not predict that the user is looking at each visual stimulation object, the EEG data of the next cycle will be collected and predicted.

[0032] Different usage scenarios have different emphasis on the performance of the algorithm. For patients who have lost their motor skills, it is more important to reduce the false alarm rate (false positive rate). For game interaction scenarios, it is more important to increase the hit rate (true positive rate). However, users cannot look at the visual stimulation object carefully in every cycle. This results in the user looking at the visual stimulation object in some cycles, but being mistakenly judged as not looking at the visual stimulation object. When the user's cooperation is low, the prediction accuracy is low. Traditional visual stimulation is mostly based on laboratory paradigms, setting a higher refresh rate to present visual stimulation, and requiring higher accuracy of time synchronization signals (trigger / marker), which is not conducive to the transformation of VEP-based BCI systems to productization. In VEP-based BCI systems with fewer targets, the previous dynamic window length method is difficult to find significance from the observation perspective of Bayesian posterior probability, making it difficult to apply in this few-target system.

[0033] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0034] Please refer to Figure 1 , Figure 1 A flowchart of a method for intention recognition based on brain-computer signals provided in an embodiment of the present invention.

[0035] like Figure 1 As shown, the intention recognition method based on brain-computer signals includes steps S101 to S106.

[0036] Step S101, determining the number of targets corresponding to the initial visual stimulation objects of the control state task corresponding to the target user and the number of targets corresponding to the initial non-visual stimulation objects corresponding to the target user.

[0037] Exemplarily, the target number corresponding to the initial visual stimulation object is determined according to the requirement corresponding to the control state task corresponding to the target user. The target number is related to the requirement corresponding to the control state task corresponding to the target user.

[0038] For example, the control state task corresponding to the target user is for controlling a wheelchair, that is, the initial visual stimulation objects include but are not limited to forward, backward, left turn and right turn, etc., and then the number of targets corresponding to the initial visual stimulation objects is determined to be four.

[0039] Exemplarily, the number of targets corresponding to the initial non-visual stimulation object is determined according to the requirements corresponding to the control state task corresponding to the target user. The initial non-visual stimulation object is the control state of the target corresponding to the non-initial visual stimulation object, which is used to reduce the false alarm rate. The initial non-visual stimulation object is also a pseudo-key, which has no direct effect on the terminal output target user. The target user still needs to use the initial visual stimulation object to switch the terminal to the control state.

[0040] Step S102: determining a target classification model corresponding to the target user according to the target quantity and the number of targets.

[0041] Exemplarily, the number of categories corresponding to the target classification model corresponding to the target user is determined according to the number of targets and the number of targets, that is, the resting state category, that is, the initial non-visual stimulation object, is included in the actual control state task (predicting the visual stimulation object), or the classification task of the actual N initial visual stimulation objects is changed to the classification of N+n categories, that is, pseudo buttons (non-visual stimulation objects). In this way, the training data corresponding to each classification type is obtained, and the model training is performed according to the training data and the classification data to obtain the target classification model.

[0042] In some embodiments, determining the target classification model corresponding to the target user based on the target quantity and the target number includes: determining the target classification number of the target classification model corresponding to the target user based on the target quantity and the target number; obtaining first EEG data corresponding to the initial visual stimulation object of the target user and obtaining second EEG data corresponding to the initial non-visual stimulation object of the target user; performing classification model training based on the first EEG data, the second EEG data and the target classification number to obtain the target classification model corresponding to the target user.

[0043] Exemplarily, the target number of initial visual stimulation objects involved in the control state task of the target user is determined. For example, if the task requires the target user to look at four different images, the target number is 4. Similarly, the target number of non-visual stimulation objects involved in the control state task of the target user is determined. For example, if the task requires three non-visual stimulation objects as the resting state, the target number is 3. Thus, the target number and the target number are added to obtain the target classification number. Assuming the target number is 4 and the target number is 3, the target classification number is 7.

[0044] Exemplarily, the target user is asked to gaze at an initial visual stimulation object (such as different images), and a first EEG data of the target user is collected in real time using an EEG acquisition device (such as an EEG helmet).

[0045] Exemplarily, in a phase-coded visual brain-computer interface system, the selected n types of pseudo-keys may be coding sequences that do not satisfy the time lag relationship T. For example, a 28-bit near-perfect sequence (1,0,1,1,0,1,1,1,1,1,0,0,1,1,0,0,1,0,1,0,0,0,0,0,0,1,1) is used for 7-target encoding, and the time lag between temporal stimulation objects is 4 frames. Then, sequences that do not satisfy the time lag relationship such as (1,0,1,0,1,1,1,1,1,0,0,1,1,0,0,1,0,1,0,0,0,0,0,1,1,1,0,1) can be used as pseudo-keys. The terminal can also additionally collect a small amount of resting-state data of the user as pseudo-keys, and the appropriate number of pseudo-keys can be selected according to different users and devices. Figure 2 The figure shows a schematic diagram of 7 targets encoded by a 28-bit near-perfect sequence. Target 2 is obtained by shifting the original sequence to the left by 4 frames to the end, and so on. The sequence with a shift number other than 4 or an integer multiple of 4 can be regarded as a pseudo-key, so as to obtain the second EEG data corresponding to the initial non-visual stimulus object when the target user looks at the corresponding screen information under the pseudo-key.

[0046] Exemplarily, a suitable classification algorithm is selected. Commonly used classification algorithms include support vector machine (SVM), random forest, deep learning model (such as convolutional neural network CNN), etc., and then the first EEG data and the second EEG data and their corresponding labels are input into the selected classification algorithm to perform model training. During the training process, the performance of the model can be evaluated using methods such as cross-validation to ensure that the model has a high classification accuracy. Thus, the target classification model is obtained.

[0047] Step S103: determining an initial coding sequence corresponding to the initial visual stimulation object, and displaying visual stimulation information corresponding to the initial visual stimulation object in a target interface according to the initial coding sequence.

[0048] For example, in a 28-bit binary sequence encoding, 0 and 1 correspond to two different pictures that fully present a cycle (1,0,1,1,0,1,0,1,1,1,1,1,0,0,1,1,0,0,1,0,1,0,0,0,0,0,0,1,1), which is the initial coding sequence of a target in the initial visual stimulation object. Because the property of a near-perfect sequence is a good autocorrelation characteristic, which can also be understood as an orthogonal characteristic, the sequence obtained after cyclic shift (0,1,0,1,1,1,1,1,0,0,1,1,0,0,1,0,1,0,0,0,0,0,1,11,0,1,1) is the initial coding sequence of another target in the initial visual stimulation object, which has a low correlation with the initial coding sequence of the previous target, so the two initial coding sequences have good separability. By analogy, the number of shifted bits depends on the number of targets. Thus, the initial coding sequence corresponding to the initial visual stimulation object is obtained.

[0049] Exemplarily, visual stimulation information corresponding to the initial visual stimulation object is displayed in the target interface according to the initial coding sequence, thereby providing support for the subsequent target user to gaze at the corresponding initial visual stimulation object.

[0050] Step S104: collecting target EEG data of the target user gazing at the target interface in real time.

[0051] Exemplarily, an EEG (electroencephalogram) device is selected, such as a portable EEG helmet or an EEG cap, and it is ensured that the target interface can clearly display the initial visual stimulus object, so that when the target user looks at the target interface, the data collection function of the EEG device is started, thereby obtaining the target EEG data when the target user looks at the target interface.

[0052] Step S105 , performing control state task identification on the target EEG data according to the target classification model to obtain a first predicted visual stimulation object corresponding to the target EEG data.

[0053] Exemplarily, control state task identification is performed on the target EEG data according to the target classification model to obtain the classification distribution result corresponding to the target EEG data, and then the classification result corresponding to the maximum probability in the classification distribution result is determined as the first predicted visual stimulation object corresponding to the target EEG data.

[0054] In some embodiments, performing control state task identification on the target EEG data according to the target classification model to obtain a first predicted visual stimulation object corresponding to the target EEG data includes: determining a window length corresponding to a target window, and when the window length is greater than a data length corresponding to the target EEG data, continuing to collect the target EEG data corresponding to the target user until the window length is less than or equal to the data length corresponding to the target EEG data; when the window length is less than or equal to the data length corresponding to the target EEG data, performing data segmentation on the target EEG data according to the target window and window step to obtain a first window of EEG data corresponding to the target EEG data; performing control state task identification on the first window of EEG data according to the target classification model to obtain the first predicted visual stimulation object corresponding to the first window of EEG data.

[0055] Exemplarily, the window length corresponding to the target window is set according to demand. When the window length is greater than the data length corresponding to the target EEG data, it is necessary to continue collecting the target EEG data corresponding to the target user until the window length is less than or equal to the data length corresponding to the target EEG data.

[0056] Exemplarily, when the window length is less than or equal to the data length corresponding to the target EEG data, the target EEG data is segmented according to the target window and the window step to obtain multiple first window EEG data corresponding to the target EEG data, and then the first window EEG data are respectively subjected to control state task identification according to the target classification model to obtain the first predicted visual stimulation object corresponding to each first window EEG data.

[0057] For example, the starting position is set to 0, that is, the starting point of the first window EEG data. The end position is set to the window length, that is, the end point of the first window EEG data. Thus, the sliding window technology is used to gradually move the window according to the window step length to extract the EEG data in each window. The EEG data in each extracted window is used as an independent first window EEG data. Repeat this process until the end of the target EEG data is reached. Thus, after obtaining each first window EEG data, it is directly input into the target classification model, so that the target classification model classifies each first window EEG data and outputs the first predicted visual stimulation object corresponding to each first window EEG data.

[0058] In some embodiments, the control state task identification of the first window EEG data according to the target classification model is performed to obtain the first predicted visual stimulation object corresponding to the first window EEG data, including: performing control state task identification on the first window EEG data according to the target classification model to obtain the target recognition probability corresponding to the first window EEG data; determining a preset threshold, and comparing the target recognition probability with the preset threshold, when the target recognition probability is less than the preset threshold, deleting the classification result of the first window EEG data, and there is no need to obtain the first predicted visual stimulation object corresponding to the first window EEG data; when the target recognition probability is greater than or equal to the preset threshold, determining the first predicted visual stimulation object corresponding to the first window EEG data according to the target recognition probability.

[0059] Exemplarily, the first window EEG data is input into the target classification model, and then the target classification model classifies the input first window EEG data and outputs the target recognition probability of each possible visual stimulation object or non-visual stimulation object.

[0060] Exemplarily, the preset threshold is a threshold used to determine whether the recognition probability is reliable, and is usually determined according to actual needs and experimental results. For example, the preset threshold can be set to 0.7, indicating that the recognition result is considered reliable only when the recognition probability is greater than or equal to 0.7.

[0061] Exemplarily, the target recognition probability is compared with a preset threshold. When the target recognition probability is less than the preset threshold, the classification result of the first window EEG data is deleted, and the corresponding predicted visual stimulation object is not recorded. When the target recognition probability is greater than or equal to the preset threshold, the first predicted visual stimulation object corresponding to the first window EEG data is determined according to the target recognition probability and recorded.

[0062] Step S106: determining the target intention corresponding to the target user according to the first predicted visual stimulation object.

[0063] Exemplarily, a mapping relationship between an initial visual stimulation object and a user's intention is established, and then after obtaining a first predicted visual stimulation object, a target intention corresponding to the first predicted visual stimulation object is obtained according to the mapping relationship.

[0064] In some embodiments, determining the target intention corresponding to the target user according to the first predicted visual stimulation object includes: determining a fusion quantity and a fusion rule; when the first predicted quantity corresponding to the first predicted visual stimulation object is equal to the fusion quantity, performing data fusion on multiple first predicted visual stimulation objects according to the fusion rule to obtain the target intention corresponding to the target user; when the first predicted quantity corresponding to the first predicted visual stimulation object is less than the fusion quantity, continuing to collect the target EEG data corresponding to the target user, and continuing to segment the target EEG data according to the target window and the window step to obtain second window EEG data; performing control state task identification on the second window EEG data according to the target classification model to obtain the second predicted visual stimulation object corresponding to the second window EEG data, until the sum of the first predicted quantity corresponding to the first predicted visual stimulation object and the second predicted quantity corresponding to the second predicted visual stimulation object is equal to the fusion quantity, and then performing data fusion on multiple first predicted visual stimulation objects and the second predicted visual stimulation objects according to the fusion rule to obtain the target intention corresponding to the target user.

[0065] For example, the fusion number refers to how many prediction results (i.e., the first predicted visual stimulation object and the second predicted visual stimulation object) are needed to perform data fusion to ultimately determine the intention of the target user. For example, the fusion number can be set to 5, indicating that 5 prediction results are needed.

[0066] For example, the fusion rule refers to how to merge multiple prediction results to determine the final target intent. For example, the fusion rule includes but is not limited to: selecting the prediction result with the most occurrences; performing weighted averaging according to the recognition probability of each prediction result, etc.

[0067] Exemplarily, the target classification model is used to perform control state task identification on the EEG data of the first window to obtain the first predicted visual stimulation object corresponding to each EEG data of the first window. When the first prediction number is equal to the fusion number, data fusion is performed on multiple first predicted visual stimulation objects according to the fusion rule to obtain the target intention corresponding to the target user. When the first prediction number is less than the fusion number, the target EEG data corresponding to the target user continues to be collected for further prediction, and the target window and window step size are used to continue to segment the target EEG data to obtain the second window EEG data, and then the target classification model is used to perform control state task identification on the EEG data of the second window to obtain the second predicted visual stimulation object corresponding to each EEG data of the second window.

[0068] Exemplarily, the second predicted number of the second predicted visual stimulation object is added to the first predicted number, the total predicted number is updated, and then the predicted number and the fused number are compared again. When the total predicted number is equal to the fused number, data fusion is performed on multiple first predicted visual stimulation objects and second predicted visual stimulation objects according to the fusion rule to obtain the target intention corresponding to the target user. When the total predicted number is still less than the fused number, the above steps are repeated to continue collecting and predicting until the total predicted number reaches the fused number.

[0069] In some embodiments, performing data fusion on multiple first predicted visual stimulation objects according to the fusion rule to obtain the target intention corresponding to the target user includes: comparing the first predicted visual stimulation object with the initial non-visual stimulation object, and when the first predicted visual stimulation object belongs to the initial non-visual stimulation object, modifying the first predicted visual stimulation object to a resting state; and performing data fusion using the fusion rule according to the modified first predicted visual stimulation object to obtain the target intention corresponding to the target user.

[0070] Exemplarily, the initial non-visual stimulus objects refer to those specific objects that should not be identified as predicted visual stimulus objects. For example, the initial non-visual stimulus objects may include resting states, background noises or other excluded stimulus objects.

[0071] Exemplarily, the target classification model is used to perform control state task identification on each first window EEG data to obtain the first predicted visual stimulation object corresponding to each window, and then each first predicted visual stimulation object is compared with the initial non-visual stimulation object. When the first predicted visual stimulation object belongs to the initial non-visual stimulation object, the first predicted visual stimulation object is modified to a resting state; when the first predicted visual stimulation object does not belong to the initial non-visual stimulation object, the first predicted visual stimulation object remains unchanged.

[0072] Exemplarily, all modified first predicted visual stimulation objects are collected, and then data fusion is performed on multiple modified first predicted visual stimulation objects according to a fusion rule to obtain the target intention corresponding to the target user, for example, the fusion rule is majority voting.

[0073] For example, assuming that the initial non-visual stimulus objects are: resting state, background noise. Assuming that the prediction results of the target classification model for multiple first window EEG data are: first window 1: image 1, first window 2: image 2, first window 3: background noise, first window 4: image 3, first window 5: resting state, among which, first window 1: image 1 (does not belong to the initial non-visual stimulus object, remains unchanged), first window 2: image 2 (does not belong to the initial non-visual stimulus object, remains unchanged), first window 3: background noise (belongs to the initial non-visual stimulus object, modified to resting state), first window 4: image 3 (does not belong to the initial non-visual stimulus object, remains unchanged), first window 5: resting state (belongs to the initial non-visual stimulus object, remains in resting state), then the modified first predicted visual stimulus object is: first window 1: image 1, first window 2: image 2, first window 3: resting state, first window 4: image 3, first window 5: resting state. Assume that the fusion rule is majority voting: majority voting is performed on the modified first predicted visual stimulus object: image 1: 1 time, image 2: 1 time, resting state: 2 times, image 3: 1 time, and then the majority voting result is resting state. Therefore, according to the fusion rule, it is determined that the target user's target intention is the corresponding resting state and no adjustment is made.

[0074] In some embodiments, after obtaining the target intent, the method further includes: obtaining a target instruction corresponding to the target intent from a database according to the target intent; and executing the target instruction so that the target user achieves the target intent according to the target instruction.

[0075] Exemplarily, the target intent of the target user is obtained, such as "resting state", "wheelchair turning left", "wheelchair turning right", etc. According to the target intent, a pre-configured database is accessed, which contains target instructions corresponding to each target intent. Each entry in the database generally contains a target intent and a target instruction associated therewith. Thus, the target intent is used as a query condition to retrieve the corresponding target instruction from the database. For example, if the target intent is "resting state", the target instruction corresponding thereto is retrieved.

[0076] Exemplarily, the target instruction is obtained from the database and executed, so that the target user achieves the target intention according to the target instruction.

[0077] The embodiment of the present invention provides a method, device, equipment and medium for intention recognition based on brain-computer signals. The method includes: by determining the target number of the initial visual stimulation object and the target number of the initial non-visual stimulation object of the control state task corresponding to the target user, customized visual stimulation information can be provided according to the personalized needs and preferences of the user. This personalized interaction method can significantly improve the user's satisfaction and user experience. Then, the target classification model corresponding to the target user is determined according to the target number and the number of targets; the initial coding sequence corresponding to the initial visual stimulation object is determined, and the visual stimulation information corresponding to the initial visual stimulation object is displayed in the target interface according to the initial coding sequence; and then by real-time acquisition of the target EEG data when the target user is looking at the target interface, the user's operation can be quickly responded to, and the real-time and fluency of the interaction can be improved. By using the target classification model to perform control state task recognition on the EEG data, the first predicted visual stimulation object of the target user can be quickly predicted, thereby capturing the target intention of the target user in real time. This fast intention prediction can significantly reduce the delay of user operation and improve the user experience. By continuously acquiring and analyzing the user's target EEG data, the target classification model can be continuously optimized to improve its recognition accuracy and robustness of the user's intention. This continuous optimization enables the system to better adapt to the needs of different users and different scenarios. It also solves the problem in related technologies that the user's cooperation often fluctuates, especially when used for a long time or with distracted attention, and may not be able to focus on a specific visual stimulus object in each cycle, which leads to the inability to accurately capture the user's gaze state in certain cycles, thereby reducing the overall prediction accuracy.

[0078] See also Figure 3 , Figure 3An intention recognition device 200 based on brain-computer signals is provided in an embodiment of the present application. The intention recognition device 200 based on brain-computer signals includes a data determination module 201, a model determination module 202, a visual determination module 203, a data acquisition module 204, a data classification module 205, and an intention determination module 206, wherein the data determination module 201 is used to determine the number of targets corresponding to the initial visual stimulation objects of the control state task corresponding to the target user and the number of targets corresponding to the initial non-visual stimulation objects corresponding to the target user; the model determination module 202 is used to determine the target classification corresponding to the target user according to the target number and the target number. model; a visual determination module 203, used to determine the initial coding sequence corresponding to the initial visual stimulation object, and display the visual stimulation information corresponding to the initial visual stimulation object in the target interface according to the initial coding sequence; a data acquisition module 204, used to collect the target EEG data corresponding to the target user's gaze at the target interface in real time; a data classification module 205, used to perform control state task recognition on the target EEG data according to the target classification model to obtain the first predicted visual stimulation object corresponding to the target EEG data; an intention determination module 206, used to determine the target intention corresponding to the target user according to the first predicted visual stimulation object.

[0079] In some implementations, the model determination module 202, in the process of determining the target classification model corresponding to the target user according to the target quantity and the target number, performs:

[0080] Determine the target classification quantity of the target classification model corresponding to the target user according to the target quantity and the number of targets;

[0081] Obtaining first EEG data corresponding to the initial visual stimulation object of the target user and obtaining second EEG data corresponding to the initial non-visual stimulation object of the target user;

[0082] Classification model training is performed according to the first EEG data, the second EEG data and the target classification quantity to obtain the target classification model corresponding to the target user.

[0083] In some implementations, the data classification module 205, in the process of performing control state task identification on the target EEG data according to the target classification model to obtain the first predicted visual stimulation object corresponding to the target EEG data, performs:

[0084] Determine a window length corresponding to the target window, and when the window length is greater than the data length corresponding to the target EEG data, continue to collect the target EEG data corresponding to the target user until the window length is less than or equal to the data length corresponding to the target EEG data;

[0085] When the window length is less than or equal to the data length corresponding to the target EEG data, data segmentation is performed on the target EEG data according to the target window and the window step to obtain the first window EEG data corresponding to the target EEG data;

[0086] The first predicted visual stimulation object corresponding to the first window EEG data is obtained by performing control state task identification on the first window EEG data according to the target classification model.

[0087] In some implementations, the data classification module 205, in the process of performing control state task identification on the first window EEG data according to the target classification model to obtain the first predicted visual stimulation object corresponding to the first window EEG data, performs:

[0088] Performing control state task recognition on the EEG data in the first window according to the target classification model to obtain a target recognition probability corresponding to the EEG data in the first window;

[0089] Determine a preset threshold, and compare the target recognition probability with the preset threshold, and when the target recognition probability is less than the preset threshold, delete the classification result of the first window EEG data, and there is no need to obtain the first predicted visual stimulation object corresponding to the first window EEG data;

[0090] When the target recognition probability is greater than or equal to the preset threshold, the first predicted visual stimulation object corresponding to the first window EEG data is determined according to the target recognition probability.

[0091] In some implementations, the intention determination module 206, in the process of determining the target intention corresponding to the target user according to the first predicted visual stimulation object, performs:

[0092] Determine the fusion quantity and fusion rules;

[0093] When the first predicted number corresponding to the first predicted visual stimulation object is equal to the fused number, data fusion is performed on a plurality of the first predicted visual stimulation objects according to the fusion rule to obtain the target intention corresponding to the target user;

[0094] When the first predicted number corresponding to the first predicted visual stimulation object is less than the fused number, the target EEG data corresponding to the target user continues to be collected, and the target EEG data continues to be segmented according to the target window and the window step to obtain the second window EEG data;

[0095] According to the target classification model, control state task identification is performed on the second window EEG data to obtain the second predicted visual stimulation object corresponding to the second window EEG data, until the sum of the first predicted number corresponding to the first predicted visual stimulation object and the second predicted number corresponding to the second predicted visual stimulation object is equal to the fusion number, and then data fusion is performed on multiple first predicted visual stimulation objects and second predicted visual stimulation objects according to the fusion rule to obtain the target intention corresponding to the target user.

[0096] In some implementations, the intention determination module 206, in the process of fusing data of the plurality of first predicted visual stimulation objects according to the fusion rule to obtain the target intention corresponding to the target user, executes:

[0097] comparing the first predicted visual stimulation object with the initial non-visual stimulation object, and when the first predicted visual stimulation object belongs to the initial non-visual stimulation object, modifying the first predicted visual stimulation object to a resting state;

[0098] The target intention corresponding to the target user is obtained by performing data fusion using the fusion rule according to the modified first predicted visual stimulation object.

[0099] In some implementations, after obtaining the target intention, the EEG signal-based intention recognition device 200 further performs:

[0100] Obtaining a target instruction corresponding to the target intention from a database according to the target intention;

[0101] Execute the target instruction so that the target user achieves the target intention according to the target instruction.

[0102] In some embodiments, the brain-computer signal-based intention recognition device 200 can be applied to a terminal device.

[0103] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the intention recognition device 200 based on brain-computer signals described above can refer to the corresponding process in the aforementioned embodiment of the intention recognition method based on brain-computer signals, and will not be repeated here.

[0104] See also Figure 4 , Figure 4 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present invention.

[0105] like Figure 4As shown, the terminal device 300 includes a processor 301 and a memory 302 , and the processor 301 and the memory 302 are connected via a bus 303 , such as an I2C (Inter-integrated Circuit) bus.

[0106] Specifically, the processor 301 is used to provide computing and control capabilities to support the operation of the entire terminal device. The processor 301 can be a central processing unit (CPU), and the processor 301 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0107] Specifically, the memory 302 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0108] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a partial structure related to the embodiment of the present invention, and does not constitute a limitation on the terminal device to which the embodiment of the present invention is applied. The specific server may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0109] Among them, the processor is used to run a computer program stored in the memory, and implement any one of the brain-computer signal-based intention recognition methods provided by the embodiments of the present invention when executing the computer program.

[0110] In one embodiment, the processor is used to run a computer program stored in the memory, and implements the following steps when executing the computer program:

[0111] Determining the number of targets corresponding to the initial visual stimulation objects of the control state task corresponding to the target user and the number of targets corresponding to the initial non-visual stimulation objects corresponding to the target user;

[0112] Determine a target classification model corresponding to the target user according to the target quantity and the number of targets;

[0113] Determining an initial coding sequence corresponding to the initial visual stimulation object, and displaying visual stimulation information corresponding to the initial visual stimulation object in a target interface according to the initial coding sequence;

[0114] Collecting target EEG data of the target user gazing at the target interface in real time;

[0115] Performing control state task identification on the target EEG data according to the target classification model to obtain a first predicted visual stimulation object corresponding to the target EEG data;

[0116] A target intention corresponding to the target user is determined according to the first predicted visual stimulation object.

[0117] In some implementations, the processor 301, in the process of determining the target classification model corresponding to the target user according to the target quantity and the target number, performs:

[0118] Determine the target classification quantity of the target classification model corresponding to the target user according to the target quantity and the number of targets;

[0119] Obtaining first EEG data corresponding to the initial visual stimulation object of the target user and obtaining second EEG data corresponding to the initial non-visual stimulation object of the target user;

[0120] Classification model training is performed according to the first EEG data, the second EEG data and the target classification quantity to obtain the target classification model corresponding to the target user.

[0121] In some implementations, the processor 301, in the process of performing control state task identification on the target EEG data according to the target classification model to obtain a first predicted visual stimulation object corresponding to the target EEG data, executes:

[0122] Determine a window length corresponding to the target window, and when the window length is greater than the data length corresponding to the target EEG data, continue to collect the target EEG data corresponding to the target user until the window length is less than or equal to the data length corresponding to the target EEG data;

[0123] When the window length is less than or equal to the data length corresponding to the target EEG data, data segmentation is performed on the target EEG data according to the target window and the window step to obtain the first window EEG data corresponding to the target EEG data;

[0124] The first predicted visual stimulation object corresponding to the first window EEG data is obtained by performing control state task identification on the first window EEG data according to the target classification model.

[0125] In some implementations, the processor 301, in the process of performing control state task identification on the first window EEG data according to the target classification model to obtain the first predicted visual stimulation object corresponding to the first window EEG data, executes:

[0126] Performing control state task recognition on the EEG data in the first window according to the target classification model to obtain a target recognition probability corresponding to the EEG data in the first window;

[0127] Determine a preset threshold, and compare the target recognition probability with the preset threshold, and when the target recognition probability is less than the preset threshold, delete the classification result of the first window EEG data, and there is no need to obtain the first predicted visual stimulation object corresponding to the first window EEG data;

[0128] When the target recognition probability is greater than or equal to the preset threshold, the first predicted visual stimulation object corresponding to the first window EEG data is determined according to the target recognition probability.

[0129] In some implementations, during the process of determining the target intention corresponding to the target user according to the first predicted visual stimulation object, the processor 301 executes:

[0130] Determine the fusion quantity and fusion rules;

[0131] When the first predicted number corresponding to the first predicted visual stimulation object is equal to the fused number, data fusion is performed on a plurality of the first predicted visual stimulation objects according to the fusion rule to obtain the target intention corresponding to the target user;

[0132] When the first predicted number corresponding to the first predicted visual stimulation object is less than the fused number, the target EEG data corresponding to the target user continues to be collected, and the target EEG data continues to be segmented according to the target window and the window step to obtain the second window EEG data;

[0133] According to the target classification model, control state task identification is performed on the second window EEG data to obtain the second predicted visual stimulation object corresponding to the second window EEG data, until the sum of the first predicted number corresponding to the first predicted visual stimulation object and the second predicted number corresponding to the second predicted visual stimulation object is equal to the fusion number, and then data fusion is performed on multiple first predicted visual stimulation objects and second predicted visual stimulation objects according to the fusion rule to obtain the target intention corresponding to the target user.

[0134] In some implementations, the processor 301, in the process of performing data fusion on the plurality of the first predicted visual stimulation objects according to the fusion rule to obtain the target intention corresponding to the target user, executes:

[0135] comparing the first predicted visual stimulation object with the initial non-visual stimulation object, and when the first predicted visual stimulation object belongs to the initial non-visual stimulation object, modifying the first predicted visual stimulation object to a resting state;

[0136] The target intention corresponding to the target user is obtained by performing data fusion using the fusion rule according to the modified first predicted visual stimulation object.

[0137] In some implementations, after obtaining the target intention, the processor 301 further executes:

[0138] Obtaining a target instruction corresponding to the target intention from a database according to the target intention;

[0139] Execute the target instruction so that the target user achieves the target intention according to the target instruction.

[0140] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the terminal device described above can refer to the corresponding process in the aforementioned embodiment of the intention recognition method based on brain-computer signals, and will not be repeated here.

[0141] An embodiment of the present invention also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any step of the method for intention recognition based on brain-computer signals provided in the description of the embodiment of the present invention.

[0142] The storage medium may be an internal storage unit of the terminal device described in the foregoing embodiment, such as a hard disk or memory of the terminal device. The storage medium may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc., equipped on the terminal device.

[0143] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0144] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0145] The serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for identifying intention based on EEG signals, characterized in that: The method comprises: Determining the number of targets corresponding to the initial visual stimulation objects of the control state task corresponding to the target user and the number of targets corresponding to the initial non-visual stimulation objects corresponding to the target user; Determine a target classification model corresponding to the target user according to the target quantity and the number of targets; Determining an initial coding sequence corresponding to the initial visual stimulation object, and displaying visual stimulation information corresponding to the initial visual stimulation object in a target interface according to the initial coding sequence; Collecting target EEG data of the target user gazing at the target interface in real time; Determine the window length corresponding to the target window, and when the window length is greater than the data length corresponding to the target EEG data, continue to collect the target EEG data corresponding to the target user until the window length is less than or equal to the data length corresponding to the target EEG data; when the window length is less than or equal to the data length corresponding to the target EEG data, perform data segmentation on the target EEG data according to the target window and the window step to obtain the first window EEG data corresponding to the target EEG data; perform control state task recognition on the first window EEG data according to the target classification model to obtain the first predicted visual stimulation object corresponding to the first window EEG data; Determine the fusion quantity and fusion rule; when the first predicted quantity corresponding to the first predicted visual stimulation object is equal to the fusion quantity, perform data fusion on multiple first predicted visual stimulation objects according to the fusion rule to obtain the target intention corresponding to the target user; when the first predicted quantity corresponding to the first predicted visual stimulation object is less than the fusion quantity, continue to collect the target EEG data corresponding to the target user, and continue to segment the target EEG data according to the target window and the window step to obtain the second window EEG data; perform control state task identification on the second window EEG data according to the target classification model to obtain the second predicted visual stimulation object corresponding to the second window EEG data, until the sum of the first predicted quantity corresponding to the first predicted visual stimulation object and the second predicted quantity corresponding to the second predicted visual stimulation object is equal to the fusion quantity, and then perform data fusion on multiple first predicted visual stimulation objects and the second predicted visual stimulation objects according to the fusion rule to obtain the target intention corresponding to the target user.

2. The method according to claim 1, characterized in that The determining the target classification model corresponding to the target user according to the target quantity and the number of targets includes: Determine the target classification quantity of the target classification model corresponding to the target user according to the target quantity and the number of targets; Obtaining first EEG data corresponding to the initial visual stimulation object of the target user and obtaining second EEG data corresponding to the initial non-visual stimulation object of the target user; Classification model training is performed according to the first EEG data, the second EEG data and the target classification quantity to obtain the target classification model corresponding to the target user.

3. The method according to claim 1, characterized in that The step of performing control state task identification on the first window EEG data according to the target classification model to obtain the first predicted visual stimulation object corresponding to the first window EEG data includes: Performing control state task recognition on the EEG data in the first window according to the target classification model to obtain a target recognition probability corresponding to the EEG data in the first window; Determine a preset threshold, and compare the target recognition probability with the preset threshold, and when the target recognition probability is less than the preset threshold, delete the classification result of the first window EEG data, and there is no need to obtain the first predicted visual stimulation object corresponding to the first window EEG data; When the target recognition probability is greater than or equal to the preset threshold, the first predicted visual stimulation object corresponding to the first window EEG data is determined according to the target recognition probability.

4. The method according to claim 1, characterized in that: The step of fusing data of the plurality of the first predicted visual stimulation objects according to the fusion rule to obtain the target intention corresponding to the target user includes: comparing the first predicted visual stimulation object with the initial non-visual stimulation object, and when the first predicted visual stimulation object belongs to the initial non-visual stimulation object, modifying the first predicted visual stimulation object to a resting state; The target intention corresponding to the target user is obtained by performing data fusion using the fusion rule according to the modified first predicted visual stimulation object.

5. The method according to any one of claims 1 to 4, characterized in that After obtaining the target intention, the method further includes: Obtaining a target instruction corresponding to the target intention from a database according to the target intention; Execute the target instruction so that the target user achieves the target intention according to the target instruction.

6. An intention recognition device based on EEG signals, characterized in that: include: A data determination module, used to determine the number of targets corresponding to the initial visual stimulation objects of the control state task corresponding to the target user and the number of targets corresponding to the initial non-visual stimulation objects corresponding to the target user; A model determination module, used to determine the target classification model corresponding to the target user according to the target quantity and the number of targets; A visual determination module, used to determine an initial coding sequence corresponding to the initial visual stimulation object, and display visual stimulation information corresponding to the initial visual stimulation object in a target interface according to the initial coding sequence; A data acquisition module, used for real-time acquisition of target EEG data corresponding to the target user gazing at the target interface; A data classification module is used to determine the window length corresponding to the target window. When the window length is greater than the data length corresponding to the target EEG data, the target EEG data corresponding to the target user is continuously collected until the window length is less than or equal to the data length corresponding to the target EEG data; when the window length is less than or equal to the data length corresponding to the target EEG data, data segmentation is performed on the target EEG data according to the target window and the window step to obtain the first window EEG data corresponding to the target EEG data; and control state task recognition is performed on the first window EEG data according to the target classification model to obtain the first predicted visual stimulation object corresponding to the first window EEG data; an intention determination module, used to determine a fusion quantity and a fusion rule; when the first predicted quantity corresponding to the first predicted visual stimulation object is equal to the fusion quantity, data fusion is performed on a plurality of the first predicted visual stimulation objects according to the fusion rule to obtain a target intention corresponding to the target user; When the first predicted number corresponding to the first predicted visual stimulation object is less than the fused number, the target EEG data corresponding to the target user continues to be collected, and the target EEG data continues to be segmented according to the target window and the window step to obtain the second window EEG data; the second window EEG data is subjected to control state task identification according to the target classification model to obtain the second predicted visual stimulation object corresponding to the second window EEG data, until the sum of the first predicted number corresponding to the first predicted visual stimulation object and the second predicted number corresponding to the second predicted visual stimulation object is equal to the fused number, and then the data of multiple first predicted visual stimulation objects and second predicted visual stimulation objects are fused according to the fusion rule to obtain the target intention corresponding to the target user.

7. A terminal device, characterized in that: The terminal device includes a processor and a memory; The memory is used to store computer programs; The processor is used to execute the computer program and implement the method for intention recognition based on electroencephalogram signals as described in any one of claims 1 to 5 when executing the computer program.

8. A computer storage medium for computer storage, characterized in that: The computer storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the method for intention recognition based on EEG signals as described in any one of claims 1 to 5.

Images