Objective behavior marking method and system based on brain-computer interface equipment

Abstract

The invention discloses an objective behavior marking method and system based on brain-computer interface equipment, relates to the technical field of brain-computer interfaces, and solves the problem that a brain-computer interface cannot specifically distinguish whether a user performs a tense working behavior or solves a difficult learning behavior, so that the brain-computer interface cannot provide accurate personalized feedback. The objective behavior marking system based on the brain-computer interface device comprises an electroencephalogram collection module which comprises an electroencephalogram collection device, and the electroencephalogram collection device is worn at the head position of a user. According to the system, after a user collects the electroencephalogram information through the electroencephalogram collection module, the user can label the electroencephalogram information in each time period through the intelligent mobile terminal, and the electroencephalogram information is compared and matched with the electroencephalogram data in the electroencephalogram database through the data association module, so that the subsequent electroencephalogram data of the user is automatically classified, and the user experience is improved. Therefore, the current behavior of the user is specifically distinguished, and the brain-computer interface can provide subsequent accurate personalized feedback.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interfaces, specifically to an objective behavior annotation method and system based on brain-computer interface devices. Background Technology

[0002] Brain-computer interface (BCI) refers to a direct connection created between the brain of a person or animal and an external device, enabling information exchange between the brain and the device. While the concept has existed for some time, significant progress only began to emerge after the 1990s. BCI technology is a revolutionary human-computer interaction technology. Its mechanism of action bypasses peripheral nerves and muscles, directly establishing a new communication and control channel between the brain and external devices. It achieves information transmission and control by capturing brain signals and converting them into electrical signals. Existing brain-computer interfaces can only infer the user's abstract physiological state based on EEG signals, such as a high focus index or anxiety characteristics. They cannot specifically distinguish whether the user is engaged in stressful work or solving difficult learning behaviors, which makes it impossible for brain-computer interfaces to provide accurate personalized feedback or conduct effective behavior analysis. Therefore, they do not meet the current needs. To address this, we propose an objective behavior annotation method and system based on brain-computer interface devices. Summary of the Invention

[0003] The purpose of this invention is to provide an objective behavior annotation method and system based on brain-computer interface devices, in order to solve the problem in the background art that brain-computer interfaces can only infer the user's abstract physiological state based on electroencephalogram (EEG) signals, such as a high focus index or anxiety characteristics, and cannot specifically distinguish whether the user is engaging in stressful work behavior or solving difficult learning behavior, thus causing the brain-computer interface to be unable to provide accurate personalized feedback or perform effective behavior analysis.

[0004] To achieve the above objectives, the present invention provides the following technical solution: an objective behavior annotation system based on a brain-computer interface device, comprising: The EEG acquisition module includes an EEG acquisition device, which is worn on the user's head and used to acquire the user's EEG signals. The annotation module, installed as an app on a smart mobile terminal, includes a tag management unit that provides an annotation interface to receive user-input behavioral events and scene tags, and records the annotation time period. Users can also customize, name, and expand the tags. The data association module is used to associate the annotation information input by the user through the annotation module with the EEG signals of the time period in which the annotation information is located, which are collected by the EEG acquisition module. The storage module is used to store and record EEG data with behavioral events and scene labels generated by the data association module.

[0005] Preferably, the data association module is deployed on a backend server, and the backend server is configured as follows: Receive labeled information with behavioral events and scene tags sent by smart mobile terminals; Based on the time period indicated by the annotation information, query the EEG signal data for that time period in the storage module; The retrieved EEG signal data is bound to the received behavioral events and scene label information to form structured data records and stored in the storage module.

[0006] A method for labeling an objective behavior labeling system based on a brain-computer interface device includes the following steps: S1: Collects the user's brainwave signals through a brain-computer interface device, determines the user's current abstract emotional state based on the attention analysis formula, and records the brainwave signal information for each time period. S2: Users open their smart mobile terminals and view the EEG signal information for each time period through the tag management unit. They then label the behavioral events and scene tags by custom editing, naming, and expanding them. S3: Receives annotation information with behavioral event and scene tags, as well as EEG information for that time period, input by the user through the annotation interface via the data association module; S4: Associate the user-input annotation information with behavioral event and scene labels with the EEG information of that time period to generate an EEG data record with specific information labels; S5: Establish an EEG database through the backend server, and summarize and record the EEG data records with specific information tags generated in step S4. Preferably, the behavioral event labels include predefined first category labels and user-defined second category labels; The scene labels include predefined third-category labels and user-defined fourth-category labels.

[0007] Preferably, the behavior event label has a hierarchical structure, including a parent behavior category and child specific behavior items.

[0008] Preferably, the parent behavior category includes learning behavior and work behavior; The specific sub-behaviors under the learning behaviors include at least reading, programming, and listening to lectures; The specific sub-behaviors under the work behaviors include at least meetings, documents, and presentations; The scene tags include at least home, office, and outdoor.

[0009] Preferably, step S2 further includes the following steps: S2.1: Provide trigger controls after displaying the analysis results of EEG signal information for each time period; S2.2: Users can click the trigger control to jump to the annotation interface and make detailed annotations based on the parent behavior category and child specific behavior items of the behavior event label; S2.3: Users can also select their location information for that time period based on scene tags.

[0010] Preferably, step S4 further includes the following steps: S4.1: Receive annotation information with behavioral event and scene labels and EEG information for that time period input by the user through the annotation interface; S4.2: Based on the EEG signals and time periods in the user-annotated information, search and match the corresponding EEG signal data for the emotion and time period in the stored EEG database; S4.3: Bind and store the matched EEG signal data with behavioral events and scene tags to form EEG data records with specific information tags, and upload them back to the EEG database for storage.

[0011] Compared with the prior art, the beneficial effects of the present invention are: This invention, through the cooperation of an EEG acquisition module and a data association module, enables users to annotate EEG information for each time period via a smart mobile terminal after the EEG acquisition module collects EEG information. This annotation is then detailed using behavioral events and scene tags. The data association module compares and matches this data with EEG data in a database containing specific information tags, automatically classifying and organizing the user's subsequent EEG data. This allows for the precise differentiation of the user's current behavior, enabling the brain-computer interface to provide accurate and personalized feedback and providing samples for iterative upgrades of the brain-computer interface algorithm. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating the system of the present invention. Detailed Implementation

[0013] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0014] Please see Figure 1 One embodiment of the present invention provides: an objective behavior annotation system based on a brain-computer interface device, comprising: The EEG acquisition module includes an EEG acquisition device, which is worn on the user's head to acquire the user's EEG signals. The annotation module, installed as an app on a smart mobile terminal, includes a tag management unit that provides an annotation interface to receive user-input behavioral events and scene tags, and records the annotation time period. Users can also customize, name, and expand the tags. The data association module is used to associate the annotation information input by the user through the annotation module with the EEG signals of the time period in which the annotation information is located, which are collected by the EEG acquisition module. The storage module is used to store and record EEG data with behavioral events and scene labels generated by the data association module.

[0015] The data association module is deployed on a backend server, which is configured as follows: Receive labeled information with behavioral events and scene tags sent by smart mobile terminals; Based on the time period indicated by the annotation information, query the EEG signal data for that time period in the storage module; The retrieved EEG signal data is bound to the received behavioral events and scene label information to form structured data records and stored in the storage module.

[0016] A method for labeling an objective behavior labeling system based on a brain-computer interface device includes the following steps: S1: Collects the user's brainwave signals through a brain-computer interface device, determines the user's current abstract emotional state based on the attention analysis formula, and records the brainwave signal information for each time period. S2: Users open their smart mobile terminals and view the EEG signal information for each time period through the tag management unit. They then label the behavioral events and scene tags by custom editing, naming, and expanding them. S3: Receives annotation information with behavioral event and scene tags, as well as EEG information for that time period, input by the user through the annotation interface via the data association module; S4: Associate the user-input annotation information with behavioral event and scene labels with the EEG information of that time period to generate an EEG data record with specific information labels; S5: Establish an EEG database through the backend server, and summarize and record the EEG data records with specific information tags generated in step S4. Behavioral event labels include predefined first-category labels and user-defined second-category labels; Scene labels include predefined third-category labels and user-defined fourth-category labels.

[0017] Behavioral event tags have a hierarchical structure, including parent behavioral categories and child specific behavioral items.

[0018] Parent behavior categories include learning behavior and work behavior; The specific sub-behaviors under learning behaviors include at least reading, programming, and listening to lectures; The specific sub-behaviors under work behaviors should at least include meetings, documents, and presentations; Scene tags should include at least home, office, and outdoor.

[0019] Step S2 also includes the following steps: S2.1: Provide trigger controls after displaying the analysis results of EEG signal information for each time period; S2.2: Users can click the trigger control to jump to the annotation interface and make detailed annotations based on the parent behavior category and child specific behavior items of the behavior event label; S2.3: Users can also select their location information for that time period based on scene tags.

[0020] Step S4 also includes the following steps: S4.1: Receive annotation information with behavioral event and scene labels and EEG information for that time period input by the user through the annotation interface; S4.2: Based on the EEG signals and time periods in the user-annotated information, search and match the corresponding EEG signal data for the emotion and time period in the stored EEG database; S4.3: Bind and store the matched EEG signal data with behavioral events and scene tags to form EEG data records with specific information tags, and upload them back to the EEG database for storage.

[0021] The annotation steps of the above-mentioned annotation method for the objective behavior annotation system based on brain-computer interface devices are specifically presented in the following flow: I. Behavior Labeling Process: When users encounter difficulties during programming debugging, they can use a brain-computer interface device to detect EEG data. Then, the user can open a mobile app to view the data and click the "Unannotate" button, at which point the annotation interface will pop up. II. Mobile Terminal Operation Process: Select the event and scenario from the annotation interface; Select the "Learn" button for the event; Select the "Office" button for the scene; III. Data Association Process: The time period during which the app submitted the user's application: Start time T0 to end time T1; After uploading the data, the backend searches for EEG data, finds the EEG data for the corresponding time period, and correlates the two.

[0022] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. An objective behavior annotation system based on a brain-computer interface device, characterized by: The application relates to a brain electrical signal data recording method and system. The system comprises: a brain electrical signal acquisition module comprising a brain electrical signal acquisition device, which is worn on a user's head position and used for acquiring the user's brain electrical signal; a labeling module in the form of an APP installed on a smart mobile terminal, comprising a label management unit, which is used for providing a labeling interface to receive a user's input behavior event and scene label and record a labeling time period, and meanwhile the user can customize, name and expand the label; a data association module, which is used for associating the labeling information input by the user through the labeling module with the brain electrical signal of the time period where the labeling information is located, which is acquired by the brain electrical signal acquisition module; 2. The objective behavior annotation system based on brain-computer interface device according to claim 1, characterized in that: a storage module, which is used for storing and recording the brain electrical data with the behavior event and scene label generated through the data association module. The data association module is arranged on a backend server, and the backend server is configured to: receive the labeling information with the behavior event and scene label sent by the smart mobile terminal; query the brain electrical signal data of the time period in the storage module according to the time period of the labeling information; 3. The labeling method of the objective behavior labeling system based on the brain-computer interface device according to any one of claims 1 to 2, characterized in that: bind the queried brain electrical signal data and the received behavior event and scene label information, form a structured data record and store the record in the storage module. The method comprises the following steps: S1: acquiring the user's brain electrical signal through a brain-computer interface device, determining the user's current abstract emotional state according to a concentration degree analysis formula and recording the brain electrical signal information of each time period; S2: the user opens the smart mobile terminal and views the brain electrical signal information of each time period through the label management unit, and then labels each behavior event and scene label through customized editing, naming and expansion; S3: receiving the labeling information with the behavior event and scene label input by the user through the labeling interface and the brain electrical information of the time period through the data association module; S4: associating the labeling information with the behavior event and scene label input by the user and the brain electrical information of the time period, generating brain electrical data records with specific information labels; 4. The marking method of the objective behavior marking system based on the brain-computer interface device according to claim 3, characterized in that: S5: establishing a brain electrical database through the backend server and recording the brain electrical data records with specific information labels generated in step S4. The behavior event label comprises a first category label predefined and a second category label customized by the user; 5. The labeling method of the objective behavior labeling system based on the brain-computer interface device according to claim 6, characterized in that: The scene label comprises a third category label predefined and a fourth category label customized by the user.

6. The labeling method of the objective behavior labeling system based on the brain-computer interface device according to claim 7, characterized in that: The behavior event label has a hierarchical structure, comprising a parent behavior category and a child specific behavior item. The parent behavior category comprises a learning behavior and a working behavior; The child specific behavior item under the learning behavior comprises at least reading, programming and listening to a class; The child specific behavior item under the working behavior comprises at least a meeting, a document and a demonstration; 7. The labeling method of the objective behavior labeling system based on the brain-computer interface device according to claim 3, characterized in that: The scene label comprises at least home, office and outdoor. Step S2 further comprises the following steps: S2.1: after displaying the analysis result of the brain electrical signal information of each time period, a trigger control is provided; S2.2: the user jumps to the labeling interface by clicking the trigger control and labels in detail according to the parent behavior category and child specific behavior item of the behavior event label; S2.3: the user can also select the place information of the time period according to the scene label.

8. The labeling method of the objective behavior labeling system based on the brain-computer interface device according to claim 3, characterized in that: Step S4 further comprises the following steps: S4.1: receiving the labeling information with the behavior event and the scene label input by the user through the labeling interface and the electroencephalogram information of the time period; S4.2: searching and matching the electroencephalogram signal data corresponding to the emotion and the time period in the stored electroencephalogram database according to the electroencephalogram signal and the time period in the user labeling information; S4.3: binding and storing the matched electroencephalogram signal data with the behavior event and the scene label to form an electroencephalogram data record with specific information labels and uploading back to the electroencephalogram database for storage record.

Images