Electroencephalogram data and behavior data synchronization method, device and system and storage medium

Through the synchronous acquisition and three-dimensional reconstruction technology of multi-camera and EEG acquisition devices, the problem of synchronization between EEG data and behavioral data in natural state is solved, high-precision data collection and recording are achieved, and the ecological effectiveness and application of data are improved. Scene.

CN119055251APending Publication Date: 2024-12-03SHENZHEN INST OF ADVANCED TECH CHINESE ACAD OF SCI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411193856.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing EEG data and behavioral data acquisition systems are difficult to achieve high-precision synchronization in natural states, and cannot fully cover the subject's activity space, resulting in timing misalignment and missing information.

Method used

Multiple cameras and EEG acquisition devices are used to send acquisition signals and trigger signals through the sending module, obtain video data and EEG data, and perform three-dimensional reconstruction and synchronization processing to ensure the precise synchronization of behavioral data and EEG data.

Benefits of technology

It realizes high-precision synchronous acquisition and recording of EEG signals and video data in natural states, improves the ecological effectiveness of data and the broadness of application scenarios, and ensures the integrity and real-timeness of data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119055251A_ABST
    Figure CN119055251A_ABST
Patent Text Reader

Abstract

The invention discloses an electroencephalogram data and behavior data synchronization method, device and system and a storage medium, and the method is used for collecting a plurality of video data and electroencephalogram data of a subject through a plurality of camera devices and an electroencephalogram collection device. Sending acquisition signals to the plurality of camera devices and the electroencephalogram acquisition device at the same time, and periodically sending trigger signals until the acquisition is finished, so as to obtain a plurality of video data and electroencephalogram data of the subject; wherein the acquisition signal is used for indicating a plurality of camera devices and electroencephalogram acquisition devices to start acquisition at the same time acquisition point, and the trigger signal is used for indicating a new acquisition point; performing three-dimensional reconstruction on the plurality of acquired video data to acquire behavior data of the subject; and performing synchronous processing on the behavior data and the electroencephalogram data based on the recorded trigger signal sequence.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of neuroscience research, and in particular, relates to a method, device, system and storage medium for synchronizing EEG data and behavioral data. Background Art

[0002] In the study and analysis of human behavior, EEG data collection and behavioral data are two crucial technologies. EEG data (EEG, Electroencephalography) reflects the bioelectric signals generated by the brain during activity, which contain rich information, such as emotions, cognitive states, attention levels, etc. At the same time, behavioral data records a person's actual actions, reaction time, eye movements, etc., providing an important basis for understanding the relationship between human psychology and behavior.

[0003] At present, EEG data and behavioral data are mainly collected through EEG acquisition equipment and video recording equipment. The existing EEG and video acquisition systems mainly use monocular two-dimensional camera equipment and EEG acquisition equipment for low-precision synchronous acquisition. However, 1. It is difficult to achieve high-precision synchronization between traditional EEG acquisition equipment and video acquisition equipment, resulting in timing misalignment problems during the analysis of behavioral data; 2. Existing EEG and video acquisition systems are usually equipped with only a single or a small number of cameras, which cannot fully cover the subject's activity space, resulting in the lack of video information; 3. In addition, existing EEG and video acquisition systems are mostly used for data acquisition under static or restrictive conditions, and it is difficult to cope with the free activities of subjects in a natural state.

[0004] Therefore, the problem of synchronizing the subjects' EEG data and behavioral data in a natural state needs to be solved urgently. Summary of the invention

[0005] The present application provides a method, device, system and storage medium for synchronizing EEG data and behavioral data, aiming to solve the problem that the EEG data and behavioral data of the subject in the natural state cannot be accurately synchronized in the existing methods.

[0006] In order to solve the above technical problems, in the first aspect, the present application provides a method for synchronizing EEG data and behavioral data, the method comprising:

[0007] In a second aspect, the present application provides an apparatus for processing EEG data and behavioral data, the apparatus comprising:

[0008] A sending module, used to send acquisition signals to multiple camera devices and EEG acquisition devices at the same time and periodically send trigger signals in response to a start instruction; and also used to send end signals to multiple camera devices and EEG acquisition devices at the same time when a set condition is met or in response to an end instruction;

[0009] An acquisition module, used for acquiring multiple video data and EEG data of a subject;

[0010] A reconstruction module, used for performing three-dimensional reconstruction on the acquired multiple video data to obtain the behavior data of the subject;

[0011] A synchronization module is used to synchronize the behavior data and the EEG data based on the recorded trigger signal sequence.

[0012] In a third aspect, the present application provides an EEG data and behavior data processing system, the system comprising a camera unit, an EEG acquisition device, a synchronization unit and a host computer;

[0013] The camera unit includes a plurality of cameras for collecting a plurality of video data of the subject;

[0014] The EEG acquisition unit is used to acquire EEG data of the subject;

[0015] The host computer is used to respond to a start instruction, send a collection signal to the multiple cameras and the EEG collection device at the same time and periodically send a trigger signal to obtain multiple video data and EEG data of the subject; and perform three-dimensional reconstruction on the multiple video data obtained to obtain the behavior data of the subject; and when a set condition is met or in response to an end instruction, send an end signal to the multiple cameras and the EEG collection device at the same time; and synchronize the behavior data and the EEG data based on the recorded trigger signal sequence;

[0016] The synchronization unit is used to receive the trigger signal and synchronously send the trigger signal to the EEG acquisition device and the multiple cameras.

[0017] In a fourth aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned method for synchronizing EEG data and behavioral data when executing the computer program.

[0018] In a fifth aspect, a storage medium stores computer-readable instructions thereon, wherein the computer-readable instructions are executed by one or more processors to implement the method for synchronizing EEG data and behavioral data as described above.

[0019] In the sixth aspect, a computer program product includes computer-readable instructions, the computer-readable instructions are stored in a storage medium, one or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements the method for synchronizing EEG data and behavioral data as described above.

[0020] Compared with the existing technology, the EEG data and behavioral data synchronization method of the present application can synchronously collect and record the EEG signals and videos of the human body under natural behavior, thereby improving the ecological validity of the data and the breadth of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic diagram of the main process of a method for synchronizing EEG data and behavioral data provided in an embodiment of the present application;

[0023] Figure 2 The use scenario intention of a system for synchronizing EEG data and behavioral data provided in an embodiment of the present application;

[0024] Figure 3 A structural block diagram of a device for synchronizing EEG data and behavioral data provided in an embodiment of the present application;

[0025] Figure 4 A structural block diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present disclosure refers to the presence of the features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0028] See also Figure 1 , Figure 1 The main flow diagram of the method for synchronously collecting EEG data and behavioral data shown in the embodiment of the present application is a schematic diagram of the main flow diagram of the method for synchronously collecting EEG data and behavioral data shown in the embodiment of the present application, which is used to collect multiple video data and EEG data of a subject through multiple camera devices and EEG acquisition devices; wherein the camera device can be various types of cameras, and the EEG acquisition device can be a wirelessly connected EEG cap or a wired EEG instrument, etc. The method process includes the following steps S100 to S300:

[0029] S100, in response to a start instruction, simultaneously sending acquisition signals to multiple camera devices and EEG acquisition devices and periodically sending trigger signals until the acquisition is completed, so as to obtain multiple video data and EEG data of the subject.

[0030] The acquisition signal is used to instruct the multiple camera devices and the EEG acquisition device to start acquisition at the same acquisition point in time, and the trigger signal is used to indicate a new acquisition point;

[0031] In this embodiment, multiple cameras are used to obtain video data of subjects in a natural state; preferably, eight cameras arranged around the subjects can be used to complete the high-speed, synchronous video shooting of eight cameras, provide all-round, multi-angle video recording, and can fully capture all behavioral details of the subjects in a free state, providing more comprehensive data support for subsequent behavioral analysis. At the same time, the frame rate reaches 160 frames, the image quality reaches 1280P resolution, and each camera device is accurately synchronized. The acquisition control program is used to realize the acquisition, management and processing of EEG data and video data. Specifically, when the start acquisition instruction input by the operator is received, the system will simultaneously send a start acquisition signal to multiple cameras and EEG acquisition devices to ensure that all cameras collect data at the same time, and will also periodically send a trigger signal as the start of a new acquisition segment.

[0032] In an exemplary embodiment, step S100 specifically includes steps S110 to S130:

[0033] S110, in response to the start instruction, establishing a main control process and multiple auxiliary processes; each of the camera device and the EEG acquisition device corresponds to an auxiliary process;

[0034] S120, sending acquisition signals to multiple camera devices and EEG acquisition devices simultaneously and periodically sending trigger signals through the main control process;

[0035] S130, respectively acquiring the video data and the EEG data through the multiple auxiliary processes.

[0036] During the data acquisition process, the acquisition control program uses multiple processes to manage and control the acquisition work of different cameras and EEG acquisition devices. Each camera corresponds to an independent process to acquire video frames; the master control process coordinates the synchronous recording of each camera and EEG acquisition device by controlling the sending of trigger signals. The memory queue is used to process the data stream of the camera and EEG acquisition device to prevent frame loss due to processing delays or bandwidth limitations. The data of each camera is passed to the saving process through a shared queue to ensure the real-time and integrity of the data. The acquisition control program continuously monitors the data flow during the acquisition process, dynamically adjusts the buffer and processing parameters, and ensures the stability of the acquisition process.

[0037] The acquisition control program uses multi-threading and multi-process mechanisms to send synchronization signals to all cameras and EEG acquisition devices, ensuring that each camera receives and processes the synchronization signal at the same time point, thereby achieving synchronous acquisition of EEG signals and videos.

[0038] In one implementation of the present embodiment, a trigger signal is sent every time the camera device collects a certain number of frames, such as 200 frames, 300 frames, etc. The specific number of frames can be adjusted according to the design. When the set conditions are met or in response to the end instruction, an end signal is sent to multiple camera devices and EEG acquisition devices at the same time. Specifically, when the host records a specified number of frames or the recording reaches a specified duration or a termination signal is received; the specified number of frames or the specified duration is not excessively restricted here. The acquisition control program sends a trigger signal indicating the end to multiple camera devices and the EEG acquisition device, terminates the acquisition process, releases multi-camera resources to end the acquisition. The end recording part mainly involves the acquisition control program sending an end command to multiple imaging devices and EEG acquisition devices after enough frames are collected or a termination signal is received, terminating the process, releasing resources and ending.

[0039] S200, performing three-dimensional reconstruction on the acquired multiple video data to obtain behavioral data of the subject.

[0040] In an exemplary embodiment, step S200 specifically includes the following steps S210 to S230:

[0041] S210, calibrating the multiple camera devices to determine parameters of each camera device;

[0042] In this embodiment, the purpose of multi-camera calibration is to obtain the intrinsic parameters and spatial position distribution of the camera equipment to provide a parameter basis for three-dimensional reconstruction. It mainly uses Zhang's calibration method to make calibration plates to estimate the pose of multiple cameras.

[0043] S220, extracting two-dimensional skeleton points of the subject from the multiple video data;

[0044] In this embodiment, a trained neural network model is used to identify and track the two-dimensional skeleton points in the video data. Specifically, the calibration plate is placed within the field of view of the camera device, and the image of the calibration plate is captured from multiple angles and positions to ensure that each camera can capture a sufficient number and quality of calibration plate images. The number of points on the calibration plate of each camera device is 7*4, and the size of the points on the chessboard is customized according to the shooting space. Each camera device takes 200 calibration plate images at different angles, 8 for a total of 1600, and then uploads them to the workspace, and uses the StereoCameraCalibratorGUI toolbox for calibration operations to obtain the internal and external parameters of the camera, including the rotation matrix R and the translation vector T, and save them locally. The yolov8 deep learning network model is used to train and predict the video, perform accurate two-dimensional human posture estimation, and use the calibrated camera parameters to perform three-dimensional reconstruction of 17 body points using the triangulation algorithm.

[0045] S230, constructing a three-dimensional model of the subject based on the camera parameters and the two-dimensional skeleton points of the subject to obtain the subject's behavior data.

[0046] The two-dimensional body bone points of the subject are mapped to three-dimensional space through a computer vision triangulation algorithm to obtain a three-dimensional skeleton model of the human body, and the subject's behavioral data can be calculated based on the three-dimensional skeleton model.

[0047] S300: Synchronously process the behavior data and the EEG data based on the recorded trigger signal sequence.

[0048] In an exemplary embodiment, step S300 specifically includes the following steps S310 to S320:

[0049] S310, checking the correctness of the trigger signal sequence;

[0050] First, the received trigger signal sequence needs to be checked for correctness. This includes confirming whether the signal is complete, whether there are any missing or erroneous trigger points, and whether the timestamp of the trigger point is accurate. This step is the basis for ensuring the reliability of the subsequent synchronization process.

[0051] S320, cutting and splicing the EEG data according to the trigger signal sequence to achieve synchronization between the behavior data and the EEG data.

[0052] After confirming that the trigger signal sequence is correct, the next step is to crop and splice the original EEG signals according to these trigger signals. Specifically, the EEG signals are divided into segments corresponding to specific events or time points, and then these segments are rearranged or spliced ​​according to the experimental design to form a continuous signal consistent with the experimental process. The key to this step is to ensure that the start and end time of each segment corresponds precisely to the timestamp of the trigger signal. This achieves high-precision synchronization of EEG signals and multiple cameras, ensuring the consistency of data in the time dimension.

[0053] The EEG data and behavior data synchronization method of the present application can collect and record the EEG signals and videos of the human body under natural behavior, thereby improving the ecological validity of the data and the breadth of application scenarios.

[0054] like Figure 3 As shown, the present application also provides an EEG data and behavior data processing device 200, which includes a sending module 201, an acquisition module 202, a reconstruction module 203 and a synchronization module 204; the sending module 201 is used to respond to a start instruction, and simultaneously send an acquisition signal to multiple cameras and EEG acquisition devices and periodically send a trigger signal; it is also used to send an end signal to multiple cameras and EEG acquisition devices when a set condition is met or in response to an end instruction; the acquisition module 202 is used to acquire multiple video data and EEG data of a subject; the reconstruction module 203 is used to perform three-dimensional reconstruction on the acquired multiple video data to acquire the behavior data of the subject; the synchronization module 204 is used to synchronize the behavior data and the EEG data based on the recorded trigger signal sequence.

[0055] In one embodiment, the sending module includes an establishing unit, a sending unit, and an acquiring unit. The establishing unit is used to establish a main control process and multiple auxiliary processes in response to a start instruction; each of the imaging devices and EEG acquisition devices corresponds to an auxiliary process; the sending unit is used to send acquisition signals to multiple imaging devices and EEG acquisition devices at the same time through the main control process and periodically send trigger signals.

[0056] In one embodiment, the reconstruction module includes a calibration unit, an extraction unit and a construction unit; the calibration unit is used to calibrate the multiple camera devices to determine the parameters of each camera device; the extraction unit is used to extract the two-dimensional skeleton points of the subject from the multiple video data; the construction unit is used to construct a three-dimensional model of the subject based on the parameters of each camera device and the two-dimensional skeleton points of the subject to obtain the subject behavior data.

[0057] In one embodiment, the synchronization module includes a detection unit and a splicing unit. The detection unit is used to check the correctness of the trigger signal sequence; the splicing unit is used to crop and splice the EEG data according to the trigger signal sequence to achieve synchronization between the behavioral data and the EEG data.

[0058] like Figure 2 As shown, the present application also provides an EEG data and behavior data processing system, the system includes a camera unit, an EEG acquisition device, a synchronization unit and a host computer 104; the camera unit includes multiple cameras 102, which are used to acquire multiple video data of a subject; the EEG acquisition unit is used to acquire the EEG data of the subject; the host computer 104 is used to respond to a start instruction, send an acquisition signal to the multiple cameras 102 and the EEG acquisition device at the same time and periodically send a trigger signal to acquire multiple video data and EEG data of the subject; and perform three-dimensional reconstruction on the acquired multiple video data to acquire the behavior data of the subject; and when a set condition is met or in response to an end instruction, send an end signal to the multiple cameras 102 and the EEG acquisition device at the same time; and synchronize the behavior data and the EEG data based on the recorded trigger signal sequence; the synchronization unit is used to receive the trigger signal and synchronously send the trigger signal to the EEG acquisition device and the multiple cameras 102.

[0059] Optionally, the synchronization unit includes an intelligent synchronization center 107 and a multi-parameter synchronizer 108 。

[0060] Optionally, the EEG acquisition unit includes an EEG cap 105 and a signal amplifier 106, and the EEG acquisition unit is wirelessly connected to the synchronization unit.

[0061] The EEG cap 105 includes non-invasive EEG electrodes, electrode wires and a cap for fixing, and is used to collect weak electrical signals generated by brain activity. The signal amplifier 106, which is used to obtain EEG potential information and transmit the amplified signal to the receiving device by wireless means, is placed at the rear end of the EEG cap 105 and can be fixed on the EEG cap 105 by magnetic force when the interface is matched.

[0062] Optionally, the camera unit includes at least eight cameras 102 arranged circumferentially around the subject.

[0063] Specifically, eight cameras 102 are installed on a fixed bracket 101. The bracket can be fixed on the top of any ceiling, plate, etc., or placed on the ground for easy disassembly. Its size is not fixed and can be adjusted according to actual needs. The eight cameras 102 of the same model all have USB3.0 interfaces. Among the eight cameras 102, four cameras 102 are installed in the middle of the four columns of the camera 102 fixing bracket, and their orientation is horizontal and slightly tilted downward; the other four cameras 102 are installed in the four corners to ensure that the camera 102 captures complete human behavior information in its field of view. The camera's field of view is toward the shooting plane 103, and the subjects move freely in the shooting plane 103 or complete the experimental tasks according to the experimental paradigm.

[0064] See also Figure 4 In an embodiment of the present application, an electronic device 4000 is provided, and the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0065] The data interaction between the processor 4001 and the memory 4003 can be realized through at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0066] Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0067] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0068] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program instructions or codes in the form of instructions or data structures and can be accessed by the electronic device 4000, but is not limited to this.

[0069] The memory 4003 stores computer-readable instructions, and the processor 4001 can read the computer-readable instructions stored in the memory 4003 through the communication bus 4002 .

[0070] The computer-readable instructions are executed by one or more processors 4001 to implement the EEG data and behavioral data synchronization acquisition system in the above-mentioned embodiments.

[0071] In addition, a storage medium is provided in an embodiment of the present application, on which computer-readable instructions are stored, and the computer-readable instructions are executed by one or more processors to implement the above-mentioned method for synchronizing EEG data and behavioral data.

[0072] A computer program product is provided in an embodiment of the present application. The computer program product includes computer-readable instructions. The computer-readable instructions are stored in a storage medium. One or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements the above-mentioned method for synchronizing EEG data and behavioral data.

[0073] Compared with related technologies, the EEG data and behavioral data synchronization method of the present application can collect and record EEG signals and videos of human body in natural behavior synchronously, which improves the ecological validity of data and the breadth of application scenarios. It can prevent frame loss during high-speed recording. It integrates efficient data processing and storage modules, which can quickly process and synchronously store large-scale video and EEG data to ensure data integrity and real-time performance. It can achieve high-precision synchronization of EEG signals and multiple cameras to ensure data consistency in the time dimension. It can complete camera calibration and three-dimensional reconstruction of human skeleton points.

[0074] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0075] The present application is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present application. In addition, under the teachings of the present application, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present application. Therefore, the present application is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of the present application are within the scope protected by the present application.

Claims

1. A method for synchronizing EEG data and behavioral data, characterized in that: The method is used to collect multiple video data and EEG data of a subject through multiple camera devices and EEG acquisition devices, and the method includes: In response to a start instruction, a collection signal is sent to multiple camera devices and EEG collection devices at the same time and a trigger signal is periodically sent until the collection is completed, so as to obtain multiple video data and EEG data of the subject; wherein the collection signal is used to instruct the multiple camera devices and the EEG collection devices to start collecting at the same time collection point, and the trigger signal is used to indicate a new collection point; Performing three-dimensional reconstruction on the acquired multiple video data to obtain behavioral data of the subject; The behavioral data and the EEG data are synchronously processed based on the recorded trigger signal sequence.

2. The method according to claim 1, characterized in that In response to a start instruction, a collection signal is sent to multiple camera devices and EEG collection devices at the same time and a trigger signal is periodically sent until the collection is completed, so as to obtain multiple video data and EEG data of the subject; including: In response to the start instruction, a main control process and a plurality of auxiliary processes are established; each of the camera device and the EEG acquisition device corresponds to an auxiliary process; The main control process simultaneously sends acquisition signals to multiple camera devices and EEG acquisition devices and periodically sends trigger signals; The video data and EEG data are obtained respectively through the multiple auxiliary processes.

3. The method according to claim 1, characterized in that The periodically sending the trigger signal includes sending a trigger signal every time the camera device captures a set number of frames.

4. The method according to claim 1, characterized in that: The synchronous processing of the behavior data and the EEG data based on the recorded trigger signal sequence includes: Performing a correctness check on the trigger signal sequence; The EEG data is trimmed and spliced ​​according to the trigger signal sequence to achieve synchronization between the behavior data and the EEG data.

5. The method according to claim 2, characterized in that: The three-dimensional reconstruction of the acquired multiple video data to obtain the behavior data of the subject includes: Calibrate the multiple camera devices to determine parameters of each camera device; Extracting two-dimensional skeleton points of the subject from the plurality of video data; A three-dimensional model of the subject is constructed based on the parameters of each of the camera devices and the two-dimensional skeleton points of the subject to obtain the subject's behavior data.

6. An electroencephalogram data and behavior data processing device, characterized in that: The device comprises: A sending module, used to send acquisition signals to multiple camera devices and EEG acquisition devices at the same time and periodically send trigger signals in response to a start instruction; and also used to send end signals to multiple camera devices and EEG acquisition devices at the same time when a set condition is met or in response to an end instruction; An acquisition module, used for acquiring multiple video data and EEG data of a subject; A reconstruction module, used for performing three-dimensional reconstruction on the acquired multiple video data to obtain the behavior data of the subject; A synchronization module is used to synchronize the behavior data and the EEG data based on the recorded trigger signal sequence.

7. A system for processing EEG data and behavioral data, characterized in that: The system includes a camera unit, an EEG acquisition device, a synchronization unit and a host computer; The camera unit includes a plurality of cameras for collecting a plurality of video data of the subject; The EEG acquisition unit is used to acquire EEG data of the subject; The host computer is used to respond to a start instruction, send a collection signal to the multiple cameras and the EEG collection device at the same time and periodically send a trigger signal to obtain multiple video data and EEG data of the subject; and perform three-dimensional reconstruction on the multiple video data obtained to obtain the behavior data of the subject; and when a set condition is met or in response to an end instruction, send an end signal to the multiple cameras and the EEG collection device at the same time; and synchronize the behavior data and the EEG data based on the recorded trigger signal sequence; The synchronization unit is used to receive the trigger signal and synchronously send the trigger signal to the EEG acquisition device and the multiple cameras.

8. The system according to claim 7, characterized in that The EEG acquisition unit includes an EEG cap and a signal amplifier, and the EEG acquisition unit is wirelessly connected to the synchronization unit.

9. The system according to claim 7, characterized in that The camera unit includes at least eight cameras arranged circumferentially around the subject.

10. A storage medium, characterized in that: The storage medium stores computer-readable instructions, which are executed by one or more processors to implement the method for synchronizing EEG data and behavioral data as described in any one of claims 1 to 5.