Method, apparatus, and system for synchronization between electroencephalogram data and behavioral data, and storage medium

Abstract

Disclosed in the present application are a method, apparatus, and system for synchronization between electroencephalogram data and behavioral data, and a storage medium. The method is used to acquire a plurality of video data and electroencephalogram data of a subject by means of a plurality of camera devices and an electroencephalogram acquisition device. The method comprises: in response to a start instruction, simultaneously sending acquisition signals to the plurality of camera devices and the electroencephalogram acquisition device, and periodically sending trigger signals until the acquisition ends, so as to obtain the plurality of video data and electroencephalogram data of the subject, wherein the acquisition signals are used to instruct the plurality of camera devices and the electroencephalogram acquisition device to start acquisition at the same acquisition time point, and the trigger signals are used to indicate a new acquisition time point; performing three-dimensional reconstruction on the acquired plurality of video data to obtain behavioral data of the subject; and performing synchronization processing on the behavioral data and the electroencephalogram data on the basis of a recorded sequence of the trigger signals.

Description

Methods, devices, systems, and storage media for synchronizing EEG data with behavioral data Technical Field

[0001] This application belongs to the field of neuroscience research, and in particular relates to a method, device, system and storage medium for synchronizing electroencephalogram (EEG) data and behavioral data. Background Technology

[0002] In the analysis of human behavior, electroencephalography (EEG) data acquisition and behavioral data are two crucial technologies. EEG data reflects the bioelectrical signals generated by the brain during activity, containing a wealth of information such as emotion, cognitive state, and attention level. Meanwhile, behavioral data records a person's actual movements, reaction time, eye movements, etc., providing important evidence for understanding the relationship between human psychology and behavior.

[0003] Currently, EEG and behavioral data are primarily collected using EEG acquisition devices and video recording devices. Existing EEG and video acquisition systems mainly rely on low-precision synchronous acquisition between monocular two-dimensional cameras and EEG acquisition devices. However, 1. Traditional EEG acquisition devices and video acquisition devices struggle to achieve high-precision synchronization, leading to timing misalignment issues during behavioral data analysis; 2. Existing EEG and video acquisition systems typically only have a single or limited number of cameras, failing to fully cover the subject's activity space, resulting in missing video information; 3. Furthermore, existing EEG and video acquisition systems are mostly used for data acquisition under static or restricted conditions, making it difficult to handle subjects' free movement in a natural state.

[0004] Therefore, the problem of synchronizing EEG and behavioral data of subjects in a natural state urgently needs to be solved.

[0005] Summary of the Invention

[0006] This application provides a method, device, system, and storage medium for synchronizing EEG data and behavioral data, aiming to solve the problem that existing methods cannot accurately synchronize EEG data and behavioral data of subjects in a natural state.

[0007] To address the aforementioned technical problems, in a first aspect, this application provides a method for synchronizing EEG data and behavioral data, the method comprising:

[0008] Secondly, this application provides a brainwave data and behavioral data processing device, the device comprising:

[0009] The transmitting module is used to send acquisition signals to multiple camera devices and EEG acquisition devices simultaneously and periodically send trigger signals in response to a start command; it is also used to send end signals to multiple camera devices and EEG acquisition devices simultaneously when set conditions are met or in response to an end command.

[0010] The acquisition module is used to acquire multiple video data and EEG data of the subject;

[0011] A reconstruction module is used to perform three-dimensional reconstruction on the acquired multiple video data to obtain the subject's behavioral data;

[0012] A synchronization module is used to synchronize the behavioral data and the EEG data based on the recorded trigger signal sequence.

[0013] Thirdly, this application provides a brainwave data and behavioral data processing system, the system including a camera unit, a brainwave acquisition device, a synchronization unit, and a host computer;

[0014] The camera unit includes multiple cameras for collecting multiple video data from the subject;

[0015] The EEG acquisition unit is used to acquire the EEG data of the subject;

[0016] The host computer is configured to, in response to a start command, simultaneously send acquisition signals to the multiple cameras and the EEG acquisition device and periodically send trigger signals to acquire multiple video data and EEG data of the subject; and perform three-dimensional reconstruction on the acquired multiple video data to acquire the subject's behavioral data; and when a set condition is met or in response to a stop command, simultaneously send a stop signal to the multiple cameras and the EEG acquisition device; and perform synchronous processing of the behavioral data and the EEG data based on the recorded trigger signal sequence.

[0017] The synchronization unit is used to receive the trigger signal and synchronously send the trigger signal to the EEG acquisition device and the multiple cameras.

[0018] Fourthly, this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-mentioned method for synchronizing EEG data and behavioral data.

[0019] Fifthly, a storage medium storing computer-readable instructions that are executed by one or more processors to implement the method for synchronizing EEG data and behavioral data as described above.

[0020] Sixthly, a computer program product comprising computer-readable instructions stored in a storage medium, wherein one or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, thereby enabling the electronic device to implement the method for synchronizing EEG data and behavioral data as described above.

[0021] Compared with existing technologies, the EEG data and behavioral data synchronization method of this application can simultaneously collect and record EEG signals and videos of the human body under natural behavior, thereby improving the ecological effectiveness of the data and the breadth of application scenarios. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 is a schematic diagram of the main flow of a method for synchronizing EEG data and behavioral data provided in an embodiment of this application;

[0024] Figure 2 illustrates a usage scenario of an EEG data and behavioral data synchronization system provided in an embodiment of this application.

[0025] Figure 3 is a structural block diagram of a brainwave data and behavioral data synchronization device provided in an embodiment of this application;

[0026] Figure 4 is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this disclosure means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0029] Please refer to Figure 1, which is a schematic diagram of the main flow of the method for synchronously acquiring EEG data and behavioral data according to an embodiment of this application. The method for synchronously acquiring EEG data and behavioral data according to an embodiment of this application is used to acquire multiple video data and EEG data of a subject through multiple camera devices and EEG acquisition devices; wherein the camera devices can be various types of cameras, and the EEG acquisition devices can be wirelessly linked EEG caps or wired EEG instruments, etc. The method includes the following steps S100 to S300:

[0030] S100, in response to the start command, simultaneously sends acquisition signals to multiple camera devices and EEG acquisition devices and periodically sends trigger signals until the acquisition ends, so as to obtain multiple video data and EEG data of the subject.

[0031] The acquisition signal is used to instruct the plurality of camera devices and the EEG acquisition device to start acquiring data at the same acquisition point at the same time, and the trigger signal is used to indicate a new acquisition point;

[0032] In this embodiment, multiple cameras are used to acquire video data of the subject in a natural state. Preferably, eight cameras arranged around the subject can be used, enabling high-speed, synchronous video recording from all eight cameras. This provides comprehensive, multi-angle video recording, capturing all behavioral details of the subject in a free-moving state, providing more comprehensive data support for subsequent behavioral analysis. Simultaneously, the frame rate reaches 160 frames per second, the image quality reaches 1280P resolution, and the cameras are precisely synchronized. A data acquisition control program is used to acquire, manage, and process EEG and video data. Specifically, when the system receives a start acquisition command from the operator, it simultaneously sends a start acquisition signal to multiple camera devices and the EEG acquisition device to ensure that all cameras acquire data at the same time. It also periodically sends a trigger signal to mark the start of a new acquisition segment.

[0033] In an exemplary embodiment, step S100 specifically includes steps S110 to S130:

[0034] S110. In response to the start command, establish a main control process and multiple auxiliary processes; each of the aforementioned camera devices and EEG acquisition devices corresponds to one auxiliary process;

[0035] S120. The main control process simultaneously sends acquisition signals to multiple camera devices and EEG acquisition devices and periodically sends trigger signals.

[0036] S130, The video data and EEG data are acquired respectively through the multiple auxiliary processes.

[0037] During data acquisition, the acquisition control program uses multiple processes to manage and control the acquisition work of different camera devices and EEG acquisition devices. Each camera device corresponds to an independent process to acquire video frames; the main control process coordinates the synchronous recording of each camera device and EEG acquisition device by controlling the sending of trigger signals. A memory queue is used to process the data streams from the camera devices and EEG acquisition devices to prevent frame loss due to processing delays or bandwidth limitations. Data from each camera device is passed to the storage process through a shared queue to ensure data real-time performance and integrity. The acquisition control program continuously monitors the data stream during acquisition, dynamically adjusting buffers and processing parameters to ensure the stability of the acquisition process.

[0038] The acquisition control program utilizes a multi-threaded and multi-process mechanism to send synchronization signals to all cameras and EEG acquisition devices, ensuring that each camera receives and processes the synchronization signal at the same time, thereby achieving synchronous acquisition of EEG signals and video.

[0039] In one embodiment of this invention, a trigger signal is sent after the camera device has acquired a certain number of frames, such as 200 frames or 300 frames. The specific number of frames can be adjusted according to the design. When a set condition is met or in response to an end command, an end signal is sent to multiple camera devices and EEG acquisition devices simultaneously. Specifically, the end signal is sent when the host recording reaches a specified number of frames, reaches a specified recording duration, or receives a termination signal; the specified number of frames or specified recording duration is not subject to excessive restrictions. The acquisition control program sends a trigger signal indicating the end to multiple camera devices and the EEG acquisition device, terminates the acquisition process, releases the resources of multiple cameras, and ends the acquisition. The end recording part mainly involves the acquisition control program sending an end command to multiple target devices and the EEG acquisition device after acquiring enough frames or receiving a termination signal, terminating the process, releasing resources, and ending the acquisition.

[0040] S200. Perform three-dimensional reconstruction on the acquired multiple video data to obtain the subject's behavioral data.

[0041] In an exemplary embodiment, step S200 specifically includes the following steps S210 to S230:

[0042] S210. Calibrate the plurality of camera devices to determine the parameters of each camera device;

[0043] In this embodiment, the purpose of multi-camera device calibration is to obtain the intrinsic parameters and spatial position distribution of the camera devices, providing a parameter basis for 3D reconstruction. This is mainly achieved by using Zhang's calibration method to create a calibration board for pose estimation of multiple cameras.

[0044] S220. Extract the two-dimensional skeletal points of the subject from the plurality of video data;

[0045] In this embodiment, a trained neural network model is used to identify and track two-dimensional skeletal points in video data. Specifically, a calibration board is placed within the field of view of the camera devices, and images of the calibration board are captured from multiple angles and positions to ensure that each camera captures a sufficient number and quality of calibration board images. The calibration board of each camera device has a 7*4 checkerboard grid, and the grid size is customized according to the shooting space. Each camera device captures 200 calibration board images from different angles, totaling 1600 images from all eight devices. These images are then uploaded to the workspace, where the StereoCameraCalibrator GUI toolbox is used for calibration to obtain the camera's intrinsic and extrinsic parameters, including the rotation matrix R and translation vector T, which are then saved locally. A YOLOv8 deep learning network model is used to train and predict the video, performing accurate two-dimensional human pose estimation. Using the calibrated camera parameters, a triangulation algorithm is used to reconstruct 3D images of 17 body points.

[0046] S230. Construct a three-dimensional model of the subject based on the parameters of each of the camera devices and the two-dimensional skeletal points of the subject, so as to obtain the subject's behavioral data.

[0047] The computer vision triangulation algorithm maps the two-dimensional body skeleton points of the subject to three-dimensional space to obtain a three-dimensional skeleton model of the human body, and the subject's behavioral data can be calculated based on the three-dimensional skeleton model.

[0048] S300. Synchronize the behavioral data and the EEG data based on the recorded trigger signal sequence.

[0049] In an exemplary embodiment, step S300 specifically includes the following steps S310 to S320:

[0050] S310. Perform a correctness check on the trigger signal sequence;

[0051] The first step is to check the correctness of the received trigger signal sequence. This includes verifying the integrity of the signals, checking for any missing or incorrect trigger points, and confirming the accuracy of the trigger point timestamps. This step is fundamental to ensuring the reliability of subsequent synchronization processes.

[0052] S320. The EEG data is cropped and spliced ​​according to the trigger signal sequence to achieve synchronization between the behavioral data and the EEG data.

[0053] After confirming the trigger signal sequence is correct, the next step is to trim and splice the original EEG signal based on these trigger signals. Specifically, this involves dividing the EEG signal into segments corresponding to specific events or time points, and then rearranging or splicing these segments according to the experimental design to form a continuous signal consistent with the experimental procedure. The key to this step is ensuring that the start and end times of each segment precisely correspond to the timestamps of the trigger signals. This achieves high-precision synchronization of the EEG signal with multiple cameras, ensuring data consistency across the time dimension.

[0054] The EEG data and behavioral data synchronization method of this application can simultaneously collect and record EEG signals and videos of the human body under natural behavior, thereby improving the ecological effectiveness of the data and the breadth of application scenarios.

[0055] As shown in Figure 3, this application also provides an EEG data and behavioral data processing device 200, which includes a sending module 201, an acquisition module 202, a reconstruction module 203, and a synchronization module 204. The sending module 201 is used to send acquisition signals to multiple camera devices and EEG acquisition devices simultaneously and periodically send trigger signals in response to a start command; it is also used to send end signals to multiple camera devices and EEG acquisition devices simultaneously when a set condition is met or in response to an end command. The acquisition module 202 is used to acquire multiple video data and EEG data of the subject. The reconstruction module 203 is used to perform three-dimensional reconstruction on the acquired multiple video data to obtain the subject's behavioral data. The synchronization module 204 is used to synchronize the behavioral data and the EEG data based on the recorded trigger signal sequence.

[0056] In one embodiment, the sending module includes an establishment unit, a sending unit, and an acquisition unit. The establishment unit is used to establish a main control process and multiple auxiliary processes in response to a start command. Each of the camera devices and EEG acquisition devices corresponds to an auxiliary process. The sending unit is used to send acquisition signals to multiple camera devices and EEG acquisition devices simultaneously through the main control process and to periodically send trigger signals.

[0057] In one embodiment, the reconstruction module includes a calibration unit, an extraction unit, and a construction unit; the calibration unit is used to calibrate the plurality of camera devices to determine the parameters of each camera device; the extraction unit is used to extract the two-dimensional skeletal points of the subject from the plurality of video data; the construction unit is used to construct a three-dimensional model of the subject based on the parameters of each camera device and the two-dimensional skeletal points of the subject to obtain the subject's behavioral data.

[0058] In one embodiment, the synchronization module includes a detection unit and a splicing unit. The detection unit is used to check the correctness of the trigger signal sequence; the splicing unit is used to trim and splice the EEG data according to the trigger signal sequence to achieve synchronization between the behavioral data and the EEG data.

[0059] As shown in Figure 2, this application also provides an EEG data and behavioral data processing system, the system including a camera unit, an EEG acquisition device, a synchronization unit, and a host computer 104; the camera unit includes multiple cameras 102 for acquiring multiple video data of the subject; the EEG acquisition unit is used to acquire the subject's EEG data; the host computer 104 is used to, in response to a start command, simultaneously send acquisition signals to the multiple cameras 102 and the EEG acquisition device and periodically send trigger signals to acquire multiple video data and EEG data of the subject; and perform three-dimensional reconstruction on the acquired multiple video data to acquire the subject's behavioral data; and when a set condition is met or in response to an end command, simultaneously send an end signal to the multiple cameras 102 and the EEG acquisition device; and perform synchronous processing on the behavioral data and the EEG data based on the recorded trigger signal sequence; the synchronization unit is used to receive the trigger signals and synchronously send the trigger signals to the EEG acquisition device and the multiple cameras 102.

[0060] Optionally, the synchronization unit includes an intelligent synchronization center 107 and a multi-parameter synchronizer 108.

[0061] Optionally, the EEG acquisition unit includes an EEG cap 105 and a signal amplifier 106, and the EEG acquisition unit is wirelessly connected to the synchronization unit.

[0062] The EEG cap 105 includes non-invasive EEG electrodes, electrode wires, and a cap for fixation, used to collect weak electrical signals generated by brain activity. A signal amplifier 106, used to acquire brain potential information and wirelessly transmit the amplified signal to a receiving device, is located at the rear end of the EEG cap 105 and can be magnetically fixed to the cap 105 if the interface is compatible.

[0063] Optionally, the camera unit includes at least eight cameras 102 arranged circumferentially around the subject.

[0064] Specifically, eight cameras 102 are mounted on a fixed bracket 101. The bracket can be fixed to any ceiling, board, or other top surface, or placed on the ground for easy disassembly. Its dimensions are not fixed and can be adjusted according to actual needs. All eight cameras 102 are of the same model and use USB 3.0 interfaces. Four cameras 102 are mounted in the middle of the four pillars of the camera 102 mounting bracket, facing horizontally with a slight downward tilt; the other four cameras 102 are mounted in the four corners to ensure that the cameras 102 capture complete human behavioral information within their field of view. The camera angles are directed towards the shooting plane 103, within which the subject can move freely or complete experimental tasks according to the experimental paradigm.

[0065] Please refer to Figure 4. An electronic device 4000 is provided in this embodiment of the application. The electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0066] Data interaction between the processor 4001 and the memory 4003 can be achieved through at least one communication bus 4002. This communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used to represent it in the figure, but this does not indicate that there is only one bus or one type of bus.

[0067] Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0068] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0069] The memory 4003 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program instructions or code in the form of instructions or data structures and accessible by the electronic device 4000, but not limited thereto.

[0070] The memory 4003 stores computer-readable instructions, and the processor 4001 can read the computer-readable instructions stored in the memory 4003 through the communication bus 4002.

[0071] The computer-readable instructions are executed by one or more processors 4001 to implement the EEG data and behavioral data synchronous acquisition system in the above embodiments.

[0072] Furthermore, this application provides a storage medium storing computer-readable instructions that are executed by one or more processors to implement the above-described method for synchronizing EEG data and behavioral data.

[0073] This application provides a computer program product including computer-readable instructions stored in a storage medium. One or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, thereby enabling the electronic device to implement the above-mentioned method for synchronizing EEG data and behavioral data.

[0074] Compared with related technologies, the EEG and behavioral data synchronization method of this application can simultaneously acquire and record EEG signals and videos of the human body during natural behavior, improving the ecological effectiveness of the data and the breadth of application scenarios. It can prevent frame loss during high-speed recording. It integrates a high-efficiency data processing and storage module, enabling rapid processing and synchronous storage of large-scale video and EEG data, ensuring data integrity and real-time performance. It can achieve high-precision synchronization of EEG signals with multiple cameras, ensuring data consistency over time. It can complete camera calibration and three-dimensional reconstruction of human skeletal points.

[0075] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0076] This application has been described through several embodiments. Those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of this application. Furthermore, based on the teachings of this application, these features and embodiments can be modified to suit specific circumstances and materials without departing from the spirit and scope of this application. Therefore, this application is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims are protected by this application.

Claims

1. A method for synchronizing electroencephalogram (EEG) data with behavioral data, characterized in that, The method for acquiring multiple video data and EEG data of a subject using multiple camera devices and EEG acquisition devices includes: In response to a start command, acquisition signals are simultaneously sent to multiple camera devices and EEG acquisition devices, and trigger signals are periodically sent until the acquisition ends, in order to acquire multiple video data and EEG data of the subject; wherein, the acquisition signals are used to instruct the multiple camera devices and the EEG acquisition devices to start acquisition at the same acquisition point at the same time, and the trigger signals are used to indicate a new acquisition point; The acquired video data is reconstructed in three dimensions to obtain the subject's behavioral data; The behavioral data and the EEG data are synchronized based on the recorded sequence of trigger signals.

2. The method according to claim 1, characterized in that, In response to a start command, acquisition signals are simultaneously sent to multiple camera devices and EEG acquisition devices, and trigger signals are periodically sent until the acquisition ends, in order to acquire multiple video and EEG data of the subject; including: In response to the start command, a main control process and multiple auxiliary processes are established; each of the aforementioned camera devices and EEG acquisition devices corresponds to one auxiliary process. The main control process simultaneously sends acquisition signals to multiple camera devices and EEG acquisition devices and periodically sends trigger signals. The video data and EEG data are acquired through the multiple auxiliary processes respectively.

3. The method according to claim 1, characterized in that, The periodic sending of the trigger signal includes sending a trigger signal once every time the camera device captures a set number of frames.

4. The method according to claim 1, characterized in that, The synchronization processing of the behavioral data and the EEG data based on the recorded trigger signal sequence includes: Perform a correctness check on the trigger signal sequence; The EEG data is cropped and spliced ​​according to the trigger signal sequence to achieve synchronization between the behavioral data and the EEG data.

5. The method according to claim 2, characterized in that, The step of performing three-dimensional reconstruction on the acquired multiple video data to obtain the subject's behavioral data includes: The plurality of camera devices are calibrated to determine the parameters of each camera device; Extract the subject's two-dimensional skeletal points from the multiple video data; The subject is constructed based on the parameters of each of the camera devices and the two-dimensional skeletal points of the subject. A three-dimensional model was created to obtain the subject's behavioral data.

6. A device for processing electroencephalogram (EEG) data and behavioral data, characterized in that, The device includes: The transmitting module is used to send acquisition signals to multiple camera devices and EEG acquisition devices simultaneously and periodically send trigger signals in response to a start command; it is also used to send end signals to multiple camera devices and EEG acquisition devices simultaneously when set conditions are met or in response to an end command. The acquisition module is used to acquire multiple video data and EEG data of the subject; A reconstruction module is used to perform three-dimensional reconstruction on the acquired multiple video data to obtain the subject's behavioral data; A synchronization module is used to synchronize the behavioral data and the EEG data based on the recorded trigger signal sequence.

7. A brainwave data and behavioral data processing system, characterized in that, The system includes a camera unit, an EEG acquisition device, a synchronization unit, and a host computer; The camera unit includes multiple cameras for collecting multiple video data from the subject; The EEG acquisition unit is used to acquire the EEG data of the subject; The host computer is configured to, in response to a start command, simultaneously send acquisition signals to the multiple cameras and the EEG acquisition device and periodically send trigger signals to acquire multiple video data and EEG data of the subject; and perform three-dimensional reconstruction on the acquired multiple video data to acquire the subject's behavioral data; and when a set condition is met or in response to a stop command, simultaneously send a stop signal to the multiple cameras and the EEG acquisition device; and perform synchronous processing of the behavioral data and the EEG data based on the recorded trigger signal sequence. The synchronization unit is used to receive the trigger signal and synchronously send the trigger signal to the EEG acquisition device and the multiple cameras.

8. The system according to claim 7, characterized in that, The EEG acquisition unit includes an EEG cap and a signal amplifier, and the EEG acquisition unit is wirelessly connected to the synchronization unit.

9. The system according to claim 7, characterized in that, The camera unit includes at least eight cameras arranged circumferentially around the subject.

10. A storage medium, characterized in that, The storage medium stores computer-readable instructions, which are executed by one or more processors to implement the method for synchronizing EEG data and behavioral data as described in any one of claims 1 to 5.

Images