Brain-computer interface system electroencephalogram data and image synchronization system and method

Abstract

The invention discloses a brain-computer interface system electroencephalogram data and image synchronization system, which comprises a PC, a receiver host, a plurality of receiver slaves and a camera, and is characterized in that the PC sends a command to the receiver host, receives and processes data of the receiver host, controls the camera to record or photograph, and receives and processes videos or images shot by the camera; starting the receiver host to send an electroencephalogram data packet to the PC, starting the receiver slaves to transmit data to the PC at the same time, and sending fixed-frequency and fixed-period pulses to the receiver slaves and the camera, so that the plurality of receiver slaves are synchronized and the camera has a photographing function. According to the brain-computer interface system electroencephalogram data and image synchronization system and method provided by the invention, unified start-stop signals and fixed-period pulses are sent from the receiver host to the receiver slave and the camera, so that the host, the slave and the camera start to transmit electroencephalogram data and images to the PC end at the same time; and the electroencephalogram data and image synchronization effect is achieved.

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interface technology, and in particular to a system and method for synchronizing electroencephalogram (EEG) data and images of a brain-computer interface system. Background Art

[0002] In the field of brain-computer interface technology, when EEG data and images are processed synchronously, time synchronization is usually difficult or unstable. Invention patent application 2024107099717 discloses a time control method, system and storage medium for EEG data, which can provide the accurate occurrence time of each frame of data, so that time information will not be lost when the EEG data is refined. The method includes: when the acquisition end and the PC end are successfully paired online, the PC end obtains the timestamp T0, and the acquisition end synchronously initializes the timer; the acquisition end continuously collects a preset amount of EEG data according to the sampling rate, and records the sampling time t of each EEG data. i , and the EEG data and the corresponding sampling time t i The data is packaged into data frames and sent to the PC. The PC continuously analyzes the data frames and calculates the sampling time t of each EEG data. i and timestamp T0 to obtain the sampling time T of each EEG data i .

[0003] However, this method can only be used when the acquisition end and the PC end are successfully paired online. The PC end obtains the timestamp T0, and the acquisition end synchronously initializes the timer, so that the acquisition end records the sampling time t of each EEG data. i , and on the PC side, according to the sampling time t of each EEG data i and timestamp T0 to obtain the sampling time T of each EEG data i , only solves the time synchronization problem between the acquisition end and the PC. It is not possible to synchronize the images captured by the camera with the EEG data collected by the acquisition end after adding the camera part, nor is it possible to solve the EEG data synchronization problem between the acquisition ends when there are multiple acquisition ends. Summary of the invention

[0004] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is that the existing EEG data and image synchronization technology can only be tested when connected to the Internet, and it is impossible to achieve synchronization between the images captured by the camera and the EEG data, and it is also impossible to achieve EEG data synchronization between multiple acquisition terminals when there are multiple acquisition terminals. The present invention provides a brain-computer interface system EEG data and image synchronization system and method, by sending a unified start-stop signal and a fixed-period pulse to the receiver slave and camera at the receiver host, so that the host, slave and camera start to transmit EEG data and images to the PC at the same time, so as to achieve the synchronization effect of EEG data and images.

[0005] To achieve the above-mentioned purpose, the present invention provides an EEG data and image synchronization system of a brain-computer interface system, including a PC, a receiver host, several receiver slaves and a camera. The PC sends commands to the receiver host, receives and processes data from the receiver host, controls the camera to record or take pictures, and receives and processes videos or images taken by the camera; the receiver host receives and processes the commands sent by the PC, starts the receiver host to send EEG data packets to the PC, and simultaneously starts the receiver slave to transmit data to the PC, and sends fixed frequency and fixed period pulses to the receiver slave and the camera, so that several receiver slaves are synchronized with each other and the camera takes pictures synchronously.

[0006] Furthermore, the PC sends commands to the receiver host through the RJ45 network port using the TCP protocol, and receives and processes data transmitted by the receiver host through the TCP protocol.

[0007] Furthermore, the PC controls the camera to record or take pictures through the USB, and receives and processes the images taken by the camera.

[0008] Furthermore, the receiver host sends EEG data packets to the PC through the RJ45 network port using the TCP protocol, and receives and processes commands sent by the PC using the TCP protocol; through the SLAVE_START command, the data transmission from the receiver host to the PC is started, and at the same time, the receiver slave is started to transmit data to the PC; through the CAMERA_SYNC command, a fixed-frequency frame pulse is sent to the receiver slave and the camera to achieve synchronization between several receiver masters and slaves and camera photo synchronization.

[0009] Further, the receiver slave receives SLAVE_START and CAMERA_SYNC commands sent by the receiver master to start data transmission to the PC and achieve synchronization between the master and slave.

[0010] Furthermore, both the receiver host and the receiver slave will count the pulses of the fixed period, and restart the counting when the count value reaches the set value, so as to ensure long-term synchronization between the receivers using different chips.

[0011] Another preferred embodiment of the present invention provides a method for synchronizing EEG data and image synchronization system of a brain-computer interface system, comprising the following steps:

[0012] The PC sends a START command to the receiver host;

[0013] After receiving the START command, the receiver host starts the master and slave devices of the receiver, and starts the data transmission and data frame counting of the master and slave devices; at the same time, it generates a camera pulse and starts the camera shooting;

[0014] The master and slave receivers and cameras are synchronized once every minute;

[0015] The PC sends a RECORD command to the receiver host;

[0016] After receiving the RECORD command, the receiver host and the camera are suspended;

[0017] After a delay, the receiver host sends a BACK_1 command to the PC through the network to start camera recording, and sends a BACK_2 command to the receiver host through the network;

[0018] After receiving the BACK_2 command sent by the PC, the receiver host resumes the data transmission and data frame counting of the receiver master and slave, resumes the camera pulse, and starts the camera shooting.

[0019] Furthermore, after receiving the RECORD command, the receiver host suspends the receiver host and the camera. Specifically, after receiving the RECORD command, the receiver host sends a STOP_SLAVE command to the receiver slave and the camera respectively to suspend the receiver host and the camera; at the same time, the data frame count value and the camera pulse count value of the receiver master and slave will also be set to 0.

[0020] Furthermore, after the receiver host receives the BACK_2 command sent by the PC, it resumes the data transmission and data frame count of the receiver master and slave and resumes the camera pulse, and starts the camera shooting, specifically including: after the receiver host receives the BACK_2 command sent by the PC, it sends the CONTINUE_SLAVE command and the CONTINUE_CAMERA command to the receiver slave and the camera respectively, resumes the data transmission and data frame count of the receiver master and slave and resumes the camera pulse, starts the camera shooting, and starts the next cycle.

[0021] Technical Effects

[0022] The present invention provides a brain-computer interface system EEG data and image synchronization system and method, which has multiple acquisition terminals, adopts a master-slave working mode, is provided with and has only one host, connects multiple slaves through lines, and after the PC connects the system to the network, when a startup command is sent to the host through TCP, the host also sends a command to the slave to start the slave, starts the transmission of EEG data and synchronization signals, ensures that the master and slave start to transmit EEG data to the PC at the same time, and synchronizes according to the synchronization signal; on the data path where the host sends a signal to the slave, a data path that sends a synchronization signal to the camera is led out, so that the camera starts to capture images at the moment of receiving the synchronization signal, ensures that the first frame image and the first packet data are transmitted to the PC at the same time, and realizes synchronization between data and images. Thus, data synchronization between multiple acquisition terminals and synchronization between data transmitted by multiple acquisition terminals and images captured by the camera are realized, and synchronization can be maintained for a long time.

[0023] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a connection diagram of an EEG data and image synchronization system of a brain-computer interface system according to a preferred embodiment of the present invention;

[0025] Figure 2 It is a synchronization function diagram of a brain-computer interface system EEG data and image synchronization system in a preferred embodiment of the present invention;

[0026] Figure 3 It is a signal waveform diagram of an EEG data and image synchronization system of a brain-computer interface system in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0028] In the following description, specific details such as specific internal procedures and techniques are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0029] like Figure 1As shown, the present invention provides an EEG data and image synchronization system of a brain-computer interface system, including a PC, a receiver host, several receiver slaves and a camera. The PC sends commands to the receiver host, receives and processes data from the receiver host, controls the camera to record or take pictures, and receives and processes videos or images taken by the camera; the receiver host receives and processes the commands sent by the PC, starts the receiver host to send EEG data packets to the PC, and simultaneously starts the receiver slave to transmit data to the PC, and sends fixed frequency and fixed period pulses to the receiver slave and the camera, so that several receiver slaves are synchronized with each other and the camera takes pictures synchronously.

[0030] The PC uses the TCP protocol to send commands to the receiver host through the RJ45 network port, and receives and processes the data transmitted by the receiver host through the TCP protocol. The PC controls the camera to record or take pictures through the USB, and receives and processes the images taken by the camera. In this embodiment, the START command is a 0xAABBEEFF string, and the RECORD command is a 0xAABBCCFF string. If other commands are needed to control the host, you can define the commands yourself and add related logic.

[0031] The receiver host sends EEG data packets to the PC through the RJ45 network port using the TCP protocol, and receives and processes commands sent by the PC using the TCP protocol; through the SLAVE_START command, the data transmission from the receiver host to the PC is started, and at the same time, the data transmission from the receiver slave to the PC is started; through the CAMERA_SYNC command, a fixed-frequency frame pulse is sent to the receiver slave and the camera to achieve synchronization between several receiver masters and slaves and the camera's photo-taking function. The receiver slave receives the SLAVE_START and CAMERA_SYNC commands sent by the receiver host to start data transmission to the PC and achieve synchronization between the master and slave. In this embodiment, the SLAVE_START command is 0xAABBEEFF, and the RECORD command is 0xAABBCCFF, and its parameters include data frame count value, camera pulse frequency (20hz), and camera pulse count value (from 0 to 1199, that is, after every 1s / 20hz*1200=60s, the data frame count value and the count value are set to 0, so that the master and slave are synchronized once, and the counting is cyclic).

[0032] The receiver host and the receiver slave will count the pulses of the fixed period, and when the count value reaches the set value, the count will be restarted to ensure long-term synchronization between receivers using different chips. The number of pulses in this embodiment is 20-50hz, preferably 20hz, corresponding to 50ms, which is convenient for calculation and PC processing.

[0033] Another preferred embodiment of the present invention provides a method for synchronizing EEG data and image synchronization system of a brain-computer interface system, comprising the following steps:

[0034] The PC sends a START command to the receiver host;

[0035] After receiving the START command, the receiver host starts the master and slave devices of the receiver, and starts the data transmission and data frame counting of the master and slave devices; at the same time, it generates a camera pulse and starts the camera shooting;

[0036] The master and slave receivers and cameras are synchronized once every minute;

[0037] The PC sends a RECORD command to the receiver host;

[0038] After receiving the RECORD command, the receiver host sends the STOP_SLAVE command to the receiver slave and camera respectively, so that the receiver host and camera are paused; at the same time, the data frame count value and camera pulse count value of the receiver master and slave will also be set to 0;

[0039] After a delay, the receiver host sends a BACK_1 command to the PC through the network to start camera recording, and sends a BACK_2 command to the receiver host through the network;

[0040] After receiving the BACK_2 command sent by the PC, the receiver host sends the CONTINUE_SLAVE command and CONTINUE_CAMERA command to the receiver slave and camera respectively, restores the data transmission and data frame count of the receiver master and slave, restores the camera pulse, starts the camera shooting, and starts the next cycle.

[0041] Specifically, as follows:

[0042] The PC sends a START command (0xAABBEEFF) to the receiver host through the network.

[0043] After receiving the START command, the receiver host pulls down the SLAVE_START signal to send the START_SLAVE command (start the master-slave, start the data transmission and data frame counting of the receiver master-slave) and the START_CAMERA command (generate camera pulses, start camera shooting) to the receiver slave and camera respectively, and generates the CAMERA_SYNC signal to send a fixed-frequency pulse to the camera, so that the camera takes pictures according to the pulse frequency (20hz in this case).

[0044] At this time, the data frame count value and camera pulse count value of the master and slave receivers start to increase from 0 at the same time. When the camera pulse count value reaches 1199, the data frame count value and camera pulse count value of the master and slave are set to 0 at the same time, which is equivalent to the master and slave receivers simultaneously counting the 20hz pulses generated by the receiver host. After one minute, the master and slave receivers achieve synchronization by setting the above two count values ​​to 0, thereby achieving synchronization of the master and slave receivers once a minute, avoiding differences in the count values ​​of the master and slave receivers after long-term operation.

[0045] If the PC sends a RECORD command (0xAABBCCFF) to the receiver host through the network, the receiver host will send a STOP_SLAVE command (pause the receiver slave) and a STOP_CAMERA command (pause the camera) to the receiver slave and the camera respectively after receiving the RECORD command, and the data frame count value and camera pulse count value of the receiver master and slave will also be set to 0. After a delay, the receiver host sends a BACK_1 command (such as 0xAABBDDFF) to the PC through the network. After receiving the BACK_1 command sent by the receiver host, the PC starts the camera recording and sends a BACK_2 command (such as 0xAABBFFFF) to the receiver host through the network. After receiving the BACK_2 command sent by the PC, the receiver host sends a CONTINUE_SLAVE command (resume the data transmission and data frame count of the receiver master and slave) and a CONTINUE_CAMERA command (resume the camera pulse and start the camera shooting) to the receiver slave and the camera respectively, and start the next cycle.

[0046] The following is an explanation of each command.

[0047] START: PC sends commands to the host through the network.

[0048] START_SLAVE: After the host receives the START command sent by the PC, it starts the master-slave machine and starts the data transmission and data frame counting between the master and slave machines.

[0049] START_CAMERA: After the host receives the START command sent by the PC, it generates a camera pulse and starts camera shooting.

[0050] RECORD: PC sends commands to the host through the network.

[0051] STOP_SLAVE: After the host receives the RECORD command sent by the PC, it suspends the data transmission between the master and slave and the slave data frame counting.

[0052] STOP_CAMERA: After the host receives the RECORD command sent by the PC, it pauses the camera pulse and turns off the camera shooting.

[0053] BACK_1: After a delay, the host sends a BACK_1 command to the PC through the network.

[0054] BACK_2: After receiving the BACK_1 command from the host, the PC starts camera recording and sends commands to the host through the network.

[0055] CONTINUE_SLAVE: After the host receives the BACK_2 command sent by the PC, it resumes the data transmission and data frame counting between the master and slave devices.

[0056] CONTINUE_CAMERA: After the host receives the BACK_2 command sent by the PC, it restores the camera pulse and starts camera shooting.

[0057] SLAVE_START: Controls the start and stop of data transmission from the slave to the PC. Low level starts and high level stops.

[0058] CAMERA_SYNC: Through pulses with a period of T, synchronization between the master and slave devices and the camera's photo-taking function are achieved.

[0059] T: Calculate the pulse period according to the set camera frame rate. For example, if the camera frame rate is set to 20 Hz, then T = 1s / 20 = 50ms.

[0060] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A brain-computer interface system EEG data and image synchronization system, characterized in that: It includes a PC, a receiver host, several receiver slaves and a camera. The PC sends commands to the receiver host, receives and processes data from the receiver host, controls the camera to record or take pictures, and receives and processes videos or images taken by the camera. The receiver host receives and processes the commands sent by the PC, starts the receiver host to send EEG data packets to the PC, and simultaneously starts the receiver slave to transmit data to the PC, and sends fixed frequency and fixed period pulses to the receiver slave and the camera, so that several of the receiver slaves are synchronized and the camera takes pictures synchronously.

2. The brain-computer interface system EEG data and image synchronization system as claimed in claim 1, characterized in that: The PC sends commands to the receiver host through the RJ45 network port using the TCP protocol, and receives and processes data transmitted by the receiver host through the TCP protocol.

3. The brain-computer interface system EEG data and image synchronization system as claimed in claim 2, characterized in that: The PC controls the camera to record or take pictures via USB, and receives and processes the images taken by the camera.

4. The brain-computer interface system EEG data and image synchronization system as claimed in claim 1, characterized in that: The receiver host sends EEG data packets to the PC via the RJ45 network port using the TCP protocol, and receives and processes commands sent by the PC using the TCP protocol; through the SLAVE_START command, the data transmission from the receiver host to the PC is started, and at the same time, the receiver slave is started to transmit data to the PC; through the CAMERA_SYNC command, a fixed-frequency frame pulse is sent to the receiver slave and the camera to achieve synchronization between several receiver masters and slaves and camera photo-taking synchronization.

5. The brain-computer interface system EEG data and image synchronization system as claimed in claim 4, characterized in that: The receiver slave receives the SLAVE_START and CAMERA_SYNC commands sent by the receiver host to start data transmission to the PC and achieve synchronization between the master and slave.

6. The brain-computer interface system EEG data and image synchronization system as claimed in claim 1, characterized in that: The receiver host and the receiver slave both count pulses of a fixed period, and restart counting when the count value reaches a set value, so as to ensure long-term synchronization between receivers using different chips.

7. A method for synchronizing an EEG data and image synchronization system of a brain-computer interface system as described in any one of claims 1 to 6, characterized in that: The following steps are involved: The PC sends a START command to the receiver host; After receiving the START command, the receiver host starts the master and slave devices of the receiver, and starts the data transmission and data frame counting of the master and slave devices; at the same time, it generates a camera pulse and starts the camera shooting; The master and slave receivers and cameras are synchronized once every minute; The PC sends a RECORD command to the receiver host; After receiving the RECORD command, the receiver host and the camera are suspended; After a delay, the receiver host sends a BACK_1 command to the PC through the network to start camera recording, and sends a BACK_2 command to the receiver host through the network; After receiving the BACK_2 command sent by the PC, the receiver host resumes the data transmission and data frame counting of the receiver master and slave, resumes the camera pulse, and starts the camera shooting.

8. A method for synchronizing EEG data and images of a brain-computer interface system as claimed in claim 7, characterized in that: After receiving the RECORD command, the receiver host suspends the receiver host and the camera. Specifically, after receiving the RECORD command, the receiver host sends a STOP_SLAVE command to the receiver slave and the camera respectively to suspend the receiver host and the camera; at the same time, the data frame count value and the camera pulse count value of the receiver master and slave will also be set to 0.

9. A method for synchronizing EEG data and images of a brain-computer interface system as claimed in claim 7, characterized in that: The method comprises the following steps: after the receiver host receives the BACK_2 command sent by the PC, the data transmission and data frame count of the receiver master and slave are restored, the camera pulse is restored, and the camera shooting is started. Specifically, the method comprises the following steps: after the receiver host receives the BACK_2 command sent by the PC, the receiver host sends the CONTINUE_SLAVE command and the CONTINUE_CAMERA command to the receiver slave and the camera respectively, restores the data transmission and data frame count of the receiver master and slave, restores the camera pulse, starts the camera shooting, and starts the next cycle.

Images