Device and method based on multi-modal physiological signal synchronization

Abstract

The invention discloses a device and method based on multi-modal physiological signal synchronization, and belongs to the technical field of signal synchronization, and the device comprises a plurality of synchronous trigger signal input interfaces which are used for receiving synchronous trigger signals from a manual input circuit, program preset input or third-party synchronous trigger equipment input; the FPGA control module is connected with the input interface and is used for detecting the synchronous trigger signal and generating a corresponding synchronous signal and a high-precision timestamp; the global unified clock management module is integrated in the FPGA control module and is used for generating and providing multiple paths of synchronous clock signals; the various synchronizing signal output interfaces are connected with the FPGA control module and used for converting the synchronizing signals and the timestamps into signals conforming to different electrophysiology equipment interface protocols and outputting the signals; and the output calibration module is integrated in the FPGA control module and is used for carrying out dynamic compensation according to the transmission delay of each output interface so as to ensure that the synchronous signals output by each interface are kept consistent in time.

Description

Technical Field

[0001] This invention belongs to the field of signal synchronization technology, and in particular relates to a device and method based on multimodal physiological signal synchronization. Background Technology

[0002] The brain is the most important part of the human body, responsible for vital functions such as movement, vision, and perception. In clinical diagnosis and neuroscience research, evoked potentials (EPPs) in brain electrical activity are measured and analyzed. However, individual EEPs are often very weak, lack significance, and are difficult to measure. Therefore, external stimuli are often needed to activate the physiological electrical activity of specific brain regions, requiring simultaneous recording of the start time of the stimulus event and the start time of the EEP data. Generally, in external stimulation experiments, multiple stimulus events are often superimposed to enhance the average evoked potential, thereby obtaining a more significant EEP signal. Therefore, synchronization of stimulation events and data is crucial during EEG signal evoked potential stimulation and data acquisition. Furthermore, modern neuroscience research has shifted from single-signal analysis to multimodal fusion analysis. For example, motor imagery studies require simultaneous acquisition of EEG (μ rhythm 8-12Hz), EMG (20-500Hz), and ECG (heart rate variability HRV) to analyze the dynamic coupling mechanism between the motor cortex and muscle activity; epileptic seizure monitoring requires combined high-frequency EEG (>500Hz), myoclonic signals (EMG), and blood oxygen saturation (SpO2) to distinguish between epileptiform discharges and motor artifacts; cognitive experiments (such as P300) require recording visual / auditory evoked potentials (EEG), skin conductance response (GSR), and eye movement data (EOG) to study attention and emotion regulation mechanisms.

[0003] There are numerous types of existing electrophysiological signal acquisition devices, including EEG devices, EMG devices, ECG devices, blood oxygenation devices, and blood pressure monitoring devices. These devices come in various models, and each operates on its own independent standard for signal acquisition and recording, making them incompatible and unrelated. However, current clinical and scientific research based on electrophysiological signals is no longer limited to the acquisition and analysis of single signals but is increasingly involving multi-dimensional, multi-channel integrated collaboration. Currently, only multiple electrophysiological acquisition devices from a few manufacturers may have some degree of synchronization or correlation. There remains a problem of ineffective synchronization between electrophysiological acquisition devices from different manufacturers, hindering the development of clinical and scientific research.

[0004] Meanwhile, with the development of science and technology and the different sampling rates of different electrophysiological signals, the amount of data transmitted by different electrophysiological signal acquisition devices varies greatly. The communication interfaces used are no longer limited to asynchronous serial port transmission. The data transmission standards and interfaces of electrophysiological acquisition devices from different manufacturers, and even different electrophysiological acquisition devices within the same manufacturer, are different. For example, EEG data acquisition devices upload data via Ethernet (1000Mbps), EMG data acquisition devices upload data via asynchronous serial port (115.2kbps), and POS data acquisition devices upload data via USB port (480Mbps). In order to simplify the signal linkage acquisition of multiple devices, wireless network-based uploading methods have gradually emerged. This kind of electrophysiological acquisition system using a hybrid interface without a unified standard has brought great technical barriers and system integration complexity to the current electrophysiological signal integration and synchronization. Summary of the Invention

[0005] The purpose of this invention is to provide a device and method based on multimodal physiological signal synchronization, which solves the problem of ineffective synchronization and linkage between electrophysiological acquisition devices from different manufacturers in the prior art.

[0006] To achieve the above objectives, the present invention provides a device based on multimodal physiological signal synchronization, comprising: Multiple synchronous trigger signal input interfaces are provided for receiving synchronous trigger signals from manual input circuits, program preset inputs, or third-party synchronous trigger devices. The FPGA control module, connected to the input interface, is used to detect the synchronization trigger signal and generate the corresponding synchronization signal and high-precision timestamp. A global unified clock management module, integrated within the FPGA control module, is used to generate and provide multiple synchronous clock signals; Multiple synchronization signal output interfaces are connected to the FPGA control module to convert synchronization signals and timestamps into signals that conform to the interface protocols of different electrophysiological devices and output them. The output calibration module, integrated within the FPGA control module, is used to dynamically compensate for the transmission delay of each output interface, ensuring that the synchronization signals output by each interface remain consistent in time.

[0007] Preferably, the multiple synchronization trigger signal input interfaces include a key input circuit, a serial communication interface, and a digital input interface for connecting to third-party synchronization devices.

[0008] Preferably, the multiple synchronous signal output interfaces include at least one of the following: USB conversion circuit, Gigabit Ethernet conversion circuit, serial port conversion circuit, parallel port conversion circuit, IO pulse conversion circuit, analog signal conversion circuit, and WIFI / Bluetooth output circuit.

[0009] Preferably, the FPGA control module also integrates a hardware tagging system. The hardware tagging system uses a hardware-level EEG synchronization event real-time detection and triggering mechanism to generate a 64-bit, 200MHz timestamp, as well as a trigger delay compensation algorithm that dynamically adjusts the ±50ns delay.

[0010] Preferably, the global unified clock management module uses the MMCM or PLL circuit inside the FPGA to generate multiple synchronous clocks from an ultra-low jitter master clock source and distribute them to the various functional modules in the FPGA control module.

[0011] Preferably, the output calibration module compensates for the inherent transmission delay of each output interface through configurable delay waiting logic, with a compensation accuracy of ±50 nanoseconds.

[0012] This invention also provides a method for synchronizing multimodal physiological signals, comprising the following steps: Step 1: Send a synchronization trigger signal through the manual input circuit, the program preset input circuit, or the input circuit of a third-party synchronization trigger device; Step 2: After the FPGA control module detects the synchronization trigger signal, it immediately generates a synchronization signal and a timestamp; Step 3: Perform output delay waiting; calibrate the transmission delay of the generated synchronization signal and timestamp according to different output interfaces; Step 4: After the waiting time is over, send the calibrated synchronization signal and timestamp to one or more electrophysiological acquisition devices through the corresponding output interface; Step 5: The electrophysiological acquisition device records physiological signals, and simultaneously records synchronization signals and timestamps to achieve time synchronization and alignment of multimodal physiological signals; Step 6: Collect physiological signals from all electrophysiological acquisition devices and synchronize and align all physiological signals based on timestamps.

[0013] Preferably, in step 3, the synchronization signal and timestamp are sent via USB protocol, TCP / UDP protocol, UART protocol, parallel port read / write logic, pulse-triggered IO, DAC control, or WIFI / Bluetooth SDIO.

[0014] Preferably, in step 3, the transmission delay calibration is to dynamically calculate and compensate for the inherent transmission delay of different interfaces such as USB, Ethernet, serial port, parallel port, IO pulse, analog output and wireless output.

[0015] Therefore, the present invention adopts the above-mentioned device and method based on multimodal physiological signal synchronization, which integrates multiple communication interfaces such as USB, UART, parallel interface, Ethernet port and wireless network, enabling high-precision synchronization of multimodal physiological signals across manufacturers and platforms. It is suitable for application scenarios such as EEG evoked potential research, brain-computer interface development and clinical neurological monitoring that require strict time alignment; it effectively solves the key technical bottlenecks in the prior art such as device heterogeneity, time scale asynchrony and interface fragmentation.

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] Figure 1 This is a hardware scheme diagram for multimodal physiological signal synchronization according to an embodiment of the present invention; Figure 2 This is a system framework diagram of the multimodal physiological signal synchronization device and electrophysiological equipment according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the FPGA program implementation framework for multimodal physiological signal synchronization according to an embodiment of the present invention. Figure 4 This is a flowchart of a multimodal physiological signal synchronization method according to an embodiment of the present invention; Figure 5 This is a code diagram of the FPGA program related modules for multimodal physiological signal synchronization according to an embodiment of the present invention; Figure 6 This is a partial interface diagram of the timing calibration module according to an embodiment of the present invention; Figure 7 This is a timing adjustment diagram of the timing calibration module - synchronous output according to an embodiment of the present invention; Figure 8 This is a timing calibration module calibration time counting diagram according to an embodiment of the present invention; Figure 9 This is a timing code diagram for network synchronization output according to an embodiment of the present invention; Figure 10 This is a partial code diagram illustrating the serial port synchronous output implementation in an embodiment of the present invention; Figure 11 This is a diagram of the synchronous output code of the SPI Bluetooth module according to an embodiment of the present invention; Figure 12 This is a code diagram of the USB synchronous output section of an embodiment of the present invention. Detailed Implementation

[0018] The following detailed description of embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0019] Please see Figures 1-12 A device based on multimodal physiological signal synchronization, comprising: Multiple synchronous trigger signal input interfaces are provided for receiving synchronous trigger signals from manual input circuits, program preset inputs, or third-party synchronous trigger devices. The manual input circuit is used to input manual commands for synchronous trigger signals; the specific circuit can be designed as a button input. The program preset input circuit is used to receive program-preset synchronous trigger commands. For example, during electrophysiological signal acquisition, when a computer-side program-preset synchronous event is triggered, the program simultaneously sends the preset synchronous event to this circuit; the communication interface can be designed as a serial port. The third-party synchronous trigger device input circuit is used to receive hardware synchronization signals from third-party synchronous event trigger devices. These three input circuits enable the input of synchronous event trigger signals for different scenarios.

[0020] The FPGA control module, connected to the input interface, is used to detect the synchronization trigger signal and generate the corresponding synchronization signal and high-precision timestamp. The FPGA control module also integrates a hardware tagging system, which uses a hardware-level EEG synchronization event to detect and trigger a 64-bit, 200MHz counting timestamp generation mechanism in real time, as well as a trigger delay compensation algorithm that dynamically adjusts the ±50ns trigger delay.

[0021] The global unified clock management module, integrated within the FPGA control module, is used to generate and provide multiple synchronous clock signals. The global unified clock management module utilizes the MMCM or PLL circuitry inside the FPGA to generate multiple synchronous clocks from an ultra-low jitter master clock source and distribute them to various functional modules within the FPGA control module.

[0022] Multiple synchronization signal output interfaces, connected to the FPGA control module, are used to convert synchronization signals and timestamps into signals conforming to the interface protocols of different electrophysiological devices and output them. These multiple synchronization signal output interfaces include at least one of the following: a USB conversion circuit, a gigabit Ethernet conversion circuit, a serial port conversion circuit, a parallel port conversion circuit, an I / O pulse conversion circuit, an analog signal conversion circuit, and a WIFI / Bluetooth output circuit. Specifically, the USB conversion circuit converts the synchronization tag and timestamp output from the FPGA from UTMI signals to USB interface signals for output to electrophysiological acquisition devices with USB interfaces; the gigabit Ethernet conversion circuit converts the synchronization tag and timestamp output from the FPGA from RGMII interface signals to network interface signals for output to electrophysiological acquisition devices with network interfaces; the serial port conversion circuit converts the synchronization tag and timestamp output from the FPGA from UTMII interface signals to network interface signals for output to electrophysiological acquisition devices with network interfaces; and the serial port conversion circuit converts the synchronization tag and timestamp output from the FPGA from UTMII interface signals to USB interface signals for output to electrophysiological acquisition devices with network interfaces. The FPGA outputs a synchronization tag and timestamp to an RS232 or RS485 level signal and outputs it to an electrophysiological acquisition device with an RS232 or RS485 interface. The parallel port conversion circuit outputs the synchronization tag and timestamp from the FPGA to an electrophysiological acquisition device with a parallel port input. The IO pulse conversion circuit outputs the synchronization tag and timestamp from the FPGA to an electrophysiological acquisition device with an IO input through an optocoupler isolation circuit. The analog signal conversion circuit converts the synchronization tag and timestamp from the FPGA to an analog signal through a DAC and inputs it synchronously to the analog input port of the electrophysiological acquisition device. The WIFI / Bluetooth output circuit converts the SDIO signal of the synchronization tag and timestamp output from the FPGA to a WIFI or Bluetooth radio frequency signal and transmits it wirelessly to an electrophysiological acquisition device with a WIFI or Bluetooth wireless receiving interface via RF.

[0023] The output calibration module, integrated within the FPGA control module, dynamically compensates for the transmission delay of each output interface to ensure that the synchronization signals output by each interface are consistent in time. Specifically, the output calibration module compensates for the inherent transmission delay of each output interface through configurable delay-waiting logic, with a compensation accuracy of ±50 nanoseconds.

[0024] Because the I / O pin functions of an FPGA can be arbitrarily configured, therefore Figure 1 The hardware scheme shown is not limited to designing a single channel for each input and input circuit; it can be designed as a multi-channel circuit as needed. The system framework for synchronizing a multimodal physiological signal synchronization device with multiple different electrophysiological devices is as follows: Figure 2 As shown. Connect the electrophysiological acquisition device to the corresponding output interface of the multimodal physiological signal synchronization device, based on the synchronization signal receiving interface used by the device. There can be 1-N electrophysiological acquisition devices for each interface. For example... Figure 3The FPGA program implementation framework for multimodal physiological signal synchronization shown first designs a system clock management module. This module generates multiple synchronous clocks from the main clock input by the hardware circuit through the FPGA's internal PLL and distributes them to the remaining modules, ensuring the synchronization of the operating clocks of each module. Three trigger detection modules (manual trigger signal detection, program-preset trigger detection, and third-party synchronous trigger detection) simultaneously detect the input of synchronous trigger signals. Since the three trigger signals typically do not occur simultaneously in the same multimodal physiological signal measurement system, as long as any of the three trigger detection modules detects a synchronous trigger signal, the synchronization signal and timestamp generation modules will be immediately triggered to generate the synchronization signal and timestamp. The generated synchronization signal and timestamp will be simultaneously output to the calibration modules of each output interface (USB output calibration, Ethernet output calibration, serial port output calibration, parallel port output calibration, IO output calibration, DAC output calibration, and wireless data calibration). Because the FPGA program runs in parallel, its data transmission time is only related to the program structure, unlike the serial operation of hardware-level software program data transmission which is related to the CPU's running state and program scheduling. Figure 2 The data transmission time of each output interface is relatively fixed. Each calibration module only needs to calibrate according to the data transmission time of the interface to ensure the synchronization signal and timestamp of each interface output. The USB protocol module assembles the synchronization information into packets according to the USB protocol and sends them to the USB port; the TCP / UDP protocol module assembles TCP / IP or UDP protocol packets and sends them to the network interface; the UART module converts the synchronization information into UART timing output to the serial port; the parallel port read / write logic outputs the data to the parallel port according to the parallel port read / write timing; the pulse trigger IO converts the synchronization information into synchronization pulses; the DAC control module transmits the synchronization information to the DAC according to the DAC read / write timing, and the DAC converts it into an analog signal output; the WIFI / Bluetooth SDIO sends the synchronization signal to the WIFI or Bluetooth module according to the SDIO timing format, and then the WIFI or Bluetooth module transmits it out wirelessly.

[0025] The process of synchronizing various electrophysiological signals using a multimodal physiological signal synchronization device is as follows: Figure 4 As shown. First, a synchronization trigger signal is sent via manual / program preset / third-party methods. Upon receiving different trigger signals, the device immediately generates a synchronization signal and performs output calibration according to the signal transmission interface, i.e., performs an output delay wait. After the wait time ends, the synchronization signal is immediately sent out. When the corresponding electrophysiological equipment receives the synchronization signal, it generates a timestamp while recording the electrophysiological signal. Subsequently, all electrophysiological equipment data is collected. The detailed steps are as follows: Step 1: Send a synchronization trigger signal through the manual input circuit, the program preset input circuit, or the input circuit of a third-party synchronization trigger device; Step 2: After the FPGA control module detects the synchronization trigger signal, it immediately generates a synchronization signal and a timestamp; Step 3: Perform output delay waiting; calibrate the transmission delay of the generated synchronization signal and timestamp according to different output interfaces; wherein, the synchronization signal and timestamp are sent via USB protocol, TCP / UDP protocol, UART protocol, parallel port read / write logic, pulse-triggered IO, DAC control, or WIFI / Bluetooth SDIO; the transmission delay calibration is to dynamically calculate and compensate for the inherent transmission delay of different interfaces such as USB, Ethernet, serial port, parallel port, IO pulse, analog output, and wireless output. Step 4: After the waiting time is over, send the calibrated synchronization signal and timestamp to one or more electrophysiological acquisition devices through the corresponding output interface; Step 5: The electrophysiological acquisition device records physiological signals, and simultaneously records synchronization signals and timestamps to achieve time synchronization and alignment of multimodal physiological signals; Step 6: Collect physiological signals from all electrophysiological acquisition devices and synchronize and align all physiological signals based on timestamps.

[0026] The specific program code implemented in the entire method is as follows: Figures 5-12 As shown.

[0027] Therefore, this invention employs the aforementioned device and method based on multimodal physiological signal synchronization. Firstly, it proposes a hardware design with multiple synchronous trigger signal input interfaces and multiple synchronous signal output interfaces, thereby simplifying the implementation of signal synchronization in the multimodal electrophysiological signal acquisition system. Then, utilizing the field programmability and parallel hardware-based operation of FPGAs, it solves the problem of inconsistent transmission times of synchronous signals across multiple output interfaces by employing interface output delay calibration. This greatly simplifies the problem of multimodal physiological signal synchronization and also addresses the issue of ineffective synchronization between electrophysiological acquisition devices from different manufacturers, bringing convenience to the development of clinical and scientific research.

[0028] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A device for synchronization based on multi-modal physiological signals, characterized in that, include: Multiple synchronous trigger signal input interfaces are provided for receiving synchronous trigger signals from manual input circuits, program preset inputs, or third-party synchronous trigger devices. The FPGA control module, connected to the input interface, is used to detect the synchronization trigger signal and generate the corresponding synchronization signal and high-precision timestamp. A global unified clock management module, integrated within the FPGA control module, is used to generate and provide multiple synchronous clock signals; Multiple synchronization signal output interfaces are connected to the FPGA control module to convert synchronization signals and timestamps into signals that conform to the interface protocols of different electrophysiological devices and output them. The output calibration module, integrated within the FPGA control module, is used to dynamically compensate for the transmission delay of each output interface, ensuring that the synchronization signals output by each interface remain consistent in time.

2. The device based on multi-modal physiological signal synchronization according to claim 1, characterized in that: Multiple synchronization trigger signal input interfaces include a key input circuit, a serial communication interface, and a digital input interface for connecting third-party synchronization devices.

3. The device for synchronization based on multi-modal physiological signals according to claim 2, characterized in that, The various synchronous signal output interfaces include at least one of the following: USB conversion circuit, Gigabit Ethernet conversion circuit, serial port conversion circuit, parallel port conversion circuit, IO pulse conversion circuit, analog signal conversion circuit, and WIFI / Bluetooth output circuit.

4. The device for synchronization based on multi-modal physiological signals according to claim 3, characterized in that: The FPGA control module also integrates a hardware tagging system. The hardware tagging system uses a hardware-level EEG synchronization event real-time detection and triggering mechanism to generate a 64-bit, 200MHz timestamp, as well as a trigger delay compensation algorithm that dynamically adjusts the ±50ns delay.

5. The device for synchronization based on multi-modal physiological signals according to claim 4, characterized in that: The global unified clock management module uses the MMCM or PLL circuit inside the FPGA to generate multiple synchronous clocks from an ultra-low jitter master clock source and distribute them to various functional modules in the FPGA control module.

6. The device for synchronization based on multi-modal physiological signals according to claim 5, characterized in that: The output calibration module compensates for the inherent transmission delay of each output interface through configurable delay waiting logic, with a compensation accuracy of ±50 nanoseconds.

7. A method for multi-modal physiological signal synchronization based on the device for multi-modal physiological signal synchronization according to any one of claims 1-6, characterized in that, Includes the following steps: Step 1: Send a synchronization trigger signal through the manual input circuit, the program preset input circuit, or the input circuit of a third-party synchronization trigger device; Step 2: After the FPGA control module detects the synchronization trigger signal, it immediately generates a synchronization signal and a timestamp; Step 3: Execute an output delay and wait; The generated synchronization signals and timestamps are calibrated for transmission delay according to different output interfaces; Step 4: After the waiting time is over, send the calibrated synchronization signal and timestamp to one or more electrophysiological acquisition devices through the corresponding output interface; Step 5: The electrophysiological acquisition device records physiological signals, and simultaneously records synchronization signals and timestamps to achieve time synchronization and alignment of multimodal physiological signals; Step 6: Collect physiological signals from all electrophysiological acquisition devices and synchronize and align all physiological signals based on timestamps.

8. The method of claim 7, wherein: In step 3, the synchronization signal and timestamp are sent via USB protocol, TCP / UDP protocol, UART protocol, parallel port read / write logic, pulse-triggered IO, DAC control, or WIFI / Bluetooth SDIO.

9. The method of claim 8, wherein: The transmission delay calibration in step 3 is dynamic calculation and compensation for inherent transmission delay of different interfaces of USB, Ethernet, serial port, parallel port, IO pulse, analog output and wireless output.

Images