Data synchronization method and device, storage medium and electronic equipment
By calibrating the clock and determining the time identification information of the synchronization signal, the problem of poor matching accuracy between neural signals and behavioral videos is solved, and high-precision synchronous registration between multiple data to be synchronized is achieved.
Patent Information
- Application Number
- CN202311543578.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-20
- Publication Date
- 2025-05-20
AI Technical Summary
In the prior art, the matching accuracy of neural signals and behavioral videos is poor, and the clock sources are different between multiple experimental devices, resulting in the matching accuracy of data synchronization being affected.
By acquiring the current time and calibrating the clock of the first device, multiple synchronization signals of the target experimental organism, including the acquisition start signal of the miniaturized two-photon microscope and the exposure signal of the behavioral camera, the time identification information corresponding to each of these signals is determined and output to the second device for synchronous registration between the multiple data to be synchronized.
High-precision synchronous registration between multiple data to be synchronized is realized, which avoids time errors caused by human factors and cumulative errors between multiple devices, and improves the accuracy of data matching.
Smart Images
Figure CN120021179A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the fields of neuroscience and biomedical imaging technology, and particularly relates to a data synchronization method, apparatus, storage medium, and electronic device. Background Art
[0002] The correlation between neural signals and behavioral actions is an important research topic in the field of neuroscience. Researchers usually use electroencephalogram, electromyogram, behavioral tests, etc. to record and analyze the changes in neural signals and behavioral actions. These data can help researchers understand the causal relationship between neural signals and behavioral actions, as well as their interaction relationship under specific environments.
[0003] When collecting neuron signals and behavioral data of experimental animals, an LED is set in the animal behavior device. When starting an experiment, the LED is lit simultaneously, and the start time of the behavioral video and image data is matched through the lighting time of the LED. When an artificial excitation event (such as electrical stimulation) occurs, the LED is also lit to indicate the occurrence time of the excitation event in the behavioral video. However, in the behavioral video, there is usually a delay between the LED lighting and the start of the image data. Moreover, the LED usually has a short duration of lighting, and multiple processes will introduce synchronization errors, resulting in poor matching accuracy between neural signals and the behavioral video.
[0004] In addition, in some behavioral devices in the related art (for example, electrical stimulation devices), time stamps are generated synchronously when an action occurs. However, due to the different clock sources among multiple experimental devices, the time stamps generated by each experimental device will cause cumulative errors, affecting the matching accuracy of data synchronization. Summary of the Invention
[0005] In view of this, embodiments of the present disclosure provide a data synchronization method, apparatus, storage medium, and electronic device.
[0006] In a first aspect, an embodiment of the present disclosure provides a data synchronization method, which is applied to a first device. The first device is signal-connected to a second device. The method includes: obtaining the current time and calibrating the clock of the first device based on the current time; obtaining a plurality of synchronization signals corresponding to a target experimental organism, where the plurality of synchronization signals correspond to a plurality of data to be synchronized. Among them, the plurality of synchronization signals include an acquisition start signal of a miniaturized two-photon microscope and an exposure signal of a behavioral camera. The plurality of data to be synchronized are used to characterize the physical signs of the target experimental organism. The plurality of data to be synchronized include two-photon image data of the target experimental organism and behavioral video data of the target experimental organism; based on the clock of the first device, determining time identification information corresponding to each of the plurality of synchronization signals; outputting the time identification information corresponding to each of the plurality of synchronization signals to the second device, so that the second device performs synchronization registration between the plurality of data to be synchronized based on the time identification information corresponding to each of the plurality of synchronization signals.
[0007] In combination with the first aspect, in some implementation manners of the first aspect, determining the time identification information corresponding to each of the plurality of synchronization signals based on the clock of the first device includes: respectively determining a plurality of occurrence identifications of the plurality of synchronization signals, where the occurrence identification corresponds to one acquisition of the miniaturized two-photon microscope or one exposure of the behavioral camera; based on the clock of the first device, respectively determining the generation time of each of the plurality of occurrence identifications, and using the generation time as the time identification information corresponding to the occurrence identification.
[0008] In combination with the first aspect, in some implementation manners of the first aspect, the synchronization signal is a TTL level signal. Respecting determining the plurality of occurrence identifications of the plurality of synchronization signals includes: when the rising edge of the synchronization signal arrives, interrupting the currently executing program; generating an occurrence identification corresponding to the rising edge.
[0009] In combination with the first aspect, in some implementation manners of the first aspect, outputting the time identification information corresponding to each of the plurality of synchronization signals to the second device includes: determining the reporting period of the time identification information based on the memory of the first device and the acquisition frequency of the miniaturized two-photon microscope and / or the exposure frequency of the behavioral camera; caching the time identification information and the occurrence identification corresponding to the time identification information within the reporting period; respectively packing and outputting the cached time identification information and occurrence identification within each reporting period to the second device.
[0010] Second aspect, an embodiment of the present disclosure provides a data synchronization method, which is applied to a second device. The second device is signal - connected to a first device. The method includes: obtaining time - identification information corresponding to each of a plurality of synchronization signals sent by the first device for a target experimental organism, where the plurality of synchronization signals correspond to a plurality of data to be synchronized. Among them, the plurality of synchronization signals include an acquisition start signal of a miniaturized two - photon microscope and an exposure signal of a behavioral camera. The plurality of data to be synchronized are used to characterize the physical signs of the target experimental organism, and the plurality of data to be synchronized include two - photon image data of the target experimental organism and behavioral video data of the target experimental organism; obtaining the plurality of data to be synchronized corresponding to the plurality of synchronization signals; based on the time - identification information corresponding to each of the plurality of synchronization signals, performing synchronous registration between the plurality of data to be synchronized. Among them, the time - identification information corresponding to each of the plurality of synchronization signals is generated by the first device based on the method of the first aspect.
[0011] In combination with the second aspect, in some implementation manners of the second aspect, performing synchronous registration between the plurality of data to be synchronized based on the time - identification information corresponding to each of the plurality of synchronization signals includes: based on the time - identification information corresponding to each of the plurality of synchronization signals, respectively determining the acquisition time of the two - photon image data and the exposure time of each video frame in the behavioral video data; arranging the two - photon image data in chronological order based on the acquisition time of the two - photon image data; associating the two - photon image data with the video frame that is closest in time, so as to determine the correspondence between the two - photon image data and the video frames in the behavioral video data.
[0012] Third aspect, an embodiment of the present disclosure provides a data synchronization device, which is applied to a first device. The first device is signal - connected to a second device. The device includes: a first acquisition module, configured to obtain the current time and calibrate the clock of the first device based on the current time; a second acquisition module, configured to obtain a plurality of synchronization signals corresponding to a target experimental organism, where the plurality of synchronization signals correspond to a plurality of data to be synchronized. Among them, the plurality of synchronization signals include an acquisition start signal of a miniaturized two - photon microscope and an exposure signal of a behavioral camera. The plurality of data to be synchronized are used to characterize the physical signs of the target experimental organism, and the plurality of data to be synchronized include two - photon image data of the target experimental organism and behavioral video data of the target experimental organism; a determination module, configured to determine the time - identification information corresponding to each of the plurality of synchronization signals based on the clock of the first device; an output module, configured to output the time - identification information corresponding to each of the plurality of synchronization signals to the second device, so that the second device performs synchronous registration between the plurality of data to be synchronized based on the time - identification information corresponding to each of the plurality of synchronization signals.
[0013] Fourthly, an embodiment of the present disclosure provides a data synchronization device, which is applied to a second device. The second device is signal-connected to a first device. The device includes: a first acquisition module, configured to acquire time identification information corresponding to each of a plurality of synchronization signals sent by the first device for a target experimental organism. The plurality of synchronization signals correspond to a plurality of data to be synchronized. Among them, the plurality of synchronization signals include an acquisition start signal of a miniaturized two-photon microscope and an exposure signal of an ethology camera. The plurality of data to be synchronized are used to characterize the physical signs of the target experimental organism. The plurality of data to be synchronized include two-photon image data of the target experimental organism and ethology video data of the target experimental organism; a second acquisition module, configured to acquire the plurality of data to be synchronized corresponding to the plurality of synchronization signals; a registration module, configured to perform synchronous registration between the plurality of data to be synchronized based on the time identification information corresponding to each of the plurality of synchronization signals.
[0014] Fifthly, an embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored. The computer program, when executed by a processor, implements the data synchronization methods in the first aspect and the second aspect.
[0015] Sixthly, an embodiment of the present disclosure provides an electronic device, including: a plurality of input interfaces, configured to access a plurality of synchronization signals; a processor, configured to implement the data synchronization method in the first aspect; a communication interface, configured to perform data transmission with an external computer.
[0016] In the embodiments of the present disclosure, determining the generation time of the data to be synchronized based on the synchronization signal can avoid time errors caused by human factors; at the same time, determining the generation time of each unit of the data to be synchronized based on the same clock can achieve synchronous registration of each unit of data and avoid cumulative errors during the registration process. Therefore, the data synchronization method provided by the embodiments of the present disclosure can achieve high-precision synchronous registration among a plurality of data to be synchronized. Description of the Drawings
[0017] Figure 1 The figure shows a schematic diagram of an application scenario of the data synchronization method provided by an exemplary embodiment of the present disclosure.
[0018] Figure 2 The figure shows a schematic flowchart of the data synchronization method provided by an exemplary embodiment of the present disclosure.
[0019] Figure 3 The figure shows a schematic flowchart of the step of determining the time identification information corresponding to each of a plurality of synchronization signals based on the clock of the first device provided by an exemplary embodiment of the present disclosure.
[0020] Figure 4 The figure shows a schematic flowchart of the step of respectively determining the occurrence identifications of a plurality of synchronization signals provided by an exemplary embodiment of the present disclosure.
[0021] Figure 5 The figure shows a timing schematic diagram of multiple TTL level signals provided by an exemplary embodiment of the present disclosure.
[0022] Figure 6 The figure shows a flowchart of the step of outputting the time identification information corresponding to each of the multiple synchronization signals to a second device provided by an exemplary embodiment of the present disclosure.
[0023] Figure 7 The figure shows a flowchart of the step of performing synchronization registration between multiple data to be synchronized based on the time identification information corresponding to each of the multiple synchronization signals provided by an exemplary embodiment of the present disclosure.
[0024] Figure 8 The figure shows a structural schematic diagram of a data synchronization device provided by an exemplary embodiment of the present disclosure.
[0025] Figure 9 The figure shows a structural schematic diagram of a data synchronization device provided by another exemplary embodiment of the present disclosure.
[0026] Figure 10 The figure shows a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners
[0027] Next, the technical solutions in the embodiments of the present disclosure will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without creative efforts shall fall within the protection scope of the present disclosure.
[0028] In the fields of brain science and neuroscience, miniaturized two-photon imaging technology is often used to collect real-time neural signals of experimental animals in a free movement state. When analyzing neural image signals, it is necessary to correspond the two-photon images with the behavioral actions of the experimental animals to study the correlation between the neural signals and behavioral actions of the experimental animals and understand how the central nervous system converts stimuli into natural behaviors. For example, how the neural signals act when the experimental animal eats.
[0029] As described above, the related art uses the lighting of the LED of the animal behavior device as the starting moment of the behavioral video and the two-photon image data, and synchronizes the two-photon image representing the neural signal and the behavioral video of the experimental animal based on the starting moment. The existing LED-based synchronization scheme has the following problems: 1. There is usually a delay between the LED lighting and the start of image data. Moreover, the LED usually has a short duration of lighting, and there is a sampling interval when the camera records a video. Therefore, using the LED lighting as the starting moment of the behavioral video and two-photon image data will introduce a synchronization error of several hundred milliseconds to 1 - 2 seconds, resulting in a poor matching accuracy between the two-photon image data and the behavioral video. 2. Only the two-photon image data at the starting moment can be aligned with the behavioral video, and it is impossible to synchronize each two-photon image with the behavioral video. 3. When multiple behavioral cameras are used for co - shooting, synchronization between multiple cameras cannot be achieved.
[0030] In the face of the above technical problems, the present disclosure provides a data synchronization method, which is applied to a first device. The first device is signal - connected to a second device. The method includes: obtaining the current time and calibrating the clock of the first device based on the current time; obtaining a plurality of synchronization signals corresponding to a target experimental organism, where the plurality of synchronization signals correspond to a plurality of data to be synchronized. Among them, the plurality of synchronization signals include the acquisition start signal of the miniaturized two - photon microscope and the exposure signal of the behavioral camera. The plurality of data to be synchronized are used to characterize the physical signs of the target experimental organism, and the plurality of data to be synchronized include the two - photon image data of the target experimental organism and the behavioral video data of the target experimental organism; determining the time identification information corresponding to each of the plurality of synchronization signals based on the clock of the first device; outputting the time identification information corresponding to each of the plurality of synchronization signals to the second device, so that the second device performs synchronization registration between the plurality of data to be synchronized based on the time identification information corresponding to each of the plurality of synchronization signals. Through the method of the present disclosure, it is possible to determine the acquisition time of each two - photon image data and the exposure time of each video frame in the behavioral video based on the synchronization signal, and then realize the synchronization registration of each two - photon image and the behavioral video, reducing the synchronization error between the two - photon image and the behavioral video.
[0031] Figure 1 The following shows a schematic diagram of the application scenario of the data synchronization method provided by an exemplary embodiment of the present disclosure. As Figure 1 shown, the present implementation environment includes a behavioral device 110, a first device 120, and a second device 130. The behavioral device 110, the first device 120, and the second device 130 are signal - connected to each other.
[0032] The behavioral apparatus 110 includes a miniaturized two-photon microscope 111 and a behavioral camera 112, and the number of the miniaturized two-photon microscope 111 and the behavioral camera 112 can be one or more. When conducting an experiment, the miniaturized two-photon microscope 111 is used to collect two-photon images corresponding to the target experimental organism, and at the same time, the behavioral camera 112 is used to collect behavioral videos corresponding to the target experimental organism. If the behavioral apparatus 110 includes multiple behavioral cameras 112 at the same time, then the multiple behavioral cameras 112 can respectively collect the behavioral characteristics of the target experimental organism from different angles to obtain multiple behavioral videos taken from different angles.
[0033] The first device 120 and the second device 130 are used to execute the data synchronization method mentioned in the embodiments of the present disclosure.
[0034] Specifically, the first device 120 can be a single-chip microcomputer, a complex programmable logic device (CPLD), a field programmable gate array (FPGA), or a computing device composed of application-specific integrated circuits, which can receive the synchronization signals generated by the miniaturized two-photon microscope 111 and the behavioral camera 112, and generate time identification information corresponding to the synchronization signals based on the data synchronization method of the embodiments of the present disclosure, and output the time identification information to the second device 130.
[0035] The second device 130 can receive the time identification information, as well as the two-photon image data and the behavioral video data, and perform high-precision synchronous registration between the two-photon image data and the behavioral video data based on the data synchronization method of the embodiments of the present disclosure. The second device 130 can be a terminal, such as a smart phone, a desktop computer, etc.; or it can also be a server, such as an independent physical server, or a cloud server capable of performing cloud computing, etc.
[0036] In some embodiments, the behavioral apparatus 110 further includes an excitation device 113. The excitation device 113 can apply electrical stimulation, food or water supply, light stimulation, sound stimulation, etc. to the target experimental organism to affect the behavioral response of the experimental animal. Similarly, when the excitation device 113 performs an excitation behavior, a synchronization signal will be generated synchronously. The synchronization signal of the excitation device 113 is transmitted to the first device 120, and the first device 120 generates corresponding time identification information and outputs it to the second device 130. The second device 130 can align the excitation behavior with the neural signals and behavioral actions generated by the excitation based on the time identification information.
[0037] Next, in combination with the accompanying drawings of the specification, the data synchronization method provided by the embodiments of the present disclosure will be illustrated by way of examples from the aspects of the first device and the second device respectively. Among them, the first device and the second device are signal-connected and can realize data transmission with each other.
[0038] Figure 2The following is a schematic flowchart of a data synchronization method provided by an exemplary embodiment of the present disclosure. As Figure 2 shown, for the first device, the data synchronization method provided by the embodiments of the present disclosure includes the following steps.
[0039] S210. Obtain the current time and calibrate the clock of the first device based on the current time.
[0040] Before the experiment starts, the first device can obtain the current time through the second device connected thereto, calibrate the clock in the first device based on the current time, and then perform timing based on the high-stability crystal oscillator in the first device to improve the accuracy of the time identification information in the embodiments of the present disclosure.
[0041] S220. Obtain a plurality of synchronization signals corresponding to the target experimental organism.
[0042] When conducting an experiment on the target experimental organism, a plurality of data to be synchronized are generated simultaneously. The data to be synchronized can characterize the physical signs of the target experimental organism, such as the neural signals or behavioral actions of the target experimental organism; specifically, the data to be synchronized can be two-photon image data capable of characterizing neuron activity, or behavioral video data capable of characterizing behavioral actions.
[0043] When shooting two-photon image data, each time the miniaturized two-photon microscope collects data, an image of the neurons of the target experimental organism (i.e., a two-photon image) is obtained, and a collection start signal is synchronously generated. There is a corresponding relationship between the collection start signal and the two-photon image data. Similarly, when shooting behavioral video data, each time a video frame of the behavioral video is shot, an exposure signal is synchronously generated. There is a corresponding relationship between the exposure signal and the video frame of the behavioral video data. In the embodiments of the present disclosure, the above-mentioned collection start signal and exposure signal are collectively referred to as synchronization signals.
[0044] In addition, the data to be synchronized can also be the application information of artificial excitation events. Each time an excitation event is applied, the excitation device can synchronously generate a signal as the synchronization signal of the application information.
[0045] Different from the signals generated by artificially lighting an LED in the related art, the synchronization signals are generated by the acquisition device of the data to be synchronized itself, so that time errors caused by human factors can be avoided.
[0046] S230. Based on the clock of the first device, determine the time identification information corresponding to each of the plurality of synchronization signals.
[0047] When capturing a synchronization signal, the time of capturing the synchronization signal is determined by the clock of the first device as the time identification information corresponding to the synchronization signal.
[0048] The first device is signal - connected to the device that generates the synchronization signal through physical media such as cables, optical fibers, and twisted - pair wires, avoiding signal attenuation or interference during transmission. At the same time, the signal delay is small, and the time to capture the synchronization signal can be approximately equal to the generation time of each unit of data to be synchronized (for example, each video frame).
[0049] It can be understood that data to be synchronized such as behavioral videos is generated through multiple exposures, and each exposure corresponds to a synchronization signal. Therefore, a unit of data to be synchronized usually corresponds to multiple time identification information.
[0050] S240, output the time identification information corresponding to each of the multiple synchronization signals to the second device.
[0051] Output the generated time identification information to the second device. In this way, the second device can determine the generation time of each unit of data to be synchronized based on the time identification information, and further determine the correspondence relationship of multiple units of data to be synchronized through the generation time, realizing the synchronous registration between multiple units of data to be synchronized.
[0052] In the embodiments of the present disclosure, determining the generation time of data to be synchronized based on the synchronization signal can avoid time errors caused by human factors; at the same time, determining the generation time of each unit of data to be synchronized based on the same clock can achieve the synchronous registration of each unit of data and avoid cumulative errors during the registration process. Therefore, the data synchronization method provided by the embodiments of the present disclosure can achieve high - precision synchronous registration between multiple units of data to be synchronized.
[0053] Next, in combination with Figure 3 A specific implementation manner for determining the time identification information is further introduced.
[0054] As Figure 3 shown, the steps of determining the time identification information corresponding to each of the multiple synchronization signals based on the clock of the first device provided by an exemplary embodiment of the present disclosure specifically include the following steps.
[0055] S231, respectively determine multiple occurrence identifications of the multiple synchronization signals.
[0056] When capturing the synchronization signal, an occurrence identification is automatically generated, and there is a corresponding relationship between the occurrence identification and the synchronization signal. At the same time, when the first device accesses multiple channels of synchronization signals simultaneously, the occurrence identification carries the identity identification of the access channel.
[0057] Exemplarily, the signal of the miniaturized two-photon microscope is connected to the first device through channel 1, and the behavioral camera 1 and the behavioral camera 2 are respectively connected to the first device through channel 2 and channel 3. When the miniaturized two-photon microscope acquires and generates a two-photon image, the synchronization signal generated synchronously with the two-photon image is transmitted into the first device through channel 1; after the first device captures the synchronization signal, a corresponding occurrence identifier is generated, and the occurrence identifier carries the identity identifier of channel 1. When the behavioral camera 1 exposes and generates a video frame, the synchronization signal generated synchronously with the video frame is transmitted into the first device through channel 2; after the first device captures the synchronization signal, a corresponding occurrence identifier is generated, and the occurrence identifier carries the identity identifier of channel 2.
[0058] In addition, the first device can also be connected to behavioral devices such as an excitation device. When an action of the behavioral device occurs, the first device generates an occurrence identifier corresponding to the action.
[0059] S232, Based on the clock of the first device, respectively determine the generation time of each of the multiple occurrence identifiers, and use the generation time as the time identifier information corresponding to the occurrence identifier.
[0060] Determine the time when the occurrence identifier is generated through the clock of the first device as the time identifier information corresponding to the occurrence identifier. In this way, a correspondence relationship is established among a unit of data to be synchronized, the synchronization signal, the occurrence identifier, and the time identifier information.
[0061] In the embodiments of the present disclosure, a correspondence relationship can be established between the synchronization signal and the time identifier information. Therefore, through the time identifier information, the time when each unit of data to be synchronized is generated can be determined.
[0062] The synchronization signal in the above embodiments can be an analog signal or a digital signal. Taking the TTL level signal as an example below, a specific implementation manner for determining the occurrence identifier is further introduced. As Figure 4 shown, the steps of respectively determining multiple occurrence identifiers of multiple synchronization signals provided by an exemplary embodiment of the present disclosure specifically include the following steps.
[0063] S2311, When the rising edge of the synchronization signal arrives, interrupt the currently executing program.
[0064] The TTL level signal is a digital signal. It uses high and low levels to represent logical states and has the advantages of simplicity, reliability, and wide application.
[0065] Figure 5 Shown is a timing diagram of multiple TTL level signals provided by an exemplary embodiment of the present disclosure. As Figure 5As shown, each signal corresponds to a data to be synchronized, and each high level in the signal corresponds to a synchronization signal. Therefore, in order to reduce the time error of synchronization signal capture, when the rising edge of the synchronization signal arrives, the processor of the first device interrupts the currently executing program to generate an occurrence identifier in real time. Herein, the rising edge of the synchronization signal refers to the instant when the signal waveform changes from low level to high level.
[0066] S2312, generate the occurrence identifier corresponding to the rising edge.
[0067] The processor of the first device synchronously generates the occurrence identifier corresponding to the rising edge.
[0068] In the embodiments of the present disclosure, it is possible to generate the corresponding occurrence identifier in real time in response to the rising edge of the synchronization signal, reduce the time error of synchronization signal capture, and improve the accuracy of the method provided by the embodiments of the present disclosure.
[0069] After determining the time identifier information, it is necessary to send the time identifier information to the second device so that the second device can perform synchronous registration among multiple data to be synchronized based on the time identifier information. The first device and the second device can be connected through communication interfaces such as a serial communication interface, a universal serial bus (USB), and Ethernet.
[0070] When the first device transmits the time identifier information to the second device, since the generation speed of the time identifier information is too fast, it is not suitable to transmit it to the second device in real time.
[0071] In some embodiments, a certain number of time identifier information can be cached in the first device and sent to the second device in a packaged manner. Specifically, as Figure 6 shown, the steps of outputting the time identifier information corresponding to each of the multiple synchronization signals to the second device provided by an exemplary embodiment of the present disclosure specifically include the following steps.
[0072] S241, determine the reporting period of the time identifier information based on the memory of the first device and the acquisition frequency of the miniaturized two-photon microscope and / or the exposure frequency of the behavioral camera.
[0073] Considering that the generation speed of the time identifier information is related to the generation speed of the synchronization signal, the acquisition frequency of the miniaturized two-photon microscope and / or the exposure frequency of the behavioral camera, and the memory of the first device jointly determine the reporting period of the first device.
[0074] S242, cache the time identifier information and the occurrence identifier corresponding to the time identifier information within the reporting period.
[0075] Cache the time identifier information generated within one reporting period and the occurrence identifier corresponding to the time identifier information in the first device until the end of this reporting period.
[0076] Specifically, when the time identification information is generated, check whether the current time meets the reporting period; if not, cache the time identification information and the corresponding occurrence identification; if so, send the cached data of the first device to the second device.
[0077] S243. Pack the time identification information and the occurrence identification cached within each reporting period and output them to the second device respectively.
[0078] Specifically, when checking whether the current time meets the reporting period, poll each channel for data to be output, and pack all the data to be output and output it to the second device.
[0079] In the embodiments of the present disclosure, by setting the reporting period and caching the time identification information and the occurrence identification within the reporting period in the first device, the transmission pressure of the first device is reduced.
[0080] The following illustrates the data synchronization method provided by the embodiments of the present disclosure from the perspective of the second device.
[0081] Continue to refer to Figure 2 , for the second device, the data synchronization method provided by the embodiments of the present disclosure includes the following steps.
[0082] S250. Obtain the time identification information corresponding to each of the multiple synchronization signals of the target experimental organism sent by the first device.
[0083] The second device receives the time identification information corresponding to each of the multiple synchronization signals sent by the first device. Among them, the time identification information corresponding to each of the multiple synchronization signals is generated by the first device based on the method provided by the embodiments of the present disclosure.
[0084] S260. Obtain multiple pieces of data to be synchronized corresponding to the multiple synchronization signals.
[0085] As introduced in the above embodiments, each piece of data to be synchronized corresponds to one signal channel, and each signal channel corresponds to multiple synchronization signals respectively.
[0086] The second device is signal-connected to acquisition devices such as a miniaturized two-photon microscope and a behavioral camera to obtain the data to be synchronized corresponding to the synchronization signals.
[0087] S270. Based on the time identification information corresponding to each of the multiple synchronization signals, perform synchronous registration among the multiple pieces of data to be synchronized.
[0088] After obtaining the data to be synchronized and the time identification information, it is possible to determine the generation time of the data to be synchronized based on the time identification information, and then realize the synchronous registration among the multiple pieces of data to be synchronized.
[0089] The following combines with Figure 7 to further introduce a specific implementation manner of synchronization registration among multiple data to be synchronized.
[0090] As Figure 7 shown, the synchronization registration steps for multiple data to be synchronized based on the time identification information corresponding to each of multiple synchronization signals provided by an exemplary embodiment of the present disclosure specifically include the following steps.
[0091] S271. Based on the time identification information corresponding to each of the multiple synchronization signals, respectively determine the acquisition time of the two-photon image data and the exposure time of each video frame in the behavioral video data.
[0092] Among them, the time identification information corresponding to each of the multiple synchronization signals is generated by the first device based on the method provided by the embodiments of the present disclosure. Therefore, each piece of time identification information corresponds to a two-photon image or a video frame respectively.
[0093] Based on the time identification information, the acquisition time of each two-photon image data and the exposure time of each video frame in the behavioral video data can be respectively determined. Since the above exposure time is determined based on the clock of the first device, the time error between multiple acquisition devices is avoided.
[0094] S272. Arrange the two-photon image data in chronological order based on the acquisition time of the two-photon image data.
[0095] After determining the acquisition time of each two-photon image, the two-photon image data can be arranged in the generated chronological order. At the same time, each video frame in the behavioral video data is also arranged based on the chronological order.
[0096] S273. Associate the two-photon image data with the video frame that is closest in time to determine the correspondence between the two-photon image data and the video frames in the behavioral video data.
[0097] If multiple miniaturized two-photon microscopes and / or multiple behavioral cameras simultaneously acquire the data to be synchronized of the target experimental organism. Then, each unit of data in the multiple data to be synchronized can be aligned based on the absolute time to establish the correspondence between each unit of data in the multiple data to be synchronized.
[0098] In the embodiments of the present disclosure, the synchronization registration of each unit of data can be achieved, and the cumulative error in the registration process can be avoided to achieve high-precision synchronization registration among multiple data to be synchronized.
[0099] The above combines with Figures 2 to 7 to describe in detail the embodiments of the data synchronization method of the present disclosure. The following combines with Figure 8 、 Figure 9, embodiments of the data synchronization device of the present disclosure are described in detail. It should be understood that the description of the embodiments of the data synchronization method corresponds to the description of the embodiments of the data synchronization device. Therefore, for parts not described in detail, reference may be made to the foregoing method embodiments.
[0100] Figure 8 The following is a schematic structural diagram of a data synchronization device provided by an exemplary embodiment of the present disclosure. Exemplarily, the data synchronization device is applied to a first device. As Figure 8 shown, the data synchronization device 800 provided by the embodiments of the present disclosure includes: a first acquisition module 810, configured to acquire the current time and calibrate the clock of the first device based on the current time; a second acquisition module 820, configured to acquire a plurality of synchronization signals corresponding to a target experimental organism, the plurality of synchronization signals corresponding to a plurality of data to be synchronized, wherein the plurality of synchronization signals include an acquisition start signal of a miniaturized two-photon microscope and an exposure signal of a behavioral camera, the plurality of data to be synchronized are used to characterize the physical signs of the target experimental organism, and the plurality of data to be synchronized include two-photon image data of the target experimental organism and behavioral video data of the target experimental organism; a determination module 830, configured to determine time identification information corresponding to each of the plurality of synchronization signals based on the clock of the first device; an output module 840, configured to output the time identification information corresponding to each of the plurality of synchronization signals to a second device, so that the second device performs synchronization registration between the plurality of data to be synchronized based on the time identification information corresponding to each of the plurality of synchronization signals.
[0101] In some embodiments, the determination module 830 is further configured to: respectively determine a plurality of occurrence identifiers of the plurality of synchronization signals, where the occurrence identifier corresponds to one acquisition of the miniaturized two-photon microscope or one exposure of the behavioral camera; based on the clock of the first device, respectively determine the generation time of each of the plurality of occurrence identifiers, and use the generation time as the time identification information corresponding to the occurrence identifier.
[0102] In some embodiments, the synchronization signal is a TTL level signal, and the determination module 830 is further configured to: when the rising edge of the synchronization signal arrives, interrupt the currently executing program; generate an occurrence identifier corresponding to the rising edge.
[0103] In some embodiments, the output module 840 is further configured to: determine a reporting period of the time identification information based on the memory of the first device and the acquisition frequency of the miniaturized two-photon microscope and / or the exposure frequency of the behavioral camera; cache the time identification information and the occurrence identifier corresponding to the time identification information within the reporting period; respectively pack and output the time identification information and the occurrence identifier cached within each reporting period to the second device.
[0104] Figure 9The following is a schematic structural diagram of a data synchronization device provided by another exemplary embodiment of the present disclosure. Exemplarily, the data synchronization device is applied to a second device. As Figure 9 shown, the data synchronization device 900 provided by the embodiment of the present disclosure includes: a first acquisition module 910, configured to acquire time identification information corresponding to each of a plurality of synchronization signals sent by a first device, where the plurality of synchronization signals correspond to a plurality of data to be synchronized, and among the plurality of synchronization signals, there are an acquisition start signal of a miniaturized two-photon microscope and an exposure signal of a behavioral camera, the plurality of data to be synchronized are used to characterize the physical signs of a target experimental organism, and among the plurality of data to be synchronized, there are two-photon image data of the target experimental organism and behavioral video data of the target experimental organism; a second acquisition module 920, configured to acquire the plurality of data to be synchronized corresponding to the plurality of synchronization signals; a registration module 930, configured to perform synchronization registration between the plurality of data to be synchronized based on the time identification information corresponding to each of the plurality of synchronization signals.
[0105] In some embodiments, the registration module 930 is further configured to: respectively determine the acquisition time of the two-photon image data and the exposure time of each video frame in the behavioral video data based on the time identification information corresponding to each of the plurality of synchronization signals; arrange the two-photon image data in chronological order based on the acquisition time of the two-photon image data; and associate the two-photon image data with the video frame closest in time to determine the correspondence between the two-photon image data and the video frames in the behavioral video data.
[0106] Next, refer to Figure 10 to describe an electronic device according to an embodiment of the present disclosure. Figure 10 The following is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure.
[0107] As Figure 10 shown, the electronic device 1000 includes one or more input interfaces 1010, a processor 1020, and a communication interface 1030.
[0108] The plurality of input interfaces 1010 may be analog signal ports or digital signal ports, and the plurality of input interfaces 1010 are configured to receive a plurality of synchronization signals corresponding to a target experimental organism.
[0109] In some embodiments, the number of input interfaces 1010 may be 8. Among them, two input interfaces 1010 are connected to a miniaturized two-photon microscope, 4 input interfaces 1010 are connected to a behavioral camera, and two input interfaces 1010 are connected to a behavior device configured by a user.
[0110] The processor 1020 can be a single-chip microcomputer, a complex programmable logic device (CPLD), a field programmable gate array (FPGA), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and can control other components in the electronic device 1000 to perform desired functions.
[0111] The communication interface 1030 can be a serial communication interface, a universal serial bus (USB), an Ethernet, or other communication interfaces. Based on the communication interface 1030, data transmission between the electronic device 1000 and an external computer can be achieved.
[0112] Of course, for simplicity, Figure 10 only some of the components in the electronic device 1000 related to the present disclosure are shown in []. In addition, according to specific application scenarios, the electronic device 1000 may further include any other appropriate components.
[0113] In addition to the above methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions that, when run on a processor, cause the processor to execute the steps in the data synchronization method according to various embodiments of the present disclosure described above in this specification.
[0114] The computer program product can be written in any combination of one or more programming languages to write program code for performing the operations of the embodiments of the present disclosure. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0115] Furthermore, an embodiment of the present disclosure may also be a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are run on a processor, cause the processor to execute the steps in the data synchronization method according to various embodiments of the present disclosure described above in this specification.
[0116] The computer-readable storage medium may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0117] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present disclosure. In addition, the above-mentioned specific details are only for illustrative and easy-to-understand purposes and are not limitations. The above details do not limit the present disclosure to necessarily adopt the above specific details for implementation.
[0118] The block diagrams of the devices, apparatuses, equipment, and systems involved in the present disclosure are only illustrative examples and do not intend to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open-ended words, meaning "including but not limited to", and can be used interchangeably with each other. The word "or" and "and" used herein refer to the word "and / or" and can be used interchangeably with each other, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to" and can be used interchangeably with each other.
[0119] It should also be noted that in the devices, equipment, and methods of the present disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present disclosure.
[0120] The above description of the disclosed aspects enables any person skilled in the art to make or use the present disclosure. Various modifications to these aspects are very obvious to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.
[0121] The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the present disclosure to the form disclosed herein. Although several example aspects and embodiments have been discussed above, those skilled in the art will recognize some variations, modifications, alterations, additions, and subcombinations thereof.
Claims
1. A data synchronization method, characterized in that: Applied to a first device, the first device is connected to a second device by signal, and the method includes: Acquire a current time, and calibrate a clock of the first device based on the current time; Acquire multiple synchronization signals corresponding to the target experimental organism, wherein the multiple synchronization signals correspond to multiple data to be synchronized, wherein the multiple synchronization signals include an acquisition start signal of a miniaturized two-photon microscope and an exposure signal of a behavioral camera, and the multiple data to be synchronized are used to characterize the vital signs of the target experimental organism, and the multiple data to be synchronized include two-photon image data of the target experimental organism and behavioral video data of the target experimental organism; Determine, based on the clock of the first device, time identification information corresponding to each of the plurality of synchronization signals; The time identification information corresponding to each of the multiple synchronization signals is output to the second device, so that the second device performs synchronization alignment between the multiple data to be synchronized based on the time identification information corresponding to each of the multiple synchronization signals.
2. The method according to claim 1, characterized in that The determining, based on the clock of the first device, time identification information corresponding to each of the plurality of synchronization signals includes: Respectively determining a plurality of occurrence identifiers of the plurality of synchronization signals, wherein the occurrence identifier corresponds to one acquisition of the miniaturized two-photon microscope or one exposure of the behavioral camera; Based on the clock of the first device, the generation time of each of the multiple occurrence identifiers is determined respectively, and the generation time is used as the time identifier information corresponding to the occurrence identifier.
3. The method according to claim 2, characterized in that The synchronization signal is a TTL level signal, and the respectively determining multiple occurrence identifiers of the multiple synchronization signals includes: When the rising edge of the synchronization signal arrives, the currently executing program is interrupted; An occurrence identifier corresponding to the rising edge is generated.
4. The method according to any one of claims 1 to 3, characterized in that: The step of outputting the time identification information corresponding to each of the plurality of synchronization signals to the second device includes: Determining a reporting period of the time identification information based on the memory of the first device and the acquisition frequency of the miniaturized two-photon microscope and / or the exposure frequency of the behavioral camera; The time identification information within a cache reporting period and the occurrence identification corresponding to the time identification information; The time identification information and the occurrence identification cached in each reporting period are packaged and output to the second device respectively.
5. A data synchronization method, characterized in that: Applied to a second device, the second device being signal-connected to a first device, the method comprising: Acquire time identification information corresponding to each of a plurality of synchronization signals corresponding to the target experimental organism sent by the first device, wherein the plurality of synchronization signals correspond to a plurality of data to be synchronized, wherein the plurality of synchronization signals include an acquisition start signal of a miniaturized two-photon microscope and an exposure signal of a behavioral camera, and the plurality of data to be synchronized are used to characterize the vital signs of the target experimental organism, and the plurality of data to be synchronized include two-photon image data of the target experimental organism and behavioral video data of the target experimental organism; Acquire the multiple data to be synchronized corresponding to the multiple synchronization signals; Based on the time identification information corresponding to each of the multiple synchronization signals, synchronization alignment is performed between the multiple data to be synchronized.
6. The method according to claim 5, characterized in that The step of performing synchronization alignment among the plurality of data to be synchronized based on the time identification information corresponding to each of the plurality of synchronization signals includes: Based on the time identification information corresponding to each of the multiple synchronization signals, respectively determine the acquisition time of the two-photon image data and the exposure time of each video frame in the behavioral video data; Arranging the two-photon image data in chronological order based on the acquisition time of the two-photon image data; The two-photon image data is associated with the temporally closest video frame to determine a correspondence between the two-photon image data and the video frame in the behavioral video data.
7. A data synchronization device, characterized in that: Applied to a first device, the first device is connected to a second device by signal, and the apparatus comprises: A first acquisition module, configured to acquire a current time and calibrate a clock of the first device based on the current time; A second acquisition module is used to acquire a plurality of synchronization signals corresponding to the target experimental organism, wherein the plurality of synchronization signals correspond to a plurality of data to be synchronized, wherein the plurality of synchronization signals include an acquisition start signal of a miniaturized two-photon microscope and an exposure signal of a behavioral camera, and the plurality of data to be synchronized are used to characterize the vital signs of the target experimental organism, and the plurality of data to be synchronized include two-photon image data of the target experimental organism and behavioral video data of the target experimental organism; A determination module, configured to determine time identification information corresponding to each of the plurality of synchronization signals based on a clock of the first device; The output module is used to output the time identification information corresponding to each of the multiple synchronization signals to the second device, so that the second device can perform synchronization alignment between the multiple data to be synchronized based on the time identification information corresponding to each of the multiple synchronization signals.
8. A data synchronization device, characterized in that: Applied to a second device, the second device is connected to a first device by signal, and the apparatus comprises: A first acquisition module is used to acquire time identification information corresponding to each of a plurality of synchronization signals corresponding to the target experimental organism sent by the first device, wherein the plurality of synchronization signals correspond to a plurality of data to be synchronized, wherein the plurality of synchronization signals include an acquisition start signal of a miniaturized two-photon microscope and an exposure signal of a behavioral camera, and the plurality of data to be synchronized are used to characterize the vital signs of the target experimental organism, and the plurality of data to be synchronized include two-photon image data of the target experimental organism and behavioral video data of the target experimental organism; A second acquisition module, used for acquiring the multiple data to be synchronized corresponding to the multiple synchronization signals; The registration module is used to perform synchronization registration among the multiple data to be synchronized based on the time identification information corresponding to each of the multiple synchronization signals.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data synchronization method according to any one of claims 1 to 6 is implemented.
10. An electronic device, characterized in that: include: Multiple input interfaces for accessing multiple synchronization signals; A processor, configured to implement the data synchronization method according to any one of claims 1 to 4; Communication interface, used for data transmission with external computer.