Method for synchronizing video signal with bioelectric signal and video bioelectric acquisition system
By dividing the initial synchronization signal into two channels, and using the synchronization characteristics of the audio and video signals to align the bioelectric signal and video signals, the problem of signal deviation in the existing technology is solved, and high-precision bioelectric signal is achieved Synchronize with the video signal.
Patent Information
- Application Number
- CN202010517471.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-09
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2040-06-09
AI Technical Summary
Existing bioelectric and video signals are difficult to achieve accurate UTC synchronization in multi-clock systems and non-networked systems, resulting in signal deviations, especially when the acquisition time is extended.
By dividing the same initial synchronization signal into two channels: audio synchronization signal and bioelectrical synchronization signal, the time synchronization relationship between the audio signal and the video signal is used to align the bioelectrical signal with the audio signal, and further align it based on the synchronization characteristics of the audio signal and the video signal. Bioelectric and video signals.
The synchronization accuracy of bioelectric signals and video signals is improved, the analysis difficulty is reduced, the signal stability is enhanced, and the bioelectric signals at the 1ms level are synchronized with audio signals, and the video signals synchronization accuracy is achieved at the 8.3ms level.
Smart Images

Figure CN111631703B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of bioelectric technology, and in particular to a method for realizing synchronization between a video signal and a bioelectric signal, a method for synchronously acquiring a video signal and a bioelectric signal, and a video bioelectric acquisition system. Background Art
[0002] In the process of collecting bioelectric signals, it is often necessary to record videos at the same time. In the process of analyzing bioelectric signals, it is necessary to observe the synchronously collected videos and analyze the subject's status. For example, you can collect videos that record the patient's gait and the doctor's instructions while collecting electromyographic signals.
[0003] Many existing technologies for synchronizing bioelectric signals and video signals are achieved through coordinated universal time (UTC). However, in multi-clock systems and non-networked systems, it is impossible to obtain accurate UTC, which causes the asynchrony between bioelectric signals and video signals, usually with a deviation of several seconds. As the acquisition time increases, the deviation may be even greater. Summary of the invention
[0004] In order to at least partially solve the problems existing in the prior art, a method for realizing synchronization between a video signal and a bioelectric signal, a method for synchronously acquiring a video signal and a bioelectric signal, and a video bioelectric acquisition system are provided.
[0005] According to one aspect of the present invention, there is provided a method for achieving synchronization between a video signal and a bioelectric signal, comprising the following steps: dividing the same initial synchronization signal into two paths to obtain an audio synchronization signal and a bioelectric synchronization signal; outputting the audio synchronization signal together with the audio signal in the form of sound, wherein the audio signal and the video signal have a time synchronization relationship; acquiring a mixed audio signal comprising the audio synchronization signal and the audio signal; inputting the bioelectric synchronization signal together with the bioelectric signal into a bioelectric acquisition device; acquiring a mixed bioelectric signal comprising the bioelectric synchronization signal and the bioelectric signal; aligning the bioelectric signal with the audio signal based on the timestamp of the bioelectric synchronization signal in the mixed bioelectric signal and the timestamp of the audio synchronization signal in the mixed audio signal; and aligning the bioelectric signal with the video signal based on the time synchronization relationship between the audio signal and the video signal and the alignment result of the bioelectric signal and the audio signal.
[0006] Exemplarily, the initial synchronization signal is a switch signal or a continuous pulse signal.
[0007] Exemplarily, the video signal and the bioelectric signal each contain a time stamp, and the time stamps of the video signal and the bioelectric signal are calibrated using the same signal source.
[0008] Exemplarily, the frequency of the audio synchronization signal falls within the ultrasonic frequency range.
[0009] Exemplarily, aligning the bioelectric signal with the audio signal includes: calculating the timestamp difference between the timestamp of the bioelectric synchronization signal in the mixed bioelectric signal and the timestamp of the audio synchronization signal in the mixed audio signal, and correcting the difference between the timestamp of the bioelectric signal and the timestamp of the audio signal to the timestamp difference.
[0010] According to another aspect of the present invention, a method for synchronously collecting video signals and bioelectric signals is also provided, which is characterized in that it includes the following steps: step S310, collecting the bioelectric signal of the object under test, and adding a timestamp to the bioelectric signal; step S320, collecting the audio signal of the object under test, and adding a timestamp to the audio signal; step S330, collecting the video signal of the object under test, and adding a timestamp to the video signal, wherein the audio signal and the video signal have a time synchronization relationship; step S340, judging whether the bioelectric signal and the audio signal need to be aligned, if so, go to step S341, otherwise go to step S310; step S341, dividing the same initial synchronization signal into two paths to obtain an audio synchronization signal and a bioelectric synchronization signal; step Step S342, output the audio synchronization signal together with the audio signal in the form of sound; step S343, collect the mixed audio signal including the audio synchronization signal and the audio signal; step S344, input the biopotential synchronization signal together with the biopotential signal into the biopotential acquisition device; step S345, collect the mixed biopotential signal including the biopotential synchronization signal and the biopotential signal; step S346, align the biopotential signal with the audio signal based on the timestamp of the biopotential synchronization signal in the mixed biopotential signal and the timestamp of the audio synchronization signal in the mixed audio signal; step S347, align the biopotential signal with the video signal based on the time synchronization relationship between the audio signal and the video signal and the alignment result of the biopotential signal and the audio signal, and go to step S310.
[0011] Exemplarily, while adding a timestamp to an audio signal, the method further includes: adding a timestamp offset to the audio signal, wherein an initial value of the timestamp offset is 0; and / or, while adding a timestamp to a video signal, the method further includes: adding a timestamp offset to the video signal.
[0012] Exemplarily, aligning the bioelectric signal with the audio signal includes: calculating a timestamp difference between a timestamp of a bioelectric synchronization signal in the mixed bioelectric signal and a timestamp of an audio synchronization signal in the mixed audio signal, and correcting a timestamp offset according to the timestamp difference.
[0013] Exemplarily, determining whether the bioelectric signal and the audio signal need to be aligned includes: determining whether the bioelectric signal and the audio signal need to be aligned based on the interval time between the current moment and the last alignment, and performing the alignment if the interval time reaches a predetermined value, otherwise not performing the alignment.
[0014] According to another aspect of the present invention, a video bioelectric acquisition system is also provided, including a synchronization center device, an audio and video acquisition device, a bioelectric acquisition device and a control device, wherein the synchronization center device is used to divide the same initial synchronization signal into two paths to obtain an audio synchronization signal and a bioelectric synchronization signal, output the audio synchronization signal together with the audio signal in the form of sound, and input the bioelectric synchronization signal together with the bioelectric signal into the bioelectric acquisition device; the audio and video acquisition device is connected to the synchronization center device, and is used to acquire a mixed audio signal including an audio synchronization signal and an audio signal, and to acquire a video signal, wherein the audio signal and the video signal have a time synchronization relationship; the bioelectric acquisition device is connected to the synchronization center device, and is used to acquire a mixed bioelectric signal including a bioelectric synchronization signal and a bioelectric signal; the control device is connected to the audio and video acquisition device and the bioelectric acquisition device, and is used to align the bioelectric signal with the audio signal based on the timestamp of the bioelectric synchronization signal in the mixed bioelectric signal and the timestamp of the audio synchronization signal in the mixed audio signal, and align the bioelectric signal with the video signal based on the time synchronization relationship between the audio signal and the video signal and the alignment result of the bioelectric signal and the audio signal.
[0015] Exemplarily, the frequency of the audio synchronization signal falls within the ultrasonic frequency range.
[0016] Exemplarily, the synchronization center equipment includes an audio playback device for outputting an audio synchronization signal together with an audio signal in the form of sound, the audio and video acquisition equipment includes an audio acquisition device for acquiring a mixed audio signal including an audio synchronization signal and an audio signal, the frequency of the audio synchronization signal falls within the frequency range audible to the human ear, and the distance between the audio acquisition device and the audio playback device is less than a preset distance.
[0017] The method for realizing synchronization between video signals and bioelectric signals, the method for synchronously acquiring video signals and bioelectric signals, and the video bioelectric acquisition system according to the embodiments of the present invention can effectively improve the synchronization accuracy of bioelectric signals and video signals.
[0018] A series of simplified concepts are introduced in the summary of the invention, which will be further described in detail in the detailed description. The summary of the invention does not mean to attempt to define the key features and essential technical features of the claimed technical solution, nor does it mean to attempt to determine the scope of protection of the claimed technical solution.
[0019] The advantages and features of the present invention are described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The following drawings of the present invention are used as part of the present invention for understanding the present invention. The drawings show the embodiments of the present invention and their descriptions, and are used to explain the principles of the present invention. In the drawings,
[0021] Figure 1 A schematic flow chart showing a method for realizing synchronization between a video signal and a bioelectric signal according to an embodiment of the present invention;
[0022] Figure 2 A schematic diagram showing a process of collecting and analyzing video signals and bioelectric signals according to an embodiment of the present invention;
[0023] Figure 3 A schematic flow chart showing a method for synchronously acquiring video signals and bioelectric signals according to an embodiment of the present invention; and
[0024] Figure 4 A schematic block diagram of a video bioelectric acquisition system according to an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0025] In the following description, a large amount of details are provided so that the present invention can be thoroughly understood. However, it will be appreciated by those skilled in the art that the following description only exemplarily illustrates a preferred embodiment of the present invention, and the present invention can be implemented without one or more such details. In addition, in order to avoid confusion with the present invention, some technical features well known in the art are not described in detail.
[0026] In order to at least partially solve the above technical problems, the embodiments of the present invention provide a method for realizing synchronization between video signals and bioelectric signals, a method for synchronously acquiring video signals and bioelectric signals, and a video bioelectric acquisition system. The video signal and bioelectric signal synchronization technology according to the embodiments of the present invention can align the video signal with the bioelectric signal by means of an audio signal synchronized with the video signal, thereby aligning the video signal with the bioelectric signal. The video signal and bioelectric signal synchronization technology provided by the embodiments of the present invention can be applied to the research and application fields of any suitable type of bioelectric signals, including but not limited to electrocardiogram signals, electromyography signals, electroencephalogram signals, etc.
[0027] Figure 1 A schematic flow chart of a method 100 for realizing synchronization between a video signal and a bioelectric signal according to an embodiment of the present invention is shown. Figure 2 A schematic diagram showing the process of collecting and analyzing video signals and bioelectric signals according to an embodiment of the present invention is shown. Figure 1-2 The implementation of method 100 is described below. Figure 1 As shown, the method 100 includes the following steps S110, S120, S130, S140, S150, S160 and S170.
[0028] In step S110, the same initial synchronization signal is divided into two paths to obtain an audio synchronization signal and a bioelectric synchronization signal.
[0029] In step S120, the audio synchronization signal is output together with the audio signal in the form of sound, wherein the audio signal and the video signal have a time synchronization relationship.
[0030] In step S130, a mixed audio signal including an audio synchronization signal and an audio signal is collected.
[0031] In step S140, the bioelectric synchronization signal and the bioelectric signal are input into the bioelectric acquisition device.
[0032] In step S150, a mixed bioelectric signal including a bioelectric synchronization signal and a bioelectric signal is collected.
[0033] In step S160, the bioelectric signal is aligned with the audio signal based on the timestamp of the bioelectric synchronization signal in the mixed bioelectric signal and the timestamp of the audio synchronization signal in the mixed audio signal. Optionally, the bioelectric signal, the audio signal and the alignment result thereof can be stored.
[0034] In step S170, the bioelectric signal is aligned with the video signal based on the time synchronization relationship between the audio signal and the video signal and the alignment result of the bioelectric signal and the audio signal. Optionally, the bioelectric signal, the video signal and the alignment result of the two can be stored.
[0035] The initial synchronization signal can be sent by the synchronization center device ( Figure 2The initial synchronization signal is generated and outputted by a first audio signal (shown as a synchronization center). Optionally, the initial synchronization signal can be transmitted on a separate channel and divided into two signals, one of which is called an audio synchronization signal and the other is called a bioelectric synchronization signal. The two synchronization signals can be superimposed on the bioelectric signal (i.e., bioelectric data) and the audio signal (i.e., audio data), respectively. The audio synchronization signal is superimposed on the audio signal in the form of sound, that is, the audio synchronization signal is played while the original audio signal is played. To this end, the audio synchronization signal can be converted into a signal format consistent with the audio signal so that it can be played in an audio playback device. It should be noted that the audio signal and the audio synchronization signal can be played by the same audio playback device or different audio playback devices, and the audio playback device can be, for example, a speaker. The audio signal and the audio synchronization signal in the form of sound output by the audio playback device can be collected by the same audio acquisition device, which collects a mixed audio signal containing an audio synchronization signal and an audio signal, and the audio acquisition device can be, for example, a pickup. The bioelectric synchronization signal can be input into the bioelectric acquisition device together with the bioelectric signal through a parallel or serial transmission mode, and then superimposed in the bioelectric acquisition device. To this end, the bioelectric synchronization signal can be converted into a signal format consistent with the bioelectric signal so that it can be collected by the bioelectric acquisition device. The bioelectric acquisition device collects a mixed bioelectric signal of the bioelectric synchronization signal and the bioelectric signal. The bioelectric acquisition device can be a bioelectric amplifier in a bioelectric acquisition system. In one example, the bioelectric synchronization signal can be generated by an external trigger source in a bioelectric acquisition system for collecting bioelectric signals. In another example, the bioelectric synchronization signal can be generated by other additional devices independent of the bioelectric acquisition system.
[0036] Those skilled in the art will know that after the mixed audio signal is collected, it can be processed to identify the position and waveform of the audio synchronization signal and the audio signal. Similarly, after the mixed bioelectric signal is collected, it can be processed to identify the position and waveform of the bioelectric synchronization signal and the bioelectric signal.
[0037] The video signal and audio signal can be in the same acquisition system. For example, under the default synchronous acquisition of the camera and the microphone, the two have time synchronization characteristics. Therefore, during signal synchronization analysis, as long as the audio synchronization signal on the audio signal and the biopotential synchronization signal on the biopotential signal are aligned, the video signal can be further aligned with the biopotential signal.
[0038] If the bioelectric signal is directly synchronized with the video signal, the collected video image needs to be processed and analyzed, and the video image processing needs to rely on certain algorithms, and the video is easily disturbed by passers-by, doctors, and other patients, resulting in reduced stability. Compared with the two-dimensional video signal, the present invention processes the one-dimensional audio signal to indirectly synchronize the bioelectric signal and the video signal, which can reduce the difficulty of analysis, and the sound is less likely to be blocked by obstacles, which can improve the stability of the signal.
[0039] By using a method of marking the audio signal and the bioelectric signal with a synchronization signal, the synchronization accuracy of the bioelectric signal and the audio signal can reach 1ms. In addition, based on the synchronization characteristics of the audio and video signals, the bioelectric signal can be further synchronized with the video signal. This synchronization method can achieve (for a 60Hz video signal) a synchronization accuracy of 8.3ms or even higher (this accuracy is limited by the video signal refresh rate). In short, different from the existing method of synchronizing the bioelectric signal and the video signal through UTC synchronization, the method for synchronizing the video signal and the bioelectric signal according to an embodiment of the present invention can effectively improve the synchronization accuracy of the bioelectric signal and the video signal.
[0040] According to an embodiment of the present invention, the initial synchronization signal may be a switch signal or a continuous pulse signal. Those skilled in the art may understand the implementation method of the initial synchronization signal being a switch signal or a continuous pulse signal, which will not be elaborated herein. Since the waveforms of audio signals and bioelectric signals are usually complex, superimposing a simple switch signal or a continuous pulse facilitates identification of the location of the synchronization signal from the mixed signal, so that synchronization can be achieved simply and quickly.
[0041] Exemplarily, the video signal and the bioelectric signal each contain a timestamp, and the timestamps of the video signal and the bioelectric signal are calibrated using the same signal source. Although the relationship between the timestamp of the video signal (mainly the start timestamp) and the timestamp of the bioelectric signal (mainly the start timestamp) is not limited in the embodiment of the present invention, those skilled in the art can understand that the timestamp of the bioelectric signal and the timestamp in the video signal are preferably precisely synchronized, and by using the same signal source to calibrate the timestamps of the video signal and the bioelectric signal, the synchronization of the video signal and the bioelectric signal can be further improved.
[0042] According to an embodiment of the present invention, the frequency of the audio synchronization signal falls within the ultrasonic frequency range, that is, the frequency of the audio synchronization signal is higher than 20KHz. The audio synchronization signal is played in the form of ultrasonic waves, so that the purpose of synchronization can be achieved without affecting the object (subject) being measured. In this case, a mixed audio signal of the audio synchronization signal and the audio signal can be collected at a frequency higher than the frequency required for collecting sounds that can be heard by the human ear.
[0043] Exemplarily, dividing the same synchronization signal into two paths may include: dividing the same synchronization signal into two paths at a fixed time. After each execution of step S110, steps S120-S170 are subsequently executed. That is, steps S110-S170 can be executed at a fixed time, and the audio signal, the video signal and the bioelectric signal can be aligned, i.e., calibrated, at a fixed time by the fixed execution, so that the synchronization of these signals can be ensured for a long time.
[0044] According to an embodiment of the present invention, aligning the bioelectric signal with the audio signal may include: calculating the timestamp difference between the timestamp of the bioelectric synchronization signal in the mixed bioelectric signal and the timestamp of the audio synchronization signal in the mixed audio signal, and correcting the difference between the timestamp of the bioelectric signal and the timestamp of the audio signal to the timestamp difference. Those skilled in the art can understand the implementation method of aligning the bioelectric signal with the audio signal based on the timestamp difference between the timestamp of the bioelectric synchronization signal and the timestamp of the audio synchronization signal, which is not described in detail herein.
[0045] According to another aspect of the present invention, a method for synchronously acquiring video signals and bioelectric signals is provided. Figure 3 FIG. 3 is a schematic flow chart of a method 300 for synchronously collecting video signals and bioelectric signals according to an embodiment of the present invention. Figure 3 As shown, method 300 includes the following steps S310, S320, S330, S340, S341, S342, S343, S344, S345, S346 and S347.
[0046] Step S310, collecting bioelectric signals of the object under test, and adding a timestamp to the bioelectric signals. Optionally, in step S310, the collected bioelectric signals may be stored.
[0047] Step S320, collect the audio signal of the object under test, and add a time stamp to the audio signal. Optionally, in step S320, the collected audio signal can be stored.
[0048] Step S330, collect the video signal of the object to be measured, add a time stamp to the video signal, wherein the audio signal and the video signal have a time synchronization relationship. Optionally, in step S330, the collected video signal can be stored.
[0049] Step S340, determining whether the bioelectric signal and the audio signal need to be aligned, if so, proceeding to step S341, otherwise proceeding to step S310.
[0050] Step S341, dividing the same initial synchronization signal into two paths to obtain an audio synchronization signal and a bioelectric synchronization signal.
[0051] Step S342: output the audio synchronization signal together with the audio signal in the form of sound.
[0052] Step S343: collecting a mixed audio signal including an audio synchronization signal and an audio signal.
[0053] Step S344, inputting the biopotential synchronization signal and the biopotential signal together into the biopotential collection device.
[0054] Step S345, collecting a mixed bioelectric signal including a bioelectric synchronization signal and a bioelectric signal;
[0055] Step S346: aligning the bioelectric signal with the audio signal based on the timestamp of the bioelectric synchronization signal in the mixed bioelectric signal and the timestamp of the audio synchronization signal in the mixed audio signal.
[0056] Step S347, aligning the bioelectric signal with the video signal based on the time synchronization relationship between the audio signal and the video signal and the alignment result of the bioelectric signal with the audio signal, and then go to step S310.
[0057] The process of synchronously collecting video signals and bioelectric signals is as follows:
[0058] 1. Bioelectric signal collection starts and the timestamp is recorded;
[0059] 2. Audio signal and video signal acquisition begins, and each records a timestamp;
[0060] 3. At any time, for example, every certain period of time, determine whether the bioelectric signal and the audio signal need to be aligned. If so, start the following operations; otherwise, continue to collect the bioelectric signal, audio signal and video signal;
[0061] 4. The synchronization center device outputs a synchronization signal and simultaneously superimposes it on the bioelectric signal and the audio signal to mark the bioelectric signal and the audio signal respectively, and it is considered that the bioelectric signal and the audio signal at this moment are collected at the same time;
[0062] 5. After the acquisition device has respectively acquired the bioelectric signal and the audio signal, it reads the synchronization mark on the signal and uses it as a reference to synchronize the signal, so as to achieve the purpose of synchronizing the bioelectric signal and the audio signal;
[0063] 6. Since the video signal and the audio signal have time synchronization characteristics, the bioelectric signal can be further synchronized with the video signal.
[0064] Through the above method, the synchronization among the video signal, audio signal and bioelectric signal can be ensured during the process of collecting the three.
[0065] Please note that Figure 3 and Figure 1 The execution order of each step of the method shown is only an example and not a limitation. The present invention is not limited to Figure 3 and Figure 1 For example, step S140 is after S130, but can be executed before or simultaneously with either S110 or S120; for another example, steps S310-S330 can be executed in any time relationship; and so on.
[0066] According to an embodiment of the present invention, while adding a timestamp to an audio signal, method 300 may further include: adding a timestamp offset to the audio signal, wherein an initial value of the timestamp offset is 0; and / or, while adding a timestamp to a video signal, method 300 may further include: adding a timestamp offset to the video signal.
[0067] The timestamp offset represents the offset of the timestamp of the audio signal or video signal relative to the timestamp of the bioelectric signal, that is, the difference between the two. By recording the timestamp offset, the synchronization of the audio signal, video signal and bioelectric signal can be recorded. Before the initial alignment, the timestamp offset can be set to 0.
[0068] According to an embodiment of the present invention, aligning the bioelectric signal with the audio signal may include: calculating the timestamp difference between the timestamp of the bioelectric synchronization signal in the mixed bioelectric signal and the timestamp of the audio synchronization signal in the mixed audio signal, and correcting the timestamp offset according to the timestamp difference.
[0069] According to an embodiment of the present invention, determining whether the bioelectric signal and the audio signal need to be aligned may include: determining whether the bioelectric signal and the audio signal need to be aligned based on the interval time between the current moment and the last alignment, performing alignment if the interval time reaches a predetermined value, otherwise not performing alignment.
[0070] The predetermined value may be any suitable value, and the present invention is not limited thereto. Through this embodiment, alignment may be performed at regular intervals, thereby ensuring that the audio signal, the video signal and the bioelectric signal remain synchronized for a relatively long period of time.
[0071] According to another aspect of the present invention, a video bioelectric acquisition system is provided. Figure 4 FIG. 4 is a schematic block diagram of a video bioelectric acquisition system 400 according to an embodiment of the present invention. Figure 4As shown, the video bioelectricity acquisition system 400 includes a synchronization center device 410, an audio and video acquisition device 420, a bioelectricity acquisition device 430 and a control device 440.
[0072] The synchronization center device 410 is used to split the same initial synchronization signal into two paths to obtain an audio synchronization signal and a biopotential synchronization signal, output the audio synchronization signal together with the audio signal in the form of sound, and input the biopotential synchronization signal together with the biopotential signal into the biopotential acquisition device.
[0073] The audio and video acquisition device 420 is connected to the synchronization center device 410 and is used to acquire a mixed audio signal including an audio synchronization signal and an audio signal, and to acquire a video signal, wherein the audio signal and the video signal have a time synchronization relationship.
[0074] The bioelectric acquisition device 430 is connected to the synchronization center device 410 and is used to acquire a mixed bioelectric signal including a bioelectric synchronization signal and a bioelectric signal.
[0075] The control device 440 is connected to the audio and video acquisition device 420 and the bioelectric acquisition device 430, and is used to align the bioelectric signal with the audio signal based on the timestamp of the bioelectric synchronization signal in the mixed bioelectric signal and the timestamp of the audio synchronization signal in the mixed audio signal, and to align the bioelectric signal with the video signal based on the time synchronization relationship between the audio signal and the video signal and the alignment result of the bioelectric signal and the audio signal.
[0076] The synchronization center device 410 can be used only to output audio synchronization signals and bioelectric synchronization signals, the audio signal can be output by other audio playback devices independent of the synchronization center device 410, and the bioelectric synchronization signal can be obtained by other bioelectric sensors independent of the synchronization center device 410 and input into the bioelectric acquisition device. Optionally, the synchronization center device 410 can also be used to output audio signals and / or bioelectric signals, which can be achieved by integrating an audio playback device for outputting audio signals and a bioelectric sensor for outputting bioelectric signals in the synchronization center device 410.
[0077] One or both of the synchronization center device 410 and the control device 440 can be implemented by any device with data processing capability and / or instruction execution capability, including but not limited to electronic devices such as personal computers and servers. In addition, one or both of the synchronization center device 410 and the control device 440 can also be a central processing unit (CPU), a microcontroller (MCU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable logic array (FPGA) or other forms of processing units with data processing capability and / or instruction execution capability. Optionally, the synchronization center device 410 and the control device 440 can be integrated together.
[0078] The audio and video acquisition device 420 may include an audio acquisition device for acquiring a mixed audio signal including an audio synchronization signal and an audio signal, and a video acquisition device for acquiring a video signal. The audio acquisition device may be, for example, a microphone, and the video acquisition device may be, for example, a camera.
[0079] According to an embodiment of the present invention, the frequency of the audio synchronization signal falls within the ultrasonic frequency range.
[0080] According to an embodiment of the present invention, the synchronization center device 410 may include an audio playback device for outputting an audio synchronization signal together with an audio signal in the form of sound, and the audio and video acquisition device 420 may include an audio acquisition device for acquiring a mixed audio signal including an audio synchronization signal and an audio signal, the frequency of the audio synchronization signal falls within the frequency range audible to the human ear, and the distance between the audio acquisition device and the audio playback device is less than a preset distance.
[0081] The preset distance can be any suitable distance, and the present invention is not limited thereto. The preset range can be set as small as possible, that is, the audio acquisition device and the audio playback device are arranged as close as possible. Those skilled in the art can understand that the frequency range that the human ear can hear is in the range of 20Hz to 20KHz. In addition to ultrasonic waves, the audio synchronization signal can also be played in the form of sound that the human ear can hear. In this case, the audio acquisition device and the audio playback device can be arranged as close as possible to reduce the impact on the object being measured.
[0082] Although the transmission medium of the video signal, audio signal and bioelectric signal is not limited in this embodiment, those skilled in the art can understand that the transmission of the video signal, audio signal and bioelectric signal can be selected by any applicable signal transmission method in the prior art. For example, the audio and video acquisition device 420 can transmit the mixed audio signal and video signal to the control device 440 through wired or wireless communication, and the bioelectric acquisition device 430 can transmit the mixed bioelectric signal to the control device 440 through wired or wireless communication.
[0083] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0084] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.
[0085] Similarly, it should be understood that in order to streamline the present invention and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present invention should not be interpreted as reflecting the following intention: the claimed invention requires more features than the features explicitly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present invention.
[0086] It will be understood by those skilled in the art that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.
[0087] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some modules in the video bioelectric acquisition system according to an embodiment of the present invention. The present invention may also be implemented as a device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.
[0088] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.
[0089] The above is only a specific embodiment of the present invention or an explanation of a specific embodiment. The protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A method for synchronizing a video signal with a bioelectric signal, It is characterized in that The following steps are involved: Splitting the same initial synchronization signal into two paths to obtain an audio synchronization signal and a bioelectric synchronization signal; Outputting the audio synchronization signal together with the audio signal in the form of sound, wherein the audio signal and the video signal have a time synchronization relationship; Collecting a mixed audio signal including the audio synchronization signal and the audio signal; Inputting the bioelectric synchronization signal together with the bioelectric signal into a bioelectric acquisition device; Collecting a mixed bioelectric signal including the bioelectric synchronization signal and the bioelectric signal; aligning the bioelectric signal with the audio signal based on a timestamp of the bioelectric synchronization signal in the mixed bioelectric signal and a timestamp of the audio synchronization signal in the mixed audio signal; Based on the time synchronization relationship between the audio signal and the video signal and the alignment result between the bioelectric signal and the audio signal, the bioelectric signal is aligned with the video signal.
2. The method according to claim 1, It is characterized in that The initial synchronization signal is a switch signal or a continuous pulse signal.
3. The method according to claim 1, It is characterized in that The video signal and the bioelectric signal each contain a time stamp, and the time stamps of the video signal and the bioelectric signal are calibrated using the same signal source.
4. The method according to claim 1, It is characterized in that The frequency of the audio synchronization signal falls within the ultrasonic frequency range.
5. The method according to claim 1, It is characterized in that The aligning the bioelectric signal with the audio signal comprises: A timestamp difference between a timestamp of the bioelectric synchronization signal in the mixed bioelectric signal and a timestamp of the audio synchronization signal in the mixed audio signal is calculated, and a difference between a timestamp of the bioelectric signal and a timestamp of the audio signal is corrected to the timestamp difference.
6. A method for synchronously collecting video signals and bioelectric signals, It is characterized in that The following steps are involved: Step S310, collecting bioelectric signals of the object under test, and adding a timestamp to the bioelectric signals; Step S320, collecting the audio signal of the object under test, and adding a timestamp to the audio signal; Step S330, collecting a video signal of the object under test, and adding a timestamp to the video signal, wherein the audio signal and the video signal have a time synchronization relationship; Step S340, determining whether the bioelectric signal and the audio signal need to be aligned, if so, proceeding to step S341, otherwise proceeding to step S310; Step S341, dividing the same initial synchronization signal into two paths to obtain an audio synchronization signal and a bioelectric synchronization signal; Step S342, outputting the audio synchronization signal together with the audio signal in the form of sound; Step S343, collecting a mixed audio signal including the audio synchronization signal and the audio signal; Step S344, inputting the bioelectric synchronization signal together with the bioelectric signal into a bioelectric acquisition device; Step S345, collecting a mixed bioelectric signal including the bioelectric synchronization signal and the bioelectric signal; Step S346, aligning the bioelectric signal with the audio signal based on the timestamp of the bioelectric synchronization signal in the mixed bioelectric signal and the timestamp of the audio synchronization signal in the mixed audio signal; Step S347, aligning the bioelectric signal with the video signal based on the time synchronization relationship between the audio signal and the video signal and the alignment result of the bioelectric signal with the audio signal, and going to step S310.
7. The method according to claim 6, It is characterized in that While adding the timestamp to the audio signal, the method further includes: adding a timestamp offset to the audio signal, wherein an initial value of the timestamp offset is 0; and / or, While adding the time stamp to the video signal, the method further includes: adding a time stamp offset to the video signal.
8. The method according to claim 7, It is characterized in that The aligning the bioelectric signal with the audio signal comprises: A timestamp difference between a timestamp of the bioelectric synchronization signal in the mixed bioelectric signal and a timestamp of the audio synchronization signal in the mixed audio signal is calculated, and the timestamp offset is corrected according to the timestamp difference.
9. The method according to claim 6, It is characterized in that The determining whether the bioelectric signal and the audio signal need to be aligned includes: Whether the bioelectric signal and the audio signal need to be aligned is determined according to the interval time between the current moment and the last alignment. If the interval time reaches a predetermined value, the alignment is performed; otherwise, the alignment is not performed.
10. A video bioelectric acquisition system, comprising a synchronization center device, an audio and video acquisition device, a bioelectric acquisition device and a control device, in, The synchronization center device is used to divide the same initial synchronization signal into two paths to obtain an audio synchronization signal and a bioelectric synchronization signal, output the audio synchronization signal together with the audio signal in the form of sound, and input the bioelectric synchronization signal together with the bioelectric signal into the bioelectric acquisition device; The audio and video acquisition device is connected to the synchronization center device, and is used to acquire a mixed audio signal including the audio synchronization signal and the audio signal, and to acquire a video signal, wherein the audio signal and the video signal have a time synchronization relationship; The bioelectric acquisition device is connected to the synchronization center device and is used to acquire a mixed bioelectric signal including the bioelectric synchronization signal and the bioelectric signal; The control device is connected to the audio and video acquisition device and the bioelectric acquisition device, and is used to align the bioelectric signal with the audio signal based on the timestamp of the bioelectric synchronization signal in the mixed bioelectric signal and the timestamp of the audio synchronization signal in the mixed audio signal, and to align the bioelectric signal with the video signal based on the time synchronization relationship between the audio signal and the video signal and the alignment result of the bioelectric signal and the audio signal.
11. The system according to claim 10, It is characterized in that The frequency of the audio synchronization signal falls within the ultrasonic frequency range.
12. The system according to claim 10, It is characterized in that The synchronization center device includes an audio playback device for outputting the audio synchronization signal together with the audio signal in the form of sound, and the audio and video acquisition device includes an audio acquisition device for acquiring a mixed audio signal including the audio synchronization signal and the audio signal. The frequency of the audio synchronization signal falls within the frequency range audible to the human ear, and the distance between the audio acquisition device and the audio playback device is less than a preset distance.
