Optical camera timestamp synchronization system and method

Abstract

The invention provides an optical camera timestamp synchronization system and method, and relates to the field of camera timestamp synchronization, and the method comprises the steps: arranging a synchronization panel in front of a camera through a support, and enabling an LED lamp matrix of the synchronization panel to face a lens of the camera; initializing a synchronization panel through a synchronization signal generator; obtaining independent video images collected by a preset number of cameras; reading LED matrix information of the synchronous panel from the independent video image; and performing timestamp alignment on each independent video image through the LED matrix information. The external LED matrix driven by hardware signals is used for synchronizing the timestamps of the cameras, so that the precision based on a hardware signal synchronization method is reserved to a certain extent, and the universality is also improved.

Description

Technical Field

[0001] The present application relates to the field of camera timestamp synchronization, and in particular to an optical camera timestamp synchronization system and method. Background Art

[0002] In today's technology application scenarios, it is increasingly common for multiple cameras to work together. As a key technology, camera timestamp synchronization can ensure that multiple cameras record data under a unified time reference.

[0003] Timestamp synchronization mainly includes: hardware synchronization, protocol synchronization, hybrid synchronization and vision-based synchronization.

[0004] Hardware synchronization mainly uses external devices or signals to achieve high-precision time synchronization, and is suitable for scenarios with extremely stringent requirements on time accuracy. The hardware triggering rule is to trigger all cameras to start collecting data at the same time through an external hardware trigger (such as FPGA). This method is simple and efficient, with high synchronization accuracy, but requires customized hardware and has poor flexibility. It is commonly used in scenarios such as industrial inspection and scientific research.

[0005] Protocol synchronization uses software protocols and algorithms to achieve timestamp alignment, but this requires the same type of camera device hardware and software development kit to support the same type of protocol.

[0006] The method that combines hardware triggering with software calibration first uses hardware triggering for preliminary synchronization, and then further calibrates the timestamp through software algorithms. This method has high accuracy, but the implementation process is relatively complex and is suitable for high-precision multi-camera systems.

[0007] The vision-based synchronization method aligns timestamps by analyzing video content, such as jointly captured moving objects or flash signals. Its advantage is that it does not require additional hardware and has high flexibility. However, the disadvantages of this method are also obvious. It depends on video content, has high computational complexity, and requires the same scene to appear in the common perspective of the cameras to be synchronized. Therefore, its application scenarios are greatly limited and it is only suitable for scenarios such as motion capture and multi-view video analysis.

[0008] In summary, hardware synchronization requires additional hardware to support signals or trigger functions; protocol synchronization requires specific hardware and software packages to support the corresponding protocols, which not only greatly increases the system cost, but also makes adaptation difficult and costly when facing different types of cameras. Although the vision-based synchronization method does not require additional hardware and has a certain degree of flexibility, it is overly dependent on video content, has a complex calculation process, and has strict requirements on application scenarios, such as the need for shared scenarios, which greatly limits its scope of application. Summary of the invention

[0009] The purpose of the present invention is to provide an optical camera timestamp synchronization system and method in order to solve the problem that the existing time step synchronization method is difficult to balance hardware cost and computational efficiency.

[0010] The above-mentioned purpose of the present application is achieved through the following technical solutions:

[0011] The system includes: a time stamp synchronization module, a bracket, a camera module, a synchronization signal generator and a computer module;

[0012] The timestamp synchronization module is connected to the camera module via a bracket;

[0013] The timestamp synchronization module is connected to a synchronization signal generator;

[0014] The computer module is connected to the time stamp synchronization module;

[0015] The camera module is used to collect at least one high-speed dynamic video data through each camera;

[0016] The timestamp synchronization module is used to add the same synchronization panel to each high-speed dynamic video data;

[0017] The bracket is used to fix each synchronization panel at the same position on each camera screen;

[0018] The synchronization signal generator is used to initialize the synchronization panel and control the synchronization display of the display lights of the synchronization panel;

[0019] The computer module is used for receiving the high-speed dynamic video data of each added synchronization panel, and synchronizing the time stamp of each high-speed dynamic video data based on the display light of the synchronization panel.

[0020] Optionally, if the camera is a rolling shutter camera, each synchronization panel of the timestamp synchronization module is arranged at the lower side of the camera screen.

[0021] Optionally, the display lights of the synchronization panel include: a low frequency coding area and a high frequency coding area;

[0022] The low-frequency coding area and the high-frequency coding area are composed of a preset number of LED lights to display coding information;

[0023] The LED lights in the low-frequency coding area use the traditional coding method of original code or CFE code to collect fixed time stamp coding information;

[0024] The high-frequency coding area uses a marquee coding method to collect accurate timestamp coding information.

[0025] A method for synchronizing timestamps of an optical camera, the method comprising:

[0026] S1: Set the synchronization panel in front of the camera through the bracket, and the LED light matrix of the synchronization panel faces the camera lens;

[0027] S2: Initialize the synchronization panel through the synchronization signal generator;

[0028] S3: Obtain independent video images captured by a preset number of cameras;

[0029] S4: Reading LED matrix information of the synchronization panel from the independent video image; aligning the timestamps of each independent video image through the LED matrix information.

[0030] Optionally, step S2 includes:

[0031] S21: connecting the synchronization signal generator to a preset number of synchronization panels through a wired connection;

[0032] S22: Initialize the synchronization panel using the synchronization signal generated by the synchronization signal generator, so that the LED matrix of the synchronization panel flashes synchronously;

[0033] S32: Disconnect the synchronization panel from the synchronization signal generator to complete the initialization of the synchronization panel.

[0034] Optionally, step S4 includes:

[0035] LED matrix information includes: low-frequency matrix information and high-frequency matrix information;

[0036] Determine the base time of each video segment of the independent video image through the low-frequency matrix information;

[0037] The timestamp of each frame image of each video segment is determined by combining the high-frequency matrix information with the basic time of each video segment;

[0038] The timestamps of independent video images are aligned using the timestamp of each frame of the image.

[0039] An electronic device includes a processor, a memory, a user interface and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes an optical camera timestamp synchronization method.

[0040] A computer-readable storage medium stores instructions. When the instructions are executed, an optical camera timestamp synchronization method is performed.

[0041] The beneficial effects of the technical solution provided by this application are:

[0042] Timestamp synchronization is achieved by identifying LED lights, which greatly reduces the complexity of calculation. A low-cost additional hardware device is designed, which can be flexibly attached to various optical cameras to enhance the versatility of the technical solution of this application. Since the additional hardware is bound to each camera one by one, each camera does not need to observe the shared scene at the same time when performing timestamp synchronization. The solution of relatively fixed synchronization panel and camera makes the present invention independent of video content and has low computational complexity and high real-time performance, breaking through the application scenario limitations of traditional visual synchronization methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present application will be further described below with reference to the accompanying drawings and embodiments, in which:

[0044] Figure 1 It is a schematic diagram of the overall structure in the embodiment of the present application;

[0045] Figure 2 is a synchronization panel position diagram in an embodiment of the present application;

[0046] Figure 3 is a schematic diagram of a synchronization panel in an embodiment of the present application;

[0047] Figure 4 It is a camera screen diagram in the embodiment of the present application;

[0048] Figure 5 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to have a clearer understanding of the technical features, purposes and effects of the present application, the specific implementation methods of the present application are now described in detail with reference to the accompanying drawings.

[0050] An embodiment of the present application provides a method for synchronizing timestamps of an optical camera.

[0051] Please refer to Figure 1 , Figure 1 : is a schematic diagram of the overall structure of an optical camera timestamp synchronization system in an embodiment of the present application, including:

[0052] A timestamp synchronization module, a bracket, a camera module, a synchronization signal generator, and a computer module;

[0053] The timestamp synchronization module is connected to the camera module via a bracket;

[0054] The timestamp synchronization module is connected to a synchronization signal generator;

[0055] The computer module is connected to the time stamp synchronization module;

[0056] The camera module is used to collect at least one high-speed dynamic video data through each camera;

[0057] The timestamp synchronization module is used to add the same synchronization panel to each high-speed dynamic video data;

[0058] The bracket is used to fix each synchronization panel at the same position on each camera screen;

[0059] The synchronization signal generator is used to initialize the synchronization panel and control the synchronization display of the display lights of the synchronization panel;

[0060] The computer module is used for receiving the high-speed dynamic video data of each added synchronization panel, and synchronizing the time stamp of each high-speed dynamic video data based on the display light of the synchronization panel.

[0061] If the camera is a rolling shutter camera, each synchronization panel of the timestamp synchronization module is arranged at the lower side of the camera screen.

[0062] Specifically, the commonality between the prior art and the present application is that both use LED matrices or QR codes to achieve timestamp synchronization. However, the differences are also quite obvious. The method of the present application continuously uses the LED matrix to add timestamps to the picture in the form of optical imaging throughout the video data acquisition process; in contrast, the prior art only uses the LED matrix or QR code in the preliminary preparation stage to align and correct the system time of each camera (the purpose of this operation is similar to NTP or PTP), and relies on the timestamp of the camera hardware in the acquisition stage.

[0063] Specifically, the existing technology has certain advantages, that is, under certain restrictions, its operation process is relatively simple. But the disadvantages should not be ignored. On the one hand, it requires the use of cameras of the same model or the same hardware standard, which undoubtedly greatly increases the difficulty and cost of hardware compatibility and modification. For example, different models of cameras have different shutter types (covering rolling shutters, mechanical, etc.), trigger modes, system delays, and even the definition of timestamps are different (such as when receiving soft / hard trigger signals, when starting exposure, when sending data, or when receiving data); on the other hand, in different scenarios, the exposure time of the camera will also change dynamically. It is difficult to accurately describe the exposure process by simply synchronizing the camera clock.

[0064] Specifically, in comparison, the technical solution of the present application has the advantages of wide applicability and can be adapted to different types of cameras; the existing system can be modified by simply adding a low-cost synchronization panel. In addition, the timestamp of the technical solution of the present application has a clear meaning, is closely related to the imaging process, and can accurately outline the exposure process.

[0065] As an embodiment, the hardware structure of the optical camera timestamp synchronization system is referenced Figure 1 , Figure 2 and Figure 3 . Figure 1 The figure is a schematic diagram of the overall structure of the optical camera timestamp synchronization system. It includes multiple optical cameras 3 to be synchronized, which are distributed in different positions and are used to collect video images. Each camera is equipped with a synchronization panel 1, and the synchronization panel 1 and the camera 3 are connected and fixed to each other through a special bracket 2, so that the synchronization panel appears in the camera field of view and keeps the relative position unchanged.

[0066] As an embodiment, the installation method of the synchronization panel 1 is not unique. The synchronization panel can be installed on the right side or the lower side of the camera field of view according to actual working conditions. Figure 2 As shown. For example, if a rolling shutter camera is used to capture high-speed dynamic images, the synchronization panel is installed on the lower side of the camera field of view to reduce the impact of imaging distortion on synchronization panel recognition. All synchronization panels are connected to a synchronization signal generator 4 through cables, and the synchronization signal generator generates a synchronization signal and initializes each synchronization panel. After the synchronization panel is initialized, the synchronization signal generator no longer needs to be connected to the synchronization panel through cables.

[0067] The display lights of the synchronization panel include: a low frequency coding area and a high frequency coding area;

[0068] The low-frequency coding area and the high-frequency coding area are composed of a preset number of LED lights to display coding information;

[0069] The LED lights in the low-frequency coding area use the traditional coding method of original code or CFE code to collect fixed time stamp coding information;

[0070] The high-frequency coding area uses a marquee coding method to collect accurate timestamp coding information.

[0071] As an example, Figure 3 Schematic diagram of synchronization panel 1. The synchronization panel consists of a substrate 102 and LED lamps 101 installed thereon. Multiple LED lamps form an LED matrix and occupy the main part of the synchronization panel to display coded information. The LED matrix is ​​divided into two areas: 1. Low-frequency coding area 103, which uses traditional coding methods such as original code or CFE code, is responsible for outputting relatively fixed basic coding information, laying the foundation for building timestamps; 2. High-frequency coding area 104, which uses a marquee coding strategy, can quickly change the coding form, carry more accurate and timely information, and improve the accuracy of timestamps.

[0072] As an embodiment, the LED matrix has flexible customization characteristics, and its shape can be personalized according to actual needs. Figure 3 As shown, without loss of generality, the figure shows a scheme in which the high-frequency coding area and the low-frequency coding area are arranged in two rows. It should be pointed out that if the LED matrix is ​​planned to be placed at the upper or lower part of the image, the synchronization panel can be modified as needed; for example, by connecting the high-frequency and low-frequency areas in a row, the height area occupied by the LED matrix in the image can be effectively reduced, thereby better adapting to different application scenarios.

[0073] A method for synchronizing timestamps of an optical camera, the method comprising:

[0074] S1: Set the synchronization panel in front of the camera through the bracket, and the LED light matrix of the synchronization panel faces the camera lens;

[0075] As an embodiment, the synchronization panel is properly placed by a bracket so that it remains relatively still with the camera, ensuring that the camera can clearly capture the synchronization panel image. Specifically, no matter how the camera moves, it is necessary to ensure that the position of the LED matrix in the imaging picture remains fixed, and at the same time, the position of the circular area of ​​each LED light in the matrix is ​​accurately recorded, and the imaging area of ​​each LED light in the image is stipulated to be no less than 10×10, in preparation for subsequent recognition and decoding work.

[0076] S2: Initialize the synchronization panel through the synchronization signal generator;

[0077] S3: Obtain independent video images captured by a preset number of cameras;

[0078] S4: Reading LED matrix information of the synchronization panel from the independent video image; aligning the timestamps of each independent video image through the LED matrix information.

[0079] As an embodiment, because the synchronization panel and the camera are relatively stationary, the positions of the light and dark areas on the synchronization panel in the image are fixed, and preliminary recognition can be completed with simple threshold processing. The recognition and decoding calculation amount is small and can meet real-time processing requirements.

[0080] 1. An external, hardware-driven synchronization panel and synchronization signal generator that has the accuracy of a hardware synchronization solution and can be flexibly bound to an existing camera system without complex modifications.

[0081] 2. The synchronization panel 1 and the camera 3 are connected by a special bracket so that the imaging area of ​​the synchronization panel in the camera remains unchanged. Figure 4The image captured by the camera is shown in the correct implementation. Under this condition, simple image cropping and single threshold processing can be used to complete the synchronous panel signal recognition, achieving the effect of low computational requirements and high real-time performance.

[0082] 3. Distinguish between low-frequency and high-frequency coding areas in the synchronization panel to further reduce the computational complexity of synchronization panel signal recognition and decoding.

[0083] 4. By counting the number of activated LED lights, the camera exposure time can be estimated, improving the accuracy and functionality of timestamp synchronization in many ways.

[0084] Step S2 includes:

[0085] S21: connecting the synchronization signal generator to a preset number of synchronization panels through a wired connection;

[0086] S22: Initialize the synchronization panel using the synchronization signal generated by the synchronization signal generator, so that the LED matrix of the synchronization panel flashes synchronously;

[0087] S32: Disconnect the synchronization panel from the synchronization signal generator to complete the initialization of the synchronization panel.

[0088] As an embodiment, the synchronization signal generator is connected to all synchronization panels through a wired connection to build a stable signal transmission channel to ensure smooth signal transmission. The synchronization panel is initialized using the synchronization signal to make the panel LED matrix flash synchronously so that the initial states of the system components are consistent.

[0089] Step S4 includes:

[0090] LED matrix information includes: low-frequency matrix information and high-frequency matrix information;

[0091] Determine the base time of each video segment of the independent video image through the low-frequency matrix information;

[0092] The timestamp of each frame image of each video segment is determined by combining the high-frequency matrix information with the basic time of each video segment;

[0093] The timestamps of independent video images are aligned using the timestamp of each frame of the image.

[0094] As an embodiment, the LED status of the low-frequency coding area is read and the basic time is obtained through decoding; then the LED status of the high-frequency coding area is read and decoded to obtain the precise time, thereby completing the timestamp synchronization; in addition, by counting the number of activated LED lights, the exposure time of the camera can also be estimated, thereby improving the accuracy and functionality of the timestamp synchronization from many aspects.

[0095] The present application also discloses an electronic device. Figure 5 , Figure 5 The electronic device 500 may include: at least one processor 501 , at least one network interface 504 , a user interface 503 , a memory 505 , and at least one communication bus 502 .

[0096] The communication bus 502 is used to realize the connection and communication between these components.

[0097] The user interface 503 may include a display screen, and the optional user interface 503 may also include a standard wired interface or a wireless interface.

[0098] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0099] The present application also discloses a computer-readable storage medium, which stores a plurality of instructions, and the instructions are suitable for a processor to load to execute the above-mentioned optical camera timestamp synchronization method.

[0100] The above are only exemplary embodiments of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure.

[0101] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. An optical camera timestamp synchronization system, characterized in that: The system includes: a time stamp synchronization module, a bracket, a camera module, a synchronization signal generator and a computer module; The timestamp synchronization module is connected to the camera module via a bracket; The timestamp synchronization module is connected to a synchronization signal generator; The computer module is connected to the time stamp synchronization module; The camera module is used to collect at least one high-speed dynamic video data through each camera; The timestamp synchronization module is used to add the same synchronization panel to each high-speed dynamic video data; The bracket is used to fix each synchronization panel at the same position on each camera screen; The synchronization signal generator is used to initialize the synchronization panel and control the synchronization display of the display lights of the synchronization panel; The computer module is used for receiving the high-speed dynamic video data of each added synchronization panel, and synchronizing the time stamp of each high-speed dynamic video data based on the display light of the synchronization panel.

2. An optical camera timestamp synchronization system as claimed in claim 1, characterized in that: If the camera is a rolling shutter camera, each synchronization panel of the timestamp synchronization module is arranged at the lower side of the camera screen.

3. The optical camera timestamp synchronization system according to claim 1, characterized in that: The display lights of the synchronization panel include: a low frequency coding area and a high frequency coding area; The low-frequency coding area and the high-frequency coding area are composed of a preset number of LED lights to display coding information; The LED lights in the low-frequency coding area use the traditional coding method of original code or CFE code to collect fixed time stamp coding information; The high-frequency coding area uses a marquee coding method to collect accurate timestamp coding information.

4. An optical camera timestamp synchronization method, implemented based on an optical camera timestamp synchronization system as claimed in any one of claims 1 to 3, characterized in that: The method comprises: S1: Set the synchronization panel in front of the camera through the bracket, and the LED light matrix of the synchronization panel faces the camera lens; S2: Initialize the synchronization panel through the synchronization signal generator; S3: Obtain independent video images captured by a preset number of cameras; S4: Reading LED matrix information of the synchronization panel from the independent video image; aligning the timestamps of each independent video image through the LED matrix information.

5. The optical camera timestamp synchronization method as claimed in claim 4, characterized in that: Step S2 includes: S21: connecting the synchronization signal generator to a preset number of synchronization panels through a wired connection; S22: Initialize the synchronization panel using the synchronization signal generated by the synchronization signal generator, so that the LED matrix of the synchronization panel flashes synchronously; S32: Disconnect the synchronization panel from the synchronization signal generator to complete the initialization of the synchronization panel.

6. The optical camera timestamp synchronization method as claimed in claim 4, characterized in that: Step S4 includes: LED matrix information includes: low-frequency matrix information and high-frequency matrix information; Determine the base time of each video segment of the independent video image through the low-frequency matrix information; The timestamp of each frame image of each video segment is determined by combining the high-frequency matrix information with the basic time of each video segment; The timestamps of independent video images are aligned using the timestamp of each frame of the image.

7. An electronic device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method as described in any one of claims 4-6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed by a computer, the method according to any one of claims 4 to 6 is executed.

Images