Binaural synchronous imaging system and method based on time consistency control
By implementing full-process time consistency control of binocular images through FPGA modules, the problems of insufficient trigger link consistency, asynchronous image processing, and poor imaging stability in low-light scenarios in existing technologies are solved. This improves the time consistency and measurement accuracy of binocular imaging systems and is suitable for low-light close-range target observation and active nighttime lighting scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-09
- Publication Date
- 2026-07-21
AI Technical Summary
Existing binocular imaging systems suffer from problems such as insufficient trigger link consistency, asynchronous image processing and output, lack of a unified time synchronization constraint mechanism, and poor imaging stability in low-light scenarios, which affect the time consistency and measurement accuracy of binocular imaging.
The system employs an FPGA-based synchronous triggering module, data receiving and parsing module, time synchronization marking module, image data processing module, exposure and illumination control module, image buffer control module, and synchronous output control module to achieve full-process time consistency control of binocular images. Through synchronous triggering signals, timestamp binding, and synchronous consistency judgment, it ensures the time consistency of image acquisition, buffering, and output, and performs joint adjustment of exposure and active illumination.
It achieves high-precision time consistency control of binocular images throughout the entire process of acquisition, buffering and output, improving the imaging stability and measurement accuracy of the system, and is suitable for low-light close-range target observation and active nighttime lighting scenarios.
Smart Images
Figure CN122438006A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of optical imaging and electronic information processing technology, specifically to a binocular synchronous imaging system and method based on time consistency control. The method of this invention is based on FPGA to realize synchronous acquisition of binocular images, time synchronization marking, synchronization buffering and synchronous output, and is suitable for application scenarios such as low-light close-range target observation, nighttime active illumination imaging and high time consistency binocular measurement. Background Technology
[0002] Binocular imaging systems acquire images of a target scene from two different perspectives and have wide applications in 3D reconstruction, target recognition, attitude measurement, and space payload applications. To ensure the measurement accuracy of a binocular system, the images acquired from the two perspectives must have high temporal consistency, thereby ensuring an accurate correspondence between the binocular images.
[0003] Existing binocular imaging systems typically employ shared trigger signals, software synchronization control, or multi-processor parallel processing to achieve synchronous binocular acquisition. However, these methods still have the following shortcomings in practical applications:
[0004] 1. Insufficient consistency of the trigger link;
[0005] Existing systems typically use a single trigger signal to drive multiple image sensors, which can easily lead to signal reflection and transmission delay differences. This results in deviations in the exposure start time of binocular images, reducing the consistency of binocular imaging time.
[0006] 2. Asynchronous processing and output of binocular images;
[0007] Differences in processing latency during the backend processing, caching, and output of binocular images can lead to inconsistent output times, affecting the matching accuracy and target measurement accuracy of binocular images.
[0008] 3. Lack of a unified time consistency constraint mechanism;
[0009] Existing systems typically do not establish a unified time reference for binocular images and lack time synchronization marking and consistency judgment mechanisms for the entire process of image acquisition, caching, and output, making it difficult to effectively verify and constrain the synchronization of binocular images.
[0010] 4. Insufficient imaging stability in low-light scenes;
[0011] In nighttime active lighting applications, existing systems typically use fixed exposure parameters or independently adjust exposure and light intensity, lacking a joint control mechanism for exposure and active lighting based on image brightness feedback. This can easily lead to problems such as image brightness fluctuations, loss of target details, and prolonged high-power operation of active lighting devices.
[0012] Therefore, it is necessary to provide a binocular synchronous imaging system and method that can achieve time consistency control throughout the entire process of binocular image acquisition, buffering and output, and has the ability to jointly and adaptively adjust exposure and active illumination. Summary of the Invention
[0013] The purpose of this invention is to provide a binocular synchronous imaging system and method based on time consistency control, in order to solve the problems of insufficient trigger link consistency, asynchronous image processing and output, lack of unified time synchronization constraint mechanism, and poor imaging stability in low-light scenes in existing binocular imaging systems. This invention achieves high-precision time consistency control of binocular images throughout the entire process of acquisition, buffering and output, and improves the imaging stability and engineering reliability of the system.
[0014] To achieve the above objectives, the present invention provides a binocular synchronous imaging system based on time consistency control. The system includes a binocular image sensor module for acquiring binocular image data; an FPGA module for realizing synchronous control, time consistency determination, and synchronous output of binocular image data; a DDR storage module for buffering the binocular image data output by the FPGA module; and an active illumination module.
[0015] The FPGA module includes a synchronization trigger module, a data receiving and parsing module, a time synchronization marker module, an image data processing module, an exposure and illumination control module, an image buffer control module, and a synchronization output control module.
[0016] The synchronous trigger control module is used to send a synchronous trigger signal to the binocular image sensor module through two physically consistent independent transmission paths, so that the exposure start time of the binocular image sensor is consistent.
[0017] The data receiving and parsing module receives image data output by the binocular image sensor and performs data recognition, time sequence recovery and data reconstruction; while recognizing valid image data, it generates corresponding timestamps through the time synchronization marking module and binds the timestamps to the corresponding image frames.
[0018] The image data processing module performs format conversion processing on the received image data with timestamps to generate image data for synchronous output;
[0019] The image caching control module writes the entire frame of the binocular image output by the image data processing module into the DDR storage module for caching;
[0020] The synchronous output control module determines the synchronization consistency based on the time difference of the binocular images and the buffer status, and controls the synchronous output of the binocular images when the synchronization conditions are met.
[0021] The synchronization consistency determination includes:
[0022] Synchronous output is allowed when the timestamp difference between the corresponding binocular images is less than a preset time threshold and both binocular images have completed full-frame buffering.
[0023] The exposure and illumination control module is used to jointly adjust the exposure parameters of the image sensor and the illumination intensity of the active illumination module based on the average brightness information of the captured images.
[0024] The present invention also provides a binocular synchronous imaging method based on time consistency control, comprising the following steps:
[0025] Step S1: The FPGA module sends a synchronous trigger signal to the binocular image sensor module through the synchronous trigger control logic to achieve binocular synchronous exposure;
[0026] Step S2: The FPGA module receives the binocular image data and performs identification and data reconstruction on the valid image data; a corresponding timestamp is generated when the valid image data is generated.
[0027] Step S3: Perform format conversion processing on the image data with timestamps to generate image data for synchronous output;
[0028] Step S4: Write the entire frame of binocular image data into the DDR storage module for caching; and determine the synchronization consistency of the binocular image time difference and cache status.
[0029] Step S5: When the stereo image meets the synchronization conditions, output the stereo image data synchronously.
[0030] The beneficial effects of the present invention: The binocular synchronous imaging system and method based on time consistency control described in this invention have the following technical advantages:
[0031] 1. Achieve end-to-end time consistency control;
[0032] By implementing synchronous trigger control, unified time reference, timestamp binding, and synchronization consistency determination, the time consistency control of binocular images is achieved throughout the entire process of acquisition, buffering, and output, thereby improving the synchronization accuracy of binocular images.
[0033] 2. Improve the verifiability of synchronization status;
[0034] By attaching a unified timestamp to valid image data and combining it with cache status for synchronization consistency constraints, the system achieves quantitative verification of the synchronization status of binocular images, thereby improving system reliability.
[0035] 3. Improve the stability of synchronous output;
[0036] By using DDR caching and unified synchronization clock control, the impact of differences in binocular image processing latency on synchronous output is reduced, achieving high-precision synchronous output.
[0037] 4. Possesses low-light adaptive imaging capability;
[0038] By dynamically adjusting the exposure time and active illumination brightness based on image brightness information, the imaging stability and environmental adaptability of close-range target observation scenarios at night can be improved.
[0039] 5. Highly applicable to engineering projects;
[0040] The system adopts a unified FPGA control architecture to realize synchronous acquisition, buffering and output of binocular images. It has the advantages of compact structure, high real-time performance and easy engineering implementation, and is suitable for high-reliability binocular imaging application scenarios. Attached Figure Description
[0041] Figure 1 This is a block diagram of the overall structure of a binocular synchronous imaging system based on time consistency control according to the present invention.
[0042] Figure 2 This is a block diagram of the internal structure of an FPGA.
[0043] Figure 3 This is a flowchart of a binocular synchronous imaging system method based on time consistency control according to the present invention. Detailed Implementation
[0044] Specific Implementation Method 1: Combination Figure 1 and Figure 2 This embodiment describes a binocular synchronous imaging system based on time consistency control. The synchronous imaging system includes a binocular image sensor module, an FPGA module, a DDR storage module, an active illumination module, an output interface module, and a communication interface module.
[0045] The binocular image sensor module is used to acquire dual-channel target scene image data acquired by two image sensors. The two image sensors use the same model of device and operate under the same exposure parameters, working mode and clock configuration conditions to ensure that the binocular image sampling state is consistent.
[0046] The FPGA module is used to implement binocular image synchronous acquisition control, time consistency determination, time synchronization marking, synchronization buffer control, and synchronous output control. The FPGA module includes a synchronization trigger module, a data receiving and parsing module, a time synchronization marking module, an image data processing module, an exposure and illumination control module, an image buffer control module, and a synchronous output control module. These modules are connected via a data bus and control signals.
[0047] The synchronous trigger control module is used to send a synchronous trigger signal to the binocular image sensor module to achieve synchronous exposure of binocular images. The synchronous trigger signal is connected to the two image sensors through two independent transmission paths. The two transmission paths are consistent in terms of trace length and electrical characteristics, thereby reducing the difference in transmission delay between the two trigger signals and improving the consistency of the start time of binocular image exposure.
[0048] In this embodiment, the synchronous trigger signal is generated simultaneously by the combinational logic inside the FPGA to ensure consistent timing of the dual-path trigger control.
[0049] The data receiving and parsing module receives binocular image data output from the binocular image sensor module, performs serial-to-parallel conversion, multi-channel alignment, data recognition, timing recovery, and data reassembly to ensure the timing consistency of binocular image data reception.
[0050] In this embodiment, the binocular image data is transmitted via an LVDS interface. The FPGA identifies valid image data by detecting synchronization signals, frame header information, and line identifier information, and performs data boundary identification and data reconstruction. During data reception, the data receiving and parsing module can filter out abnormal data to improve image data stability.
[0051] The time synchronization marking module is used to generate a corresponding timestamp when valid image data is generated, and bind the timestamp to the corresponding image frame, thereby realizing time synchronization marking of binocular images. The time synchronization marking module is implemented using a unified time base.
[0052] In this embodiment, the unified time base generates second information based on the platform clock and combines it with the FPGA's internal 20MHz clock for high-precision timing, with a time synchronization mark accuracy of less than or equal to 0.05μs. Since both stereo image data are processed by the same FPGA, the stereo image timestamps are generated based on the same time base, thus ensuring the consistency of stereo image timestamps.
[0053] The image data processing module is used to perform format conversion processing on the image data marked with timestamps by the received time synchronization mark module to generate image data for synchronous output.
[0054] In this embodiment, the FPGA interpolates the original Bayer format image data and converts it to YCbCr format image data to reduce image data transmission bandwidth and improve image output efficiency. The image data processing is implemented using a pipelined architecture to improve real-time processing capabilities.
[0055] The exposure and illumination control module is used to jointly adjust the exposure parameters of the image sensor and the illumination intensity of the active illumination module based on the image brightness information. It performs statistical analysis on the output image brightness information and dynamically adjusts the exposure time and PWM duty cycle based on the average image brightness. The system defaults to an initial state with an exposure time of 5ms and a PWM duty cycle of 50%.
[0056] When the image brightness is below a preset threshold, the exposure time is increased and the PWM duty cycle is increased simultaneously; when the image brightness is above the preset threshold, the exposure time is decreased and the PWM duty cycle is decreased simultaneously. The exposure time is adjusted in 5ms increments, and the PWM duty cycle is adjusted in 10ms increments, thereby reducing the impact of prolonged high-power operation of the active lighting module on the device's lifespan.
[0057] The image caching control module is used to write the entire frame of binocular image data output by the image data processing module into the DDR storage module for caching. The DDR storage module adopts a double-buffering or multi-buffering management method to reduce image read / write conflicts and improve the stability of binocular image synchronous caching.
[0058] The synchronization output control module is used to determine the synchronization consistency based on the time difference of the binocular images and the buffer status, and to control the synchronous output of the binocular images when the synchronization conditions are met.
[0059] In this embodiment, the synchronization consistency determination includes the following conditions:
[0060] (1) The difference in timestamps between the corresponding binocular images is less than a preset time threshold;
[0061] (2) Both binocular images are buffered for the entire frame.
[0062] When the above conditions are met simultaneously, the binocular images are allowed to be output synchronously; otherwise, it is determined that the current binocular images do not belong to the images acquired at the same time, and synchronous output is prohibited.
[0063] When the system is operating in 15fps mode, the preset time threshold is less than 66ms.
[0064] The synchronous output control module synchronously reads the binocular image data from the DDR storage module based on a unified synchronous clock, and simultaneously outputs the binocular image data through the output interface module.
[0065] The output interface module adopts a three-wire LVDS interface and achieves synchronous binocular image output based on a 30MHz double-edge transmission mode. The communication interface module is connected to the FPGA module and is used to receive external control commands and configuration parameters.
[0066] In this embodiment, by uniformly calibrating the DDR readout timing, the start time of the binocular image output is kept consistent, and the binocular image output delay accuracy is less than 0.02μs.
[0067] The system described in this embodiment is suitable for low-light, close-range target observation scenarios at night, achieving nighttime target imaging through an active illumination module. The effective target detection range is 3m, and the target recognition accuracy reaches 3cm. Specific embodiment two, combined with... Figure 3 This embodiment describes an imaging method for a binocular synchronous imaging system based on time consistency control, as described in Specific Embodiment 1. This method, implemented using the aforementioned binocular synchronous imaging system, includes the following steps:
[0068] Step S1. System initialization;
[0069] After the system is powered on, the FPGA completes the internal logic initialization and configures the working mode, exposure parameters, output mode and clock parameters of the binocular image sensor. At the same time, it initializes the DDR memory module, output interface module and communication interface module.
[0070] Step S2. Camera status configuration;
[0071] After the FPGA completes its internal logic initialization, it configures and changes the camera state based on the remote control commands received from the camera, the exposure time calculated by automatic exposure, and the PWM.
[0072] Step S3. Synchronously trigger binocular image acquisition;
[0073] The synchronous trigger control module generates dual synchronous trigger signals and sends them to the binocular image sensor through two physically consistent independent transmission paths to achieve binocular synchronous exposure.
[0074] Step S4. Image data reception and time synchronization marker;
[0075] The data receiving and parsing module receives image data output from the binocular image sensor and identifies and reconstructs valid image data.
[0076] When valid image data is generated, the time synchronization marker module generates a corresponding timestamp based on a unified time reference and binds the timestamp to the corresponding image frame.
[0077] Step S5. Image data processing;
[0078] The image data processing module performs format conversion on the received image data with timestamps and generates stereo image data for synchronous output.
[0079] Step S6. Combined adjustment of exposure and active illumination;
[0080] The exposure and illumination control module jointly adjusts the exposure time and active illumination brightness based on image brightness information. When the image brightness is below a preset threshold, the exposure time is increased and the active illumination brightness is increased simultaneously; when the image brightness is above the preset threshold, the exposure time is decreased and the active illumination brightness is decreased simultaneously.
[0081] That is: determine whether it is manual exposure mode. If so, proceed to step S7; otherwise, determine whether the average value meets the requirements. If so, proceed to step S7; otherwise, calculate the exposure time and PWM, and return to step S2.
[0082] Step S7. Image caching;
[0083] The image caching control module writes the entire frame of the binocular image into the DDR storage module for caching.
[0084] Step S8. Synchronization consistency determination;
[0085] The FPGA determines synchronization consistency based on the difference in timestamps between the binocular images and the buffer status.
[0086] Synchronous output is allowed when the time difference between the two images is less than a preset time threshold and both images have completed full-frame buffering. The synchronous output control module reads the two-view image data in the DDR synchronously based on a unified synchronous clock and outputs the two-view image data simultaneously through the output interface module; otherwise, synchronous output is prohibited.
[0087] The exposure and illumination control module described in this embodiment jointly adjusts the exposure time and active illumination brightness based on image brightness information. A synchronous step-by-step adjustment method is adopted for the exposure parameters and active illumination brightness to reduce the impact of prolonged high-power operation of the active illumination module on device lifespan.
[0088] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0089] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.
Claims
1. A binocular synchronous imaging system based on time consistency control, characterized by: The system includes a binocular image sensor module for acquiring binocular image data; An FPGA module used to realize the synchronous control, time consistency determination and synchronous output of binocular image data; A DDR storage module and an active illumination module are used to cache the binocular image data output by the FPGA module. The FPGA module includes a synchronization trigger module, a data receiving and parsing module, a time synchronization marker module, an image data processing module, an exposure and illumination control module, an image buffer control module, and a synchronization output control module. The synchronous trigger control module is used to send a synchronous trigger signal to the binocular image sensor module through two physically consistent independent transmission paths, so that the exposure start time of the binocular image sensor is consistent. The data receiving and parsing module receives image data output by the binocular image sensor and performs data recognition, time sequence recovery and data reconstruction; while recognizing valid image data, it generates corresponding timestamps through the time synchronization marking module and binds the timestamps to the corresponding image frames. The image data processing module performs format conversion processing on the received image data with timestamps to generate image data for synchronous output; The image caching control module writes the entire frame of the binocular image output by the image data processing module into the DDR storage module for caching; The synchronous output control module determines the synchronization consistency based on the time difference of the binocular images and the buffer status, and controls the synchronous output of the binocular images when the synchronization conditions are met. The synchronization consistency determination includes: Synchronous output is allowed when the timestamp difference between the corresponding binocular images is less than a preset time threshold and both binocular images have completed full-frame buffering. The exposure and illumination control module is used to jointly adjust the exposure parameters of the image sensor and the illumination intensity of the active illumination module based on the average brightness information of the captured image.
2. The binocular synchronous imaging system based on time consistency control according to claim 1, characterized in that: The time synchronization marker module is based on a unified time reference, uses the platform clock to generate second information, and performs high-precision timing according to the FPGA internal clock.
3. The binocular synchronous imaging system based on time consistency control according to claim 2, characterized in that: The FPGA has an internal clock frequency of 20MHz and a time synchronization mark accuracy of less than 0.05μs.
4. The binocular synchronous imaging system based on time consistency control according to claim 1, characterized in that: The preset time threshold is less than the single-frame acquisition cycle of the binocular image.
5. The binocular synchronous imaging system based on time consistency control according to claim 1, characterized in that: The synchronous output control module synchronously reads the binocular image data from the DDR storage module based on a unified synchronous clock, and synchronously outputs the binocular image data through the output interface module.
6. The binocular synchronous imaging system based on time consistency control according to claim 1, characterized in that: It also includes an output interface module, which adopts a three-wire LVDS interface and performs synchronous output based on a 30MHz double-edge transmission method, with a binocular image output delay accuracy of less than 0.02μs.
7. A binocular synchronous imaging method based on time consistency control, characterized by: This method is implemented using a binocular synchronous imaging system based on time consistency control as described in any one of claims 1-6, and includes the following steps: Step S1: The FPGA module sends a synchronous trigger signal to the binocular image sensor module through the synchronous trigger control logic to achieve binocular synchronous exposure; Step S2: The FPGA module receives the binocular image data and performs identification and data reconstruction on the valid image data; a corresponding timestamp is generated when the valid image data is generated. Step S3: Perform format conversion processing on the image data with timestamps to generate image data for synchronous output; Step S4: Write the entire frame of binocular image data into the DDR storage module for caching; and determine the synchronization consistency of the binocular image time difference and cache status. Step S5: When the stereo image meets the synchronization conditions, output the stereo image data synchronously.
8. The method according to claim 7, characterized in that: In step S5, when the difference in timestamps between the corresponding binocular images exceeds a preset time threshold, it is determined that the current binocular images do not belong to the images acquired at the same time, and synchronous output is prohibited.
9. The method according to claim 7, characterized in that: In step S2, the timestamp is generated based on the platform clock and the FPGA internal clock, and the time synchronization mark accuracy is less than 0.05μs.
10. The method according to claim 7, characterized in that: The method also includes jointly adjusting the exposure parameters and active illumination brightness based on image brightness information, and dynamically adjusting the exposure parameters and active illumination brightness using a synchronous stepping method.