Naked eye 3D binocular image acquisition and real-time processing system based on hardware synchronous triggering
By using hardware synchronous triggering and stereo parallax correction, the problems of visual tearing and crosstalk in binocular image acquisition systems have been solved, achieving a highly stable and realistic naked-eye 3D display effect.
Patent Information
- Application Number
- CN202610060000.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-16
- Publication Date
- 2026-03-17
AI Technical Summary
Existing binocular image acquisition systems suffer from mismatch between left and right views when capturing images of moving objects, leading to visual tearing and dizziness. Furthermore, the lack of optical grating adaptation algorithms results in severe crosstalk, affecting stereoscopic perception.
The binocular image sensor is driven by a hardware synchronous triggering method that generates a global synchronization pulse signal through FPGA. Combined with stereo parallax correction and sub-pixel interleaving modules, it realizes real-time image processing and raster adaptation, eliminates time errors and reduces crosstalk.
The image acquisition timestamp error was controlled at the nanosecond level, eliminating motion parallax and visual tearing, improving the stability and comfort of stereoscopic vision, and reducing crosstalk through grating adaptation algorithm, thus enhancing the realism of stereoscopic perception.
Smart Images

Figure CN121691641A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing and 3D display technology, and relates to a naked-eye 3D binocular image acquisition and real-time processing system based on hardware synchronous triggering. Background Technology
[0002] With the continuous advancement of display and image processing technologies, naked-eye 3D display technology has garnered increasing attention in various display terminals, human-computer interaction systems, and visual application scenarios due to its inherent advantage of providing a stereoscopic visual experience without the need for special glasses. Naked-eye 3D displays typically rely on multi-view images or binocular image information, simulating human parallax by mimicking subtle spatial differences between left and right view images, thereby reconstructing the stereoscopic visual experience in the brain.
[0003] However, current mainstream binocular image acquisition solutions have significant limitations. Many systems use two independent universal serial bus cameras for software-based synchronization. This method is highly dependent on the operating system's scheduling mechanism and is easily affected by system load fluctuations, resulting in a frame time difference of tens of milliseconds between the left and right image acquisitions. When shooting moving objects, this time difference causes mismatch between the left and right views. After the light is split by the optical grating, users will experience noticeable visual tearing and dizziness. In addition, general-purpose camera systems often lack dedicated image synthesis algorithms for the characteristics of naked-eye 3D display gratings, and cannot effectively adapt to the optical parameters of lenticular lenses or slit gratings. This results in high crosstalk in the final displayed 3D image, weakening the stereoscopic effect and affecting the viewing experience.
[0004] Therefore, the industry urgently needs a binocular image acquisition and processing solution that can achieve strict synchronization at the hardware level to fundamentally eliminate time errors and integrate raster adaptation algorithms to improve the overall performance and user experience of naked-eye 3D display. Summary of the Invention
[0005] In view of this, the purpose of this invention is to provide a naked-eye 3D binocular image acquisition and real-time processing system based on hardware synchronous triggering, which aims to solve the problems of asynchronous binocular image acquisition and high latency in 3D format conversion in the prior art.
[0006] To achieve the above objectives, the present invention provides the following technical solution:
[0007] A naked-eye 3D binocular image acquisition and real-time processing system based on hardware synchronous triggering includes a binocular acquisition front-end, a timing synchronization control unit, an image signal processing unit, a stereo parallax correction module, a sub-pixel interleaving module, a data transmission unit, and a naked-eye 3D display terminal; The binocular acquisition front end includes a first image sensor and a second image sensor, both of which have the same optical axis parallelism and physical baseline distance, and are used to acquire left viewpoint and right viewpoint image signals respectively. The timing synchronization control unit is electrically connected to the binocular acquisition front end and is configured to output a global synchronization pulse signal to the external trigger pins of the two image sensors and provide a unified pixel reference clock. The image signal processing unit is used to receive raw RAW data and perform depixelation, white balance and automatic exposure gain calculation. The stereo parallax correction module stores the calibration parameter matrix of the binocular camera, which is used to perform epipolar correction on the input left and right image data. The subpixel interleaving module is configured to establish a mapping relationship between the physical pixels of the screen and the corrected binocular image data based on the microlens array parameters of the display terminal, and generate a composite view frame. The data transmission unit uses a double-buffered direct memory access (DMA) mechanism for high-speed data transfer. The glasses-free 3D display terminal is used to transmit the composite view frame to the glasses-free 3D display screen through a high-speed serial interface.
[0008] Furthermore, the timing synchronization control unit uses a field-programmable gate array (FPGA) as its hardware carrier, and simultaneously sends frame synchronization signals to the first and second image sensors through independent pins; the rising edge time of the frame synchronization signal is less than one pixel clock cycle.
[0009] Furthermore, the system has a master-slave automatic exposure linkage mechanism; The image signal processing unit calculates the exposure time and gain value based solely on the histogram statistics of the first image sensor, and writes these parameters into the register of the second image sensor in real time via the internal integrated circuit (I2C) control bus.
[0010] Furthermore, the stereo parallax correction module employs a lookup table-based remapping logic; The system pre-obtains the distortion coefficients and rotation / translation matrices using the checkerboard calibration method, generates a coordinate mapping table, and stores it in non-volatile memory; During operation, the logic circuit directly reads the corresponding source pixel value from the coordinate mapping table, performs bilinear interpolation, and outputs a row-aligned corrected image.
[0011] Furthermore, the subpixel rearrangement algorithm executed by the subpixel interleaving module specifically includes obtaining the period parameters, tilt angle, and current total number of viewpoints of the display screen bonding grating; For each physical sub-pixel coordinate on the screen, calculate its corresponding viewpoint index; If the region is determined to be the left viewpoint region, the pixel value is extracted from the corresponding coordinates of the corrected left image; If the region is determined to be the right viewpoint region, then the pixel values are extracted from the corrected right image.
[0012] Furthermore, the system's internal data transmission architecture adopts a dual-channel DMA ping-pong buffer mechanism; Allocate acquisition buffers and display buffers in Double Data Rate (DDR) memory; While the stereo data of the current frame is being written to the acquisition buffer via DMA, the composite view data of the previous frame that has been processed is simultaneously read out to the display interface.
[0013] Furthermore, the naked-eye 3D display terminal unit supports dynamic resolution adjustment; when insufficient system bandwidth or increased transmission link bit error rate is detected, the output composite view frame rate or resolution is automatically reduced.
[0014] Furthermore, the binocular image acquisition module is encapsulated on a rigid printed circuit board (PCB), and the center distance between the two image sensors is set to 60mm~65mm.
[0015] The beneficial effects of this invention are as follows: (1) The present invention adopts a hardware-level synchronization mechanism, which generates a high-frequency global trigger pulse through a timing synchronization control unit to directly drive the image sensor in the binocular acquisition front end. This physical-level synchronization method completely avoids the delay introduced by software scheduling, controls the timestamp error of left and right image acquisition to an extremely low level, effectively eliminates motion parallax and visual tearing in dynamic scenes, thereby improving the stability and comfort of stereoscopic vision.
[0016] (2) The system ensures the consistency of the brightness response of the binocular images through a master-slave automatic exposure linkage mechanism. The image signal processing unit adjusts the exposure parameters in real time based on the histogram statistics of the master device, forcing the slave device to respond synchronously. Combined with the pre-calibration remapping logic of the stereo parallax correction module, the system can automatically correct lens distortion and installation errors, and output a line-aligned epipolar corrected image, laying a solid foundation for subsequent processing.
[0017] (3) The embedded raster adaptation algorithm enables the system to generate composite views in real time based on the optical parameters of the naked-eye 3D display screen. The sub-pixel interleaving module realizes sub-pixel-level rearrangement by calculating the mapping relationship between the physical pixels of the screen and the viewpoint image, which significantly reduces optical crosstalk and enhances the realism and naturalness of the stereoscopic effect.
[0018] (4) The system adopts a dual-channel Direct Memory Access (DMA) ping-pong buffer mechanism, which opens up dual buffers for acquisition and display in memory, supporting parallel operations of data transfer and processing. This streaming architecture effectively copes with the throughput pressure of high-resolution binocular data, realizes pipelined processing with zero waiting time, and ensures the smoothness and continuity of real-time display.
[0019] (5) The naked-eye 3D display terminal supports dynamic resolution adjustment. When insufficient system bandwidth or degraded transmission link quality is detected, the output frame rate or resolution is automatically adjusted to prioritize the continuity of stereoscopic vision. This adaptive capability improves the stability and reliability of the system in complex environments.
[0020] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description
[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a block diagram of the overall system structure according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the system hardware synchronous data acquisition process in an embodiment of the present invention. Detailed Implementation
[0022] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0023] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0024] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.
[0025] Figure 1 The diagram shows the overall system structure of this invention, illustrating the connection relationships between the binocular acquisition module, the timing synchronization control unit, the image processing unit, the data transmission unit, and the naked-eye 3D display terminal.
[0026] Figure 2 This is a flowchart of the system hardware synchronous acquisition process according to an embodiment of the present invention, illustrating the overall system solution implementation process.
[0027] System hardware architecture and connectivity, such as Figure 1 As shown, the core control unit of this system adopts a heterogeneous architecture of "FPGA + ARM". Acquisition chain: Two identical CMOS image sensors are selected, sharing the same crystal oscillator to eliminate clock drift. The SYSCLK pins of both sensors are connected to the I / O pins of the FPGA. Processing chain: The FPGA is responsible for logic control and timing generation, while the ARM is responsible for running stereo correction and interleaving algorithms. The FPGA and ARM work together via a high-speed bus.
[0028] Hardware synchronous acquisition process as follows Figure 2As shown, the system executes strict hardware synchronization logic: Upon system power-on initialization, the FPGA configures the resolution, frame rate, and operating mode of the left and right sensors to Slave Mode via the I2C bus. An internal timer within the FPGA generates a 30Hz or 60Hz global synchronization pulse signal. The rising edge of the pulse signal simultaneously reaches the trigger pins of both sensors, immediately resetting the photodiodes and initiating exposure. After exposure, data is transmitted in parallel to the processing unit via the MIPI CSI-2 interface. At this point, the timestamp error between the two images is limited only by the difference in PCB trace length, typically on the nanosecond level, completely eliminating motion parallax.
[0029] Image processing and raster adaptation based on Figure 1 The architecture, with its internal processing unit executing the following logic: Epipolar Correction: Original images often contain lens distortion and installation errors. The processing unit reads the calibration data stored in Flash and resamples the left and right images, ensuring that the matching point of any point in the left image lies on the same horizontal scan line as the right image. This is a prerequisite for comfortable naked-eye 3D viewing. Subpixel Interleaving: Assuming the display screen uses an RGB arrangement and the raster is an oblique lenticular lens raster, the algorithm interleaves the subpixels of the left and right parallax images according to the raster coverage rules, recombining them into a "hybrid image".
[0030] Naked-eye 3D display terminal: The mixed image is sent to an LCD screen covered with a grating via an HDMI or DP interface. The human eye sees only the pixel information that originally belonged to the left image through the grating, and the right eye sees only the information of the right image, thus reconstructing the 3D image in the brain.
[0031] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A hardware synchronization trigger based naked eye 3D binocular image acquisition and real-time processing system, characterized in that: The system comprises a binocular acquisition front end, a time sequence synchronization control unit, an image signal processing unit, a stereo disparity correction module, a sub-pixel interleaving module, a data transmission unit and a naked-eye 3D display terminal. The binocular acquisition front end comprises a first image sensor and a second image sensor, both having the same optical axis parallelism and physical baseline distance, for respectively acquiring left-view and right-view image signals. The time sequence synchronization control unit is electrically connected with the binocular acquisition front end, configured to output a global synchronization pulse signal to the external trigger pins of the two image sensors and provide a unified pixel reference clock. The image signal processing unit is used for receiving original RAW data, performing demosaicing, white balance and automatic exposure gain calculation. The stereo disparity correction module stores a calibration parameter matrix of the binocular camera, for performing epipolar line correction on the input left and right image data. The sub-pixel interleaving module is configured to establish a mapping relationship between the screen physical pixels and the corrected binocular image data according to the microlens array parameters of the display terminal, and generate a composite view frame. The data transmission unit adopts a double-buffer direct memory access (DMA) mechanism for high-speed data flow conversion. The naked-eye 3D display terminal is used for transmitting the composite view frame to the naked-eye 3D display screen through a high-speed serial interface.
2. The hardware synchronization trigger based 3D binocular image acquisition and real-time processing system according to claim 1, wherein: The time sequence synchronization control unit adopts a field programmable gate array (FPGA) as a hardware carrier, and sends a frame synchronization signal to the first and second image sensors through independent pins; the rising edge time of the frame synchronization signal is less than one pixel clock cycle.
3. The hardware synchronization trigger based 3D binocular image acquisition and real-time processing system according to claim 1, wherein: The system has a master-slave type automatic exposure linkage mechanism. The image signal processing unit only calculates the exposure time and gain value according to the histogram statistical data of the first image sensor, and writes the parameters into the register of the second image sensor in real time through an internal integrated circuit (I2C) control bus.
4. The hardware synchronization trigger based 3D binocular image acquisition and real-time processing system according to claim 1, wherein: The stereo disparity correction module adopts a remapping logic based on a lookup table. The system pre-acquires distortion coefficients and rotation and translation matrices through a checkerboard calibration method, generates a coordinate mapping table and stores it in a non-volatile memory. During operation, a logic circuit directly reads corresponding source pixel values according to the coordinate mapping table to perform bilinear interpolation, and outputs a line-aligned corrected image.
5. The hardware synchronization trigger based 3D binocular image acquisition and real-time processing system according to claim 1, wherein: The sub-pixel rearrangement algorithm executed by the sub-pixel interleaving module specifically includes obtaining the period parameters, tilt angles and current total number of viewpoints of the display screen grating; For each physical sub-pixel coordinate on the screen, the corresponding viewpoint index is calculated; If it is determined as a left-viewpoint region, the pixel value is extracted from the corresponding coordinate of the corrected left image; If it is determined as a right-viewpoint region, the pixel value is extracted from the corrected right image.
6. The hardware synchronization trigger based 3D binocular image acquisition and real-time processing system according to claim 1, wherein: The data transmission architecture inside the system adopts a double-channel DMA ping-pong buffer mechanism; A collection buffer and a display buffer are opened in a double data rate (DDR) memory; When the binocular data of the current frame is being written into the collection buffer through DMA, the composite view data of the last frame that has been processed is read out to the display interface at the same time.
7. The hardware synchronization trigger based 3D binocular image acquisition and real-time processing system according to claim 1, wherein: The naked-eye 3D display terminal unit supports dynamic resolution adjustment; when it is detected that the system bandwidth is insufficient or the transmission link error rate is increased, the composite view frame rate or resolution output is automatically reduced.
8. The hardware synchronization trigger based 3D binocular image acquisition and real-time processing system according to claim 1, wherein: The binocular image acquisition module is packaged on a rigid printed circuit board (PCB) board, and the center distance of the two image sensors is set to 60mm-65mm.
Citation Information
Cited By
RAW domain epipolar correction and demosaicing combined processing architecture and method
CN122089560A
A raw domain epipolar correction and demosaicing combined processing system and method
CN122089560B
