Binocular camera data synchronization processing method and system and stereoscopic vision equipment

Abstract

The invention discloses a data synchronization processing method and system of a binocular camera and stereoscopic vision equipment. The method comprises the following steps: performing internal clock synchronization on each first-in first-out buffer and a corresponding camera, and performing working clock synchronization on each first-in first-out buffer; acquiring image data acquired by each camera; and based on each first-in first-out buffer, obtaining output data corresponding to each camera according to the corresponding image data. According to the invention, the asynchronous first-in first-out buffer is used to ensure that the data stream of each camera can be independently processed, data is ensured to be written into and read from the FIFO according to a correct clock and a synchronous signal, data processing of different clock domains is connected through the asynchronous characteristic of the FIFO, and ordered transmission of the data is ensured, so that HSYNC synchronization of the binocular camera is realized, and the accuracy of the HSYNC synchronization of the binocular camera is improved. And the data processing accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of visual imaging technology, and in particular to a data synchronization processing method and system for a binocular camera and a stereoscopic vision device. Background Art

[0002] A binocular camera is an imaging device that simulates the human eye visual system. By shooting the same scene with two cameras at the same time, stereoscopic visual information can be obtained, thereby achieving depth perception and three-dimensional reconstruction. Frame synchronization technology refers to ensuring that the two cameras are synchronized in time in order to obtain accurate stereo information. In binocular cameras, frame synchronization technology is crucial because if the frames of the two cameras are not synchronized, there will be a time difference between the left and right images, which will affect the accuracy of the depth information. The following characteristics are in the binocular camera frame synchronization technology: First, the requirements of the stereo matching algorithm: The binocular camera obtains depth information by calculating the disparity information between the left and right images. This is usually implemented using a stereo matching algorithm, which requires the left and right images to be synchronized in time to ensure the accuracy and consistency of the matching. Second, the difference in camera hardware: Even if the two cameras are of the same model, due to hardware differences, such as differences in device manufacturing, clock drift, etc., the frame rate and exposure time of the two cameras may have slight differences. These differences need to be corrected by frame synchronization technology. Third, the needs of depth sensing applications: In some applications that require real-time depth perception, such as robot navigation, three-dimensional reconstruction, etc., it is crucial to ensure the frame synchronization of the binocular camera to improve the robustness and accuracy of the system.

[0003] In the prior art, software synchronization usually involves timestamp calibration during the image processing stage, which may introduce processing delays, resulting in a slight time difference between the two camera images when they are actually captured. The scalability of hardware synchronization is too poor to meet the needs of multiple CIS sensors and MIPI or DVP interfaces.

[0004] However, in the existing frame synchronization technology solution, due to the internal clock drift of the image sensor, even if the two cameras are set with the same exposure time, their horizontal synchronization signals may have time differences, and the order of the horizontal synchronization signals of the two cameras may also be uncertain. This asynchrony will affect the accuracy and effect of the stereoscopic vision system.

[0005] Therefore, the existing multi-channel camera synchronization technology needs to be improved and developed. Summary of the invention

[0006] The main purpose of the present application is to provide a data synchronization processing method, system and stereo vision device for a binocular camera, aiming to solve the problem in the prior art that due to the internal clock drift of the camera's image sensor, even if the two cameras are set with the same exposure time, their horizontal synchronization signals may have time differences, resulting in low data processing accuracy.

[0007] A first aspect of an embodiment of the present application provides a data synchronization processing method for a binocular camera, which is applied to a stereoscopic vision device, wherein the stereoscopic vision device includes multiple cameras and multiple first-in-first-out buffers corresponding to each of the cameras; the data synchronization processing method for the binocular camera includes the following steps: the data synchronization processing method for the binocular camera includes: synchronizing the internal clock of each of the first-in-first-out buffers with the corresponding camera, and synchronizing the working clock of each of the first-in-first-out buffers; acquiring image data collected by each of the cameras; and obtaining output data corresponding to each of the cameras based on the corresponding image data based on each of the first-in-first-out buffers.

[0008] Optionally, in one embodiment of the present application, the multiple cameras include a first camera and a second camera, the multiple first-in-first-out buffers include a first buffer and a second buffer, the first camera is connected to the first buffer, and the second camera is connected to the second buffer; the internal clock of each first-in-first-out buffer is synchronized with the corresponding camera, and the working clock of each first-in-first-out buffer is synchronized, specifically including: synchronizing a first write clock corresponding to the first buffer with a first internal clock corresponding to the first camera, and synchronizing a second write clock corresponding to the second buffer with a second internal clock corresponding to the second camera; synchronizing a first read clock corresponding to the first buffer with a working clock, and synchronizing a second read clock corresponding to the second buffer with the working clock.

[0009] Optionally, in one embodiment of the present application, the multiple image data include first image data captured by the first camera and second image data captured by the second camera, and the multiple output data include first output data and second output data; the output data corresponding to each camera is obtained based on each first-in-first-out buffer according to the corresponding image data, specifically including: generating a first horizontal synchronization signal according to the first image data, and generating a second horizontal synchronization signal according to the second image data; according to the first horizontal synchronization signal, inputting the first image data into the first buffer to obtain first output data, and according to the second horizontal synchronization signal, inputting the second image data into the second buffer to obtain second output data.

[0010] Optionally, in one embodiment of the present application, the first camera has a first image sensor, and the second camera has a second image sensor; generating a first horizontal synchronization signal according to the first image data, and generating a second horizontal synchronization signal according to the second image data, specifically includes: when the first image sensor collects the first image data, generating the first horizontal synchronization signal corresponding to the first camera; when the second image sensor collects the second image data, generating the second horizontal synchronization signal corresponding to the second camera.

[0011] Optionally, in one embodiment of the present application, the first image data is input into the first buffer according to the first horizontal synchronization signal to obtain first output data, and the second image data is input into the second buffer according to the second horizontal synchronization signal to obtain second output data, specifically including: when the first horizontal synchronization signal is activated, the first image data is written into the first buffer, and when the second horizontal synchronization signal is activated, the second image data is written into the second buffer; when it is detected that the first image data exists in the first buffer and the second image data exists in the second buffer, a read operation is performed on the first image data to obtain first output data, and a read operation is performed on the second image data to obtain second output data.

[0012] Optionally, in one embodiment of the present application, when it is detected that the first image data exists in the first buffer and the second image data exists in the second buffer, a read operation is performed on the first image data to obtain first output data, and a read operation is performed on the second image data to obtain second output data, specifically including: obtaining a first empty flag signal corresponding to the first buffer and a second empty flag signal corresponding to the second buffer; when the first empty flag signal is in a non-empty state and the second empty flag signal is in a non-empty state, a read operation is performed on the first image data to obtain first output data, and a read operation is performed on the second image data to obtain second output data.

[0013] Optionally, in one embodiment of the present application, when it is detected that the first image data exists in the first buffer and the second image data exists in the second buffer, a reading operation is performed on the first image data to obtain first output data, and a reading operation is performed on the second image data to obtain second output data, specifically including: obtaining a first data count signal corresponding to the first buffer and a second data count signal corresponding to the second buffer; when the first data count signal and the second data count signal both exceed a preset threshold, a reading operation is performed on the first image data to obtain first output data, and a reading operation is performed on the second image data to obtain second output data.

[0014] A second aspect of the embodiment of the present application further provides a data synchronization processing system for a binocular camera, wherein the data synchronization processing system for a binocular camera comprises: A clock synchronization module, used to synchronize the internal clock of each first-in-first-out buffer with the corresponding camera, and synchronize the working clock of each first-in-first-out buffer; An image acquisition module, used to acquire image data collected by each of the cameras; A data synchronization module is used to obtain output data corresponding to each camera according to the corresponding image data based on each of the first-in-first-out buffers.

[0015] A third aspect of an embodiment of the present application further provides a stereoscopic vision device, wherein the stereoscopic vision device comprises: a memory, a processor, and a data synchronization processing program for a binocular camera stored in the memory and executable on the processor, wherein the data synchronization processing program for the binocular camera implements the steps of the data synchronization processing method for the binocular camera as described above when the data synchronization processing program for the binocular camera is executed by the processor.

[0016] A fourth aspect of an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a data synchronization processing program for a binocular camera, and when the data synchronization processing program for the binocular camera is executed by a processor, the steps of the data synchronization processing method for the binocular camera as described above are implemented.

[0017] Beneficial effect: The present application provides a data synchronization processing method, system and stereo vision device for a binocular camera. The method ensures that the data stream of each camera can be processed independently by using an asynchronous first-in-first-out buffer, and can ensure that data is written to and read from the FIFO according to the correct clock and synchronization signal. The asynchronous characteristics of the FIFO connect data processing in different clock domains, ensuring the orderly transmission of data, thereby realizing HSYNC synchronization of the binocular camera and improving the accuracy of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a schematic diagram of a time difference formed by image sensors of various cameras having respective internal clocks in the prior art; Figure 2 It is a flow chart of a preferred embodiment of the data synchronization processing method of the binocular camera of the present application; Figure 3 This is a schematic diagram of two image data being written and read out at the same time in a preferred embodiment of the data synchronization processing method for a binocular camera of the present application; Figure 4 It is a structural diagram of a preferred embodiment of the data synchronization processing system of the binocular camera of the present application; Figure 5 This is a structural diagram of a preferred embodiment of the stereoscopic vision device of the present application.

[0020] Description of reference numerals: 10. Data synchronization processing system of binocular camera; 100. Clock synchronization module; 200. Image acquisition module; 300. Data synchronization module. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical scheme and effect of the present application clearer and more specific, the technical scheme in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. The described embodiments are only possible technical implementations of the present application, not all possible implementations. Based on the embodiments in the present application, those skilled in the art can completely combine the embodiments of the present application to obtain other embodiments without creative work, and these embodiments are also within the scope of protection of the present application.

[0022] First, the nouns involved in the embodiments of the present application are introduced: CIS: CMOS Image Sensor (CMOS complementary metal oxide semiconductor, CIS image sensor); VSYNC:Vertical Synchronization(vertical synchronization signal); FSYNC: Frame Synchronization (frame synchronization signal); HSYNC:Horizontal Synchronization(horizontal synchronization signal); FIFO: First In First Out (First In First Out buffer).

[0023] In the related art, frame synchronization of a binocular camera system can be achieved through both software and hardware.

[0024] In the software synchronization method, first, camera driver synchronization: ensure that the images of the two cameras are synchronized in time by adjusting the camera driver. This usually involves implementing a synchronization mechanism at the camera driver level to perform time calibration before capturing the image. Second, timestamp calibration: each image captured by the camera is marked with a timestamp. In the subsequent image processing stage, the synchronization of the image is determined by comparing the timestamps. This requires ensuring that the camera's internal clock is accurate and the accuracy of the timestamp is high enough. Third, hardware trigger signal and software synchronization: start the camera capture process through a hardware trigger signal, and ensure that the camera's frame rate and exposure time are synchronized at the software level. This usually requires the camera device to support external triggering function.

[0025] In the hardware synchronization method, first, external trigger signal: trigger the two cameras to start capturing images at the same time through an external hardware signal. This can be an electrical signal generated by an external device, such as a hardware trigger or a dedicated synchronization device. The camera must support external trigger input. Second, hardware clock synchronization: use a unified hardware clock source to ensure that the internal clocks of the two cameras are synchronized. This can be achieved by connecting the two cameras to the same clock source or through a dedicated hardware synchronization module. Third, hardware trigger combined with synchronization signal: in some high-precision applications, hardware trigger and synchronization signal can be used in combination. The external hardware trigger signal is used to start the capture process, while the hardware synchronization signal ensures that the two cameras are completely synchronized in time.

[0026] Problems with software synchronization methods: First, processing delay: Software synchronization usually involves timestamp calibration during the image processing stage, which may introduce processing delays, resulting in a slight time difference between the two camera images when they are actually captured. Second, camera driver differences: Camera drivers produced by different manufacturers may differ, which may cause synchronization to be inaccurate or difficult to achieve. Third, timestamp accuracy: The accuracy of software synchronization depends on the accuracy of the timestamp, and the timestamp accuracy that some cameras may provide may not be sufficient to meet the needs of high-precision applications.

[0027] Problems with hardware synchronization methods: Device dependency: Hardware synchronization methods usually require camera devices to have specific hardware support, so they are highly dependent on the device, and there may be compatibility issues between cameras of different models or brands. Cost: Some hardware synchronization methods may require additional hardware devices (such as external triggers, synchronization modules), which may increase the cost of the system. Complexity: Hardware synchronization usually requires a more complex setup and calibration process, especially in multi-camera systems, which may increase the complexity of system deployment and maintenance. Flexibility: Some hardware synchronization methods may not be flexible enough to adapt to different scenarios and application requirements.

[0028] Although most of the current image sensors provide a hardware synchronization interface (FSYNC) for the vertical synchronization signal (VSYNC), the horizontal synchronization signal (HSYNC) will not be synchronized as the internal clock of the image sensor (CIS) fluctuates.

[0029] See also Figure 1 , since each camera's CIS sensor has its own internal clock, even if the same exposure time is set, there will be some time difference (T) in HSYNC, and there is uncertainty in the order of HSYNC_1 and HSYNC_2. Among them, T represents the time difference between the first line of the two cameras, which is equal to the number of pixels ± 1 / pixel frequency. HSYNC_1 and HSYNC_2: These are the horizontal synchronization signals of the two camera sensors. In video transmission, HSYNC is used to mark the beginning of a line of video data. In a binocular camera system, the two cameras generate their own HSYNC signals to synchronize the capture and transmission of their respective image data lines. IMG_DATA_1 and IMG_DATA_2: These refer to the image data signals from the two cameras. Each camera outputs its own image data stream, which needs to be synchronized with their respective HSYNC signals to ensure that the data is processed and analyzed correctly.

[0030] Secondly, the architecture of the stereoscopic vision device of the embodiment of the present application is introduced.

[0031] In some possible implementations, the stereoscopic vision device includes a plurality of cameras and a first-in-first-out buffer corresponding to each of the plurality of cameras.

[0032] In an embodiment of the present application, the plurality of cameras include a first camera and a second camera, the plurality of first-in-first-out buffers include a first buffer and a second buffer, the first camera is connected to the first buffer, the second camera is connected to the second buffer, the first camera has a first image sensor, and the second camera has a second image sensor.

[0033] Specifically, a FIFO buffer is used, and a FIFO buffer is connected to the image data stream of each camera. The function of the FIFO is to temporarily store data and output the data in a first-in-first-out order.

[0034] The following describes the data synchronization processing method, system and stereo vision device of the binocular camera of the embodiment of the present application with reference to the accompanying drawings. In view of the problem that due to the internal clock drift of the image sensor of the camera in the above-mentioned related art, even if the two cameras are set with the same exposure time, their horizontal synchronization signals may have time differences, resulting in low data processing accuracy, the present application provides a data synchronization processing method for binocular cameras, in which the data stream of each camera is ensured to be processed independently by using an asynchronous first-in-first-out buffer, and the data can be written and read from the FIFO according to the correct clock and synchronization signal. The data processing in different clock domains is connected through the asynchronous characteristics of the FIFO, and the orderly transmission of data is ensured, thereby realizing the HSYNC synchronization of the binocular camera and improving the data processing accuracy. As a result, the technical problem that due to the internal clock drift of the image sensor of the camera in the related art, even if the two cameras are set with the same exposure time, their horizontal synchronization signals may have time differences, resulting in low data processing accuracy is solved.

[0035] In order to solve the horizontal synchronization signal (HSYNC) synchronization problem in the binocular camera system in the prior art, the embodiment of the present application adopts a binocular camera HSYNC sequential reset mechanism based on asynchronous FIFO, which can solve the HSYNC synchronization problem of the binocular camera with lower resources and more flexible scalability.

[0036] Understandably, "asynchronous" means that the clock signals used for the read and write operations of the FIFO are not synchronized, that is, they can run independently at different clock frequencies or phases. "First-In-First-Out" (FIFO) is a data caching method that ensures that the first data stored will be retrieved first. Specifically, asynchronous FIFO has the following characteristics: Clock independence: The core feature of an asynchronous FIFO is that its read and write operations can be performed at completely different clock frequencies. This means that data written to the FIFO can be in one clock domain (for example, a clock synchronized with the camera sensor) and read in another clock domain (for example, a clock synchronized with the data processing unit).

[0037] Data synchronization: Due to the independence of the clock, asynchronous FIFO is particularly suitable for situations where data needs to be synchronized between dissimilar clock domains. It can avoid metastable problems that may occur when data is directly transmitted across clock domains.

[0038] First In, First Out: FIFOs ensure that data is read out in the same order they were written. This is achieved by simple pointer or flag management, pointing to the memory location that is currently writable or readable.

[0039] Buffering and flow control: During data transmission, FIFO can be used as a buffer to balance data transmission rate differences and prevent data overflow or loss.

[0040] Modular design: Due to its asynchronous nature, FIFO can be easily integrated into a variety of designs, especially in complex electronic systems that need to handle multiple clock domains.

[0041] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0042] The data synchronization processing method of the binocular camera described in the preferred embodiment of the present application is as follows: Figure 2 As shown, the data synchronization processing method of the binocular camera includes the following steps: In step S101, each of the first-in-first-out buffers is synchronized with the corresponding camera in terms of internal clock, and each of the first-in-first-out buffers is synchronized in terms of working clock.

[0043] In one possible implementation, a first write clock corresponding to the first buffer is synchronized with a first internal clock corresponding to the first camera, and a second write clock corresponding to the second buffer is synchronized with a second internal clock corresponding to the second camera; a first read clock corresponding to the first buffer is synchronized with a working clock, and a second read clock corresponding to the second buffer is synchronized with the working clock.

[0044] It should be noted that the two cameras are connected to their own FIFOs (first-in-first-out buffers), and each FIFO is responsible for the data stream of one camera. Establishing independent FIFOs for each camera is the basis of the entire system, ensuring that the data stream of each camera can be processed independently.

[0045] Specifically, when setting the write operation, the write clock of each FIFO is synchronized with the internal clock of each camera, which means that the data is written to the FIFO according to the camera's clock, and the write enable signal is controlled by the HSYNC signal of each camera. When HSYNC is activated, the data starts to be written to the FIFO according to the camera's clock, ensuring that the data can be correctly written according to the camera's clock frequency. When setting the read operation, the read clock of the FIFO is synchronized with the working clock of the post-processing module (such as the image processor), and the read enable signal is determined by the "non-empty" state of the two FIFOs, which means that the read operation will only be performed when there is data in the FIFO. This step ensures that the data can be read by the post-processing module at the appropriate time.

[0046] In step S102, image data collected by each camera is obtained.

[0047] In a possible implementation manner, the plurality of image data include first image data acquired by the first camera and second image data acquired by the second camera, and the plurality of output data include first output data and second output data.

[0048] In step S103, based on each of the first-in-first-out buffers, output data corresponding to each of the cameras is obtained according to the corresponding image data.

[0049] In one possible implementation, a first horizontal synchronization signal is generated based on the first image data, and a second horizontal synchronization signal is generated based on the second image data; based on the first horizontal synchronization signal, the first image data is input into the first buffer to obtain first output data, and based on the second horizontal synchronization signal, the second image data is input into the second buffer to obtain second output data.

[0050] It should be noted that, in the embodiment of the present application, even if there is a time difference or order uncertainty in the HSYNC signals of the two cameras, the FIFO can ensure the correct order and synchronization of the data. This is because the data is read in the order in which they enter the FIFO. Due to the asynchronous nature of the FIFO, it can smoothly transmit data between different clock domains, thereby solving the HSYNC synchronization problem caused by the internal clock drift of the CIS sensor. In this way, no matter which CIS sensor starts to transmit data first, the present application can effectively solve the synchronization problem.

[0051] That is to say, this application solves the HSYNC synchronization problem in the binocular camera system by connecting the two cameras to FIFO respectively and using the asynchronous characteristics of FIFO to achieve synchronous transmission and processing of data. This method not only has low resource usage, but also has good scalability and flexibility, and can adapt to CIS sensors of different types and brands. In addition, it avoids the complexity and limitations of traditional hardware synchronization methods, as well as the processing delays that may be introduced by software synchronization methods.

[0052] In a possible implementation, when the first image sensor acquires the first image data, the first horizontal synchronization signal corresponding to the first camera is generated; when the second image sensor acquires the second image data, the second horizontal synchronization signal corresponding to the second camera is generated.

[0053] Specifically, when the camera's image sensor finishes reading a line of image data, it sends out an HSYNC signal to indicate the end of the line of image data at the receiving end and to prepare to receive the next line of data. The horizontal synchronization signal provides a time reference for the image data, ensuring that each line of image data can be read by the receiving end at the correct time. In the image sensor, an HSYNC signal is generated every time a line of image data is acquired, which indicates the validity and end of the line of data.

[0054] Furthermore, when the image sensor starts to read a line of image data, the HSYNC signal will be activated, and it will usually be in a low state at this time, indicating that valid image data is being transmitted. Once the line of data is read, the HSYNC signal will return to a high level to inform the receiving end that the transmission of the line of data has ended. During the transmission process, the time interval between the rising and falling edges of the HSYNC signal determines the duration of a line of image data, which is critical for the display or processor to correctly interpret the image. At the same time, the repetition period of the HSYNC signal determines the line rate of the image, that is, how many lines of image data can be transmitted per second.

[0055] It can be understood that the write enable signal is used to enable the data writing process, while the read enable signal is used to enable the data reading process. The horizontal synchronization signal (HSYNC) is mainly used to indicate the start of scanning a line of data; while the write enable signal is used to control the data writing process. Specifically, in a binocular camera system, the FIFO write operation of each camera is controlled by its own HSYNC signal. When the camera's image sensor finishes reading a line of image data, it sends an HSYNC signal, which also acts as a write enable signal, indicating that data can begin to be written to the FIFO.

[0056] In one possible implementation, when the first horizontal synchronization signal is activated, the first image data is written to the first buffer, and when the second horizontal synchronization signal is activated, the second image data is written to the second buffer; when it is detected that the first image data exists in the first buffer and the second image data exists in the second buffer, the first image data is read to obtain first output data, and the second image data is read to obtain second output data.

[0057] FIFO is a storage structure with first-in, first-out characteristics. It includes a write end and a read end. Data enters the FIFO at the write end and is read at the read end. The design purpose of FIFO is to solve the problem of data transmission between different clock domains and ensure that data is processed in the order in which it enters.

[0058] In the examples of this application, see Figure 3 , connect the two cameras to FIFO respectively, the write clock of each FIFO is the camera clock, and the write enable is the HSYNC of each CIS sensor. The read clock is the working clock of the post-processing module, and the read enable signal is the inverse of the empty signal of the two FIFOs. FIFO_a is the first buffer, FIFO_b is the second buffer, IMG_DATA_1 is the first image data, and IMG_DATA_2 is the second image data.

[0059] In one possible implementation, a first empty flag signal corresponding to the first buffer and a second empty flag signal corresponding to the second buffer are obtained; when the first empty flag signal is in a non-empty state and the second empty flag signal is in a non-empty state, a read operation is performed on the first image data to obtain first output data, and a read operation is performed on the second image data to obtain second output data.

[0060] It can be understood that the empty signal is a status signal indicating whether the FIFO is empty. This signal is activated (usually high) when there is no data word available to read in the FIFO. The basic function of the empty signal is to prevent data from being read from an empty FIFO. Trying to read data from an empty FIFO is not only meaningless, but may also lead to system instability and erroneous data operations. Only when the empty signal indicates that the FIFO is not empty will the read enable signal be activated to allow data reading, thereby ensuring the rationality and effectiveness of the data reading operation. The generation of the empty signal is usually based on the word_count signal. When word_count is zero, the empty signal is activated.

[0061] Specifically, the read enable signal is generated by the logical negation of the empty signal, that is, when the empty signal indicates that the FIFO is empty, the read enable signal is not activated, and no read operation is performed at this time. Only when the empty signal indicates that the FIFO is not empty, the read enable signal is activated to allow data to be read. This ensures that data will not be read from an empty FIFO.

[0062] In another possible implementation, a first data count signal corresponding to the first buffer and a second data count signal corresponding to the second buffer are obtained; when both the first data count signal and the second data count signal exceed a preset threshold, a read operation is performed on the first image data to obtain first output data, and a read operation is performed on the second image data to obtain second output data.

[0063] It is understandable that the word_count signal (data count signal) usually refers to the number of data (data words) currently stored in the FIFO. This signal provides a quantitative indicator of how much data has been filled inside the FIFO. By monitoring the word_count signal, the system can determine the current state of the FIFO - for example, whether it is close to full or empty. This is crucial for controlling further read and write operations of data to avoid overflow or unnecessary operations. Specifically, start reading data when the amount of data reaches a certain threshold, or suspend writing when the amount of data is below a certain level to ensure data integrity and efficient operation of the system. The update of word_count is usually closely related to the writing and reading operations of data. Whenever data is written to the FIFO, the count increases; whenever data is read out, the count decreases. This dynamic update mechanism enables word_count to reflect the internal state of the FIFO in real time.

[0064] Specifically, the word_count signal may directly affect the generation of the read enable signal. For example, in some cases, even if the FIFO is not empty, if the amount of data in it does not reach a certain preset threshold, the read enable signal may be temporarily deactivated. This may be done to ensure that each read can process a certain amount of data and improve efficiency.

[0065] Next, a data synchronization processing system for a binocular camera proposed in an embodiment of the present application is described with reference to the accompanying drawings.

[0066] Figure 4 Schematic diagram of a data synchronization processing system for a binocular camera according to an embodiment of the present application.

[0067] like Figure 4 As shown, the data synchronization processing system 10 of the binocular camera includes: a clock synchronization module 100 , an image acquisition module 200 and a data synchronization module 300 .

[0068] Specifically, the clock synchronization module 100 is used to synchronize the internal clock of each of the first-in-first-out buffers with the corresponding camera, and synchronize the working clock of each of the first-in-first-out buffers; An image acquisition module 200, used to acquire image data collected by each of the cameras; The data synchronization module 300 is used to obtain the output data corresponding to each camera according to the corresponding image data based on each of the first-in-first-out buffers.

[0069] Figure 5 A schematic diagram of the structure of a stereoscopic vision device provided in an embodiment of the present application. The stereoscopic vision device may include: A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0070] When the processor 502 executes the program, the data synchronization processing method for the binocular camera provided in the above embodiment is implemented.

[0071] Furthermore, the stereoscopic vision device also includes: The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0072] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0073] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0074] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EIS) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0075] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0076] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0077] This embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the data synchronization processing method of the binocular camera as described above is implemented.

[0078] One embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the present application Figure 1 The data synchronization processing method of the binocular camera provided by any embodiment in the corresponding embodiment.

[0079] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0080] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0081] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0082] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a "computer-readable storage medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable storage media include the following: an electrical connection with one or N wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary, and then storing it in a computer memory.

[0083] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0084] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0085] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0086] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

[0087] It should be understood that the application of the present application is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to this application.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data synchronization processing method for a binocular camera, characterized in that: Applied to a stereoscopic vision device, the stereoscopic vision device comprises a plurality of cameras and a plurality of first-in-first-out buffers corresponding to each of the cameras; The data synchronization processing method of the binocular camera includes: Synchronizing the internal clock of each of the first-in-first-out buffers with the corresponding camera, and synchronizing the working clock of each of the first-in-first-out buffers; Acquire image data collected by each of the cameras; Based on each of the first-in-first-out buffers, output data corresponding to each of the cameras is obtained according to the corresponding image data.

2. The data synchronization processing method of the binocular camera according to claim 1, characterized in that: The plurality of cameras include a first camera and a second camera, the plurality of first-in-first-out buffers include a first buffer and a second buffer, the first camera is connected to the first buffer, and the second camera is connected to the second buffer; The step of synchronizing each of the first-in-first-out buffers with the corresponding camera's internal clock and synchronizing each of the first-in-first-out buffers with a working clock specifically includes: Synchronize a first write clock corresponding to the first buffer with a first internal clock corresponding to the first camera, and synchronize a second write clock corresponding to the second buffer with a second internal clock corresponding to the second camera; A first read clock corresponding to the first buffer is synchronized with a working clock, and a second read clock corresponding to the second buffer is synchronized with the working clock.

3. The data synchronization processing method of the binocular camera according to claim 2, characterized in that: The plurality of image data include first image data acquired by the first camera and second image data acquired by the second camera, and the plurality of output data include first output data and second output data; The obtaining, based on each of the first-in-first-out buffers, output data corresponding to each of the cameras according to the corresponding image data specifically includes: generating a first horizontal synchronization signal according to the first image data, and generating a second horizontal synchronization signal according to the second image data; According to the first horizontal synchronization signal, the first image data is input into the first buffer to obtain first output data, and according to the second horizontal synchronization signal, the second image data is input into the second buffer to obtain second output data.

4. The data synchronization processing method of the binocular camera according to claim 3, characterized in that: The first camera has a first image sensor, and the second camera has a second image sensor; The step of generating a first horizontal synchronization signal according to the first image data and generating a second horizontal synchronization signal according to the second image data specifically includes: When the first image sensor acquires the first image data, generating the first horizontal synchronization signal corresponding to the first camera; When the second image sensor collects the second image data, the second horizontal synchronization signal corresponding to the second camera is generated.

5. The data synchronization processing method of the binocular camera according to claim 3, characterized in that: The step of inputting the first image data into the first buffer according to the first horizontal synchronization signal to obtain first output data, and inputting the second image data into the second buffer according to the second horizontal synchronization signal to obtain second output data specifically includes: When the first horizontal synchronization signal is activated, the first image data is written into the first buffer, and when the second horizontal synchronization signal is activated, the second image data is written into the second buffer; When it is detected that the first image data exists in the first buffer and the second image data exists in the second buffer, a read operation is performed on the first image data to obtain first output data, and a read operation is performed on the second image data to obtain second output data.

6. The data synchronization processing method of the binocular camera according to claim 5, characterized in that: When it is detected that the first image data exists in the first buffer and the second image data exists in the second buffer, a reading operation is performed on the first image data to obtain first output data, and a reading operation is performed on the second image data to obtain second output data, specifically including: Acquire a first empty flag signal corresponding to the first buffer and a second empty flag signal corresponding to the second buffer; When the first empty flag signal is in a non-empty state and the second empty flag signal is in a non-empty state, a read operation is performed on the first image data to obtain first output data, and a read operation is performed on the second image data to obtain second output data.

7. The data synchronization processing method of a binocular camera according to claim 5, characterized in that: When it is detected that the first image data exists in the first buffer and the second image data exists in the second buffer, a reading operation is performed on the first image data to obtain first output data, and a reading operation is performed on the second image data to obtain second output data, specifically including: Acquire a first data counting signal corresponding to the first buffer and a second data counting signal corresponding to the second buffer; When both the first data count signal and the second data count signal exceed a preset threshold, a reading operation is performed on the first image data to obtain first output data, and a reading operation is performed on the second image data to obtain second output data.

8. A data synchronization processing system for a binocular camera, characterized in that: The data synchronization processing system of the binocular camera includes: A clock synchronization module, used to synchronize the internal clock of each first-in-first-out buffer with the corresponding camera, and synchronize the working clock of each first-in-first-out buffer; An image acquisition module, used to acquire image data collected by each of the cameras; A data synchronization module is used to obtain output data corresponding to each camera according to the corresponding image data based on each of the first-in-first-out buffers.

9. A stereoscopic vision device, characterized in that: The stereoscopic vision device includes: a memory, a processor, and a data synchronization processing program for a binocular camera stored in the memory and executable on the processor. When the data synchronization processing program for the binocular camera is executed by the processor, the steps of the data synchronization processing method for a binocular camera as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a data synchronization processing program for a binocular camera, and when the data synchronization processing program for a binocular camera is executed by a processor, the steps of the data synchronization processing method for a binocular camera according to any one of claims 1 to 7 are implemented.

Images