Multi-camera synchronization performance detection and calibration system and method

Abstract

The application provides a multi-camera synchronization performance detection and calibration system and method, comprising a time cursor device, a synchronization trigger device and a processing unit. The method comprises: providing a time cursor device with visual appearances changing with time according to a predetermined encoding rule; sending a trigger signal to multiple cameras through the synchronization trigger device to shoot a calibration object; the processing unit collects images, decodes the visual appearances in the images to determine the exposure time of each camera, calculates the synchronization error between the cameras, and controls the delay module to adjust the trigger delay of at least one camera in a closed-loop manner based on the error to reduce the synchronization error. The application uses a Gray code encoding rule to solve the problem of decoding ambiguity or error at the moment of switching the state of the calibration target in the prior art, realizes automatic synchronization calibration with high robustness and high precision, and has the advantages of non-invasive and low cost.

Description

Technical Field

[0001] This invention relates to image processing technology, and more specifically, to a system and method for detecting and calibrating the synchronization performance of multiple cameras. Background Technology

[0002] In many applications such as robot vision, 3D reconstruction, and embodied intelligence, it is often necessary to deploy multiple cameras to simultaneously capture the same scene from different perspectives to ensure the accuracy of subsequent data fusion and analysis. To achieve multi-camera synchronization, existing technologies generally employ hardware synchronization triggering schemes, where a unified trigger source sends trigger signals to all cameras. However, this scheme cannot completely guarantee the consistency of the actual exposure times of each camera. This is because different cameras (even those of the same model) have slight differences in aspects such as the length of the trigger signal transmission cable, internal electronic link delay, sensor response characteristics, and firmware processing time. These differences result in an inherent and varying trigger delay between each camera receiving the trigger signal and the actual start of exposure. This delay difference is the main reason for the degradation of the actual synchronization performance of multi-camera systems.

[0003] To measure and calibrate this synchronization error, several solutions have been proposed in the prior art. For example, one approach involves having all cameras to be calibrated simultaneously capture images on a rapidly changing time display device (such as a digital tube timer). The actual exposure time of each camera is then deduced by reading the time readings from the images captured by each camera, and the synchronization error between the cameras is calculated accordingly. Subsequently, the system can adjust the timing of the next trigger signal sent to each camera based on the calculated synchronization error, for example, by applying a certain trigger delay to cameras that were exposed too early, thus forming a feedback control closed loop to gradually calibrate and reduce the synchronization error. However, this type of approach has an inherent drawback: when the camera's exposure occurs precisely at the moment the digital display device changes (e.g., the digital tube changes from "9" to "0"), the image captures a blurry or incomplete intermediate state. In this case, decoding the time readings in the image becomes very difficult and may even result in serious decoding errors. This lack of robustness in decoding at the moment of target state switching limits the accuracy of synchronization error measurement and the reliability of the entire calibration system, especially in applications requiring microsecond-level or even higher precision synchronization. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this invention is to provide a system and method for multi-camera synchronization performance detection and calibration.

[0005] A multi-camera synchronization performance detection and calibration system provided by the present invention includes: A time cursor device, the visual appearance of which is driven by a reference clock to change over time according to a predetermined encoding rule; The synchronous triggering device is configured to send trigger signals to multiple cameras respectively, and the synchronous triggering device provides each of the multiple cameras with an independent programmable trigger delay channel. The processing unit is configured as follows: Acquire images captured by multiple cameras, including the time cursor device; The visual appearance in each of the images is decoded to determine the exposure time of the camera corresponding to that image; based on the determined exposure times of each camera, the synchronization error between the plurality of cameras is calculated; Based on the calculated synchronization error, the synchronization triggering device is controlled to adjust the triggering delay of at least one camera to reduce the synchronization error in a closed-loop manner.

[0006] Furthermore, the time cursor device includes a counter composed of multiple light-emitting diodes, each light-emitting diode corresponding to a binary bit of Gray code; The combination of the on / off states of these multiple LEDs corresponds to the Gray code-encoded count value.

[0007] Furthermore, the encoding rule of the time cursor device has an encoding step cycle, which is not less than the maximum exposure time set by the plurality of cameras when shooting, and the count increments at most once within any exposure window.

[0008] Furthermore, the processing unit is also configured to perform sub-step interpolation calculations, which include: When the visual appearance presented in the image corresponds to the intermediate state of the state transition of the encoding rule, a fractional time offset is calculated based on the gray value of the light-emitting diode in the transition state in the visual appearance.

[0009] Furthermore, the processing unit is also configured to: When at least one of the plurality of cameras is a rolling shutter camera, for each of the at least one rolling shutter camera, the progressive exposure time of the rolling shutter camera is determined by analyzing the changes in the visual appearance presented along the rolling shutter scanning direction in a single frame of the image captured by it, and the line scan rate of the rolling shutter camera is calculated based on the progressive exposure time.

[0010] A method for detecting and calibrating multi-camera synchronization performance according to the present invention includes: A time cursor device is provided, the visual appearance of which is driven by a reference clock to change over time according to a predetermined encoding rule; A synchronous triggering device capable of providing an independent programmable trigger delay for each of the multiple cameras sends trigger signals to the multiple cameras to capture the time cursor device; Acquire images captured by the multiple cameras; The visual appearance in each of the images is decoded to determine the exposure time of the camera corresponding to that image; Based on the determined exposure times of each camera, the synchronization error between the multiple cameras is calculated; Based on the calculated synchronization error, the triggering delay of the synchronization triggering device for at least one camera is adjusted in a closed-loop manner to reduce the synchronization error.

[0011] Furthermore, the time cursor device includes a counter composed of multiple light-emitting diodes, each light-emitting diode corresponding to a binary bit of Gray code; The combination of the on / off states of these multiple LEDs corresponds to the Gray code-encoded count value.

[0012] Furthermore, the encoding rule of the time cursor device has an encoding step cycle, which is not less than the maximum exposure time set by the plurality of cameras when shooting, and the count increments at most once within any exposure window.

[0013] Furthermore, it also includes a step of performing sub-step interpolation calculations, wherein performing sub-step interpolation calculations includes: When the visual appearance presented in the image corresponds to the intermediate state of the state transition of the encoding rule, a fractional time offset is calculated based on the gray value of the light-emitting diode in the transition state in the visual appearance.

[0014] Furthermore, when at least one of the plurality of cameras is a rolling shutter camera, for each of the at least one rolling shutter camera, the progressive exposure time of the rolling shutter camera is determined by analyzing the changes in the visual appearance presented along the rolling shutter scanning direction in a single frame of the image captured by it, and the line scan rate of the rolling shutter camera is calculated based on the progressive exposure time.

[0015] Compared with the prior art, the present invention has the following beneficial effects: This invention improves measurement robustness and accuracy. It employs Gray code to encode time information, utilizing the characteristic that only one bit flips between adjacent values. This fundamentally solves the decoding ambiguity or error problem caused by multiple simultaneous changes in bits during target state switching in existing technologies, significantly enhancing the robustness and accuracy of exposure time measurements.

[0016] This invention achieves integrated and automated detection and calibration. By combining high-precision detection with closed-loop feedback control, it can accurately measure synchronization errors and automatically calculate and issue trigger delay compensation, iterating and optimizing until the error converges. The entire process requires no manual intervention, realizing integrated "detection-calibration" and greatly improving calibration efficiency.

[0017] This invention achieves non-invasive, low-cost calibration. It transforms the "exposure moment," which is difficult to measure directly, into "image information" that can be captured and decoded by the camera itself. This eliminates the need for expensive and complex hardware such as external photodetectors for each camera, reducing system costs and deployment complexity, and requires no hardware modifications to the camera under test.

[0018] This ensures system-level accuracy. The time cursor device and synchronization trigger device in this invention can both be driven by a unified reference clock, ensuring that time encoding, trigger signals, and delay compensation are all performed under the same time base, thus guaranteeing the accuracy of measurement and calibration. Attached Figure Description

[0019] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 A schematic diagram of the overall composition of the multi-camera synchronization performance detection and calibration system provided in the embodiments of this application; Figure 2 A schematic diagram of the time cursor device provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the synchronization triggering device provided in the embodiments of this application; Figure 4 This is a schematic diagram of the internal functional modules of the processing unit provided in the embodiments of this application; Figure 5 A flowchart illustrating the multi-camera synchronization performance detection and calibration method provided in this application embodiment; Figure 6 This is a schematic diagram illustrating the Gray code encoding principle used in the embodiments of this application; Figure 7 This is a schematic diagram illustrating the principle of sub-step grayscale interpolation provided in an embodiment of this application. Figure 8 This is a schematic diagram of the imaging effect of the rolling shutter camera in an embodiment of this application. Detailed Implementation

[0020] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0021] Example 1 This application provides a multi-camera synchronization performance detection and calibration system and method to achieve accurate measurement and automated calibration of the exposure times of multiple cameras.

[0022] Reference Figure 1 This diagram illustrates the overall composition of a multi-camera synchronous closed-loop calibration system according to an embodiment of this application. Specifically, the system may include: a time cursor device, a synchronization trigger device, a group of cameras to be calibrated (e.g., camera 1 to camera N), and a processing unit. As an optional implementation, to ensure the uniformity of the system's time reference, a highly stable reference clock can also be set in the system. The output signal of the reference clock can be simultaneously fed to both the time cursor device and the synchronization trigger device. Thus, the encoding of time information, the generation of trigger signals, and the delay control are all performed under the same precise time coordinate system, thereby fundamentally guaranteeing the accuracy of subsequent measurements and calibrations.

[0023] Specifically, a time cursor device, as a type of time cursor device, functions to transform abstract time information into a visual appearance that changes over time and can be captured by a camera. This device needs to be placed within the common field of view of all cameras to be calibrated to ensure that each camera can clearly capture its appearance. For example... Figure 3 As shown, the synchronization trigger device, also known as a synchronization trigger, is responsible for distributing exposure trigger signals to all cameras. It should be noted that this synchronization trigger device provides each camera with an independent, programmable trigger delay channel, allowing the processing unit to apply a precisely controllable delay to the trigger signal sent to any camera as needed. The cameras connect to the processing unit via data interfaces such as Universal Serial Bus, Ethernet, or camera links to transmit captured image data. Correspondingly, the synchronization trigger device also connects to the processing unit via a control interface to receive delay compensation commands from the processing unit. The processing unit, as the control and computing core of the system, can be implemented using devices with sufficient computing power, such as a system-on-a-chip, a field-programmable gate array, an industrial computer, or a server.

[0024] Combined with reference Figure 2 and Figure 6 , Figure 2 This is a schematic diagram of a specific structure of a time cursor device. Figure 6This demonstrates the encoding principle of Gray code. In one embodiment of this application, the time cursor device can be implemented as a strip of light-emitting diodes arranged in a straight line. The strip contains, for example, 32 time cursor LEDs, each corresponding to one binary bit. The device integrates a 32-bit counter driven by a reference clock input, which accumulates at the frequency of the reference clock. To improve the robustness of decoding, the output value of the counter is not directly used to control the LEDs, but is first converted by a Gray code encoder.

[0025] It is understandable that Gray code, also known as cyclic binary code, has the key characteristic that only one bit changes between any two adjacent count values. For example, when changing from the value 3 (Gray code 0010) to the value 4 (Gray code 0110), only the second bit changes from 0 to 1. In contrast, in ordinary binary code, when changing from 3 (0011) to 4 (0100), three bits are flipped simultaneously. In rapidly changing scenarios, if the camera's exposure occurs precisely at the moment of code transition, using ordinary binary code can easily lead to blurry or incorrect image capture due to the intermediate states caused by multiple bit flips, resulting in decoding failure. Conversely, with Gray code, since only one bit changes, even if an intermediate state is captured, its uncertainty is limited to that unit, thus greatly improving the reliability and accuracy of decoding at the moment of state transition. Therefore, in this embodiment, the value of the 32-bit counter is converted into a 32-bit Gray code in real time, and each bit of the Gray code controls the on (representing 1) or off (representing 0) state of the corresponding time cursor LED.

[0026] Assuming the reference clock frequency is 1MHz, the encoding step cycle of the time cursor device is... The interval is 1 microsecond. That is, every microsecond, the pattern composed of 32 time-marker LEDs is updated to represent a new time count value. To ensure the uniqueness of the decoding, an important constraint must be met: the encoding step cycle of the time-marker device. It should not be less than the maximum exposure time set by the camera under test during shooting, that is, it should satisfy the formula:

[0027] Where i is the camera index. This constraint ensures that during any exposure, the camera's sensor records at most one complete Gray code pattern, thus avoiding decoding ambiguity caused by exposure time spanning multiple encoded patterns.

[0028] The 32-bit brightness pattern captured by the camera is decoded using Gray code to reconstruct the corresponding count value. Multiplying this count value by the counting step period gives the absolute exposure time of the camera in that frame. 32 bits provide... Each of these distinct states, taking microsecond-level steps as an example, can achieve approximately [number] non-repeating distances. (Approximately 71 minutes) is sufficient to cover a single detection / calibration cycle without causing time ambiguity.

[0029] Gray code involves only one bit flip between two adjacent count values. Therefore, at any given moment, at most one LED is in an intermediate on / off state during a counter increment. This occurs when the camera exposure window has just crossed one count increment. At that time, the captured image showed 31 LEDs in a stable and clearly readable state, with only one LED exhibiting a transition state in the middle (half-brightness); thus, it can be concluded that the exposure time was within the optimal range. and Between these points, the decoding error is strictly limited to within a certain range. Within a counting step. In contrast, if ordinary binary code is used, when crossing carry (such as... When multiple bits flip simultaneously, capturing the instant of the transition can cause widespread errors. Gray code thus significantly improves the decoding robustness of the time cursor under high-speed changes.

[0030] Since the 32-bit Gray code counter is an absolute code that is unique throughout the entire process and is traversed sequentially as the binary counter increments—each count value appears only once in the entire non-repeating counting process, and there is no periodic repetition—there is no need to set up an additional "coarse code / fine code" segmentation or disambiguation mechanism. The 32-bit pattern decoded in any frame uniquely corresponds to an absolute moment.

[0031] Please refer to the reference. Figure 4 This demonstrates the functional module division within the processing unit. Logically, the software or firmware within the processing unit can be divided into three core modules: an image acquisition module, a decoding and analysis module, and a calibration and control module. The image acquisition module is responsible for communicating with all cameras, receiving and buffering the transmitted image data. The decoding and analysis module, as the core of data processing, executes a specific algorithm on each acquired image: first, it locates the imaging area of ​​the time cursor device in the image, then identifies the on / off state of each time cursor LED to decode the corresponding Gray code value; subsequently, this module converts the Gray code value in reverse to a regular binary count value and multiplies this count value by the encoding step period. (In this example, it is 1 microsecond), thus accurately retrieving the true exposure time when the camera captured this frame. The calibration control module, as the top-level decision and control unit, is responsible for coordinating the entire calibration process. Based on the exposure time data of each camera provided by the decoding and analysis module, it calculates the synchronization error and generates corresponding delay compensation commands to be sent to the synchronization triggering device.

[0032] Now, please refer to the reference. Figure 5The figure illustrates in detail the complete workflow of the multi-camera synchronous closed-loop calibration method provided in the embodiments of this application in the form of a flowchart.

[0033] The entire process is initiated by the calibration control module. In the initial state, the calibration control module sets the trigger delay compensation for all camera channels to zero.

[0034] Step S101, Synchronous Trigger. The calibration control module sends a command to the synchronous triggering device, requesting it to send trigger signals to all cameras simultaneously. Since the initial compensation is zero, theoretically all cameras receive the trigger pulse at the same time. After receiving the signal, each camera undergoes its own inherent internal delay before starting exposure and capturing the time cursor device.

[0035] Step S102: Image Acquisition. After shooting, each camera sends image data containing a time cursor pattern to the processing unit via the data interface. The image acquisition module within the processing unit is responsible for receiving and storing these images from different cameras.

[0036] Step S103, Decoding and Calculation. The image acquisition module transmits image data to the decoding and analysis module, which processes each image. For example, for an image from camera 1, the module decodes the captured Gray code pattern and calculates its exposure time t1. Similarly, images from cameras 2 to N are processed to obtain their respective exposure times t2, ..., t. N Subsequently, the decoding and analysis module calculates the synchronization error between these cameras. As an optional implementation, the latest exposure time among all cameras can be selected as the reference time t. ref = max(t1, t2, …, t N Then, the lead time of each camera relative to this reference is calculated, i.e., the synchronization error Δt. i = t ref - t i .

[0037] Step S104, Calibration Compensation. The decoding and analysis module will calculate the synchronization errors Δt1, Δt2, …, Δt for each camera. N The error values ​​are then passed to the calibration control module. Based on these error values, the calibration control module calculates a new trigger delay compensation for each camera. The compensation logic is that for cameras that expose too early, their trigger delay needs to be increased to make them "wait" for a period of time before exposing. A simple compensation strategy is to use the calculated synchronization error Δt... iThis serves as the delay compensation amount to be applied during the next trigger. For example, if camera 1 is 15μs ahead of the reference camera (i.e., Δt1 = 15μs), the calibration control module will generate an instruction requiring the synchronization triggering device to apply a 15μs delay to the trigger channel of camera 1 during the next trigger. The calibration control module will then send the new compensation amount calculated for all cameras to the synchronization triggering device, which will update the delay registers of each internal channel.

[0038] Step S105: Determine if the requirements are met. The calibration control module checks the calculated maximum synchronization error max(Δt1, Δt2, …, Δt) N The system checks whether the error is less than a preset target accuracy threshold, such as 1 μs. If the error is still greater than this threshold, the calibration is not yet complete, and the process returns to step S101 to begin a new round of iteration. During this process, the synchronization trigger device sends a trigger signal using the newly updated delay compensation. This "trigger-acquisition-decoding-calculation-compensation" process forms a closed loop. Each iteration makes the actual exposure time of each camera tend to be consistent, and the synchronization error gradually decreases. Conversely, if the judgment result is that the maximum synchronization error is less than or equal to the preset threshold, the calibration is considered successful, and the process ends. At this time, the final delay compensation value of each channel stored in the synchronization trigger device will be fixed as the synchronization parameter when the multi-camera system is working normally.

[0039] Through the above-described automated closed-loop process, the system provided in this embodiment can automatically and accurately calibrate a system composed of multiple cameras of different models and cable lengths, which may initially have a synchronization error of tens of microseconds, to within 1μs within a few seconds (usually only a few iterations are needed to converge), thereby achieving high-efficiency and high-precision synchronization calibration.

[0040] Example 2 This embodiment, based on Embodiment 1, introduces a technical solution to further improve the accuracy of exposure time measurement, namely, a sub-step grayscale interpolation method. This method can achieve an encoding step period T much longer than that of the time cursor device. step This improves the temporal resolution, thereby increasing the final accuracy of synchronization calibration to the nanosecond level.

[0041] The system hardware structure is exactly the same as in Embodiment 1, including a time cursor device, a synchronization trigger device, a camera, and a processing unit. The improvement in this embodiment mainly lies in the algorithm enhancement of the decoding and analysis module deployed within the processing unit.

[0042] Reference Figure 7This figure illustrates the principle of sub-step grayscale interpolation provided in this embodiment. In Embodiment 1, the decoding and analysis module determines the exposure time by decoding the complete brightness and darkness pattern; its temporal resolution is limited by the encoding step period T. step (e.g., 1 μs). However, when the camera's exposure window (whose duration is T) exp When the exposure starts at time t0 and ends at time t1, exactly spanning a state transition of the time cursor device, a special phenomenon will be captured in the image.

[0043] Because the time cursor device uses Gray code encoding, each state transition involves only the state flipping of one time cursor LED. Assuming a transition occurs between the count value N and its adjacent count value N+1, the absolute time of this transition is τ = (N+1) * T. step If the camera's exposure window [t0, t1] exactly covers the transition time τ, then the brightness of the LED corresponding to the transition in the image will neither be completely off nor completely on, but will present an intermediate grayscale value. The decoding and analysis module, through enhanced algorithms, can use this information to inversely determine a more precise exposure time.

[0044] Its specific implementation may include the following steps: When the decoding and analysis module is processing an image from a camera, if it finds that only one LED in the time cursor area is in an intermediate grayscale state, while the other LEDs are in a stable fully lit or fully off state, then the sub-step interpolation calculation process is triggered. 1. Identification and Classification: The algorithm first identifies one stable LED and one LED in a transition state.

[0045] 2. Determine adjacent count values: Based on the on / off pattern of the 31 stable LEDs, a Gray code value can be unambiguously decoded. This value corresponds to one of the two adjacent count values ​​N and N+1 at which the time cursor is located during the exposure period. For example, if the stable bit is decoded as count value N, then it can be known that the exposure window spans the transition from N to N+1.

[0046] 3. Determining the transition direction: Since the count values ​​N and N+1 are known and differ by only one bit, the transition direction can be uniquely determined by comparing the bit values ​​(0 or 1) of these two count values ​​at the position of the LED in the transition phase. For example, if the count value N is 0 at this bit and the count value N+1 is 1 at this bit, the transition direction is "off → on". Conversely, it is "on → off". It should be noted that the unambiguity of this step is a key advantage brought by Gray code.

[0047] 4. Grayscale Quantization: The algorithm reads the pixel grayscale value of the switching LED and normalizes it to the [0,1] interval to obtain the normalized grayscale value f. This value can be approximately considered to be proportional to the total exposure time T. exp The percentage of time the LED is in the "on" state.

[0048] 5. Interpolation calculation: Based on the determined jump direction, different interpolation formulas are applied to calculate the position of the exposure window relative to the jump time τ.

[0049] If the transition direction is "off → on" (0 → 1), it means that the LED is off in the first half of the exposure window and becomes on after the transition time τ. Therefore, the acquired grayscale value f is mainly contributed by the second half of the exposure window [τ, t1]. The following approximate relationship can be established: f ≈ (t1 - τ) / T exp .

[0050] If the transition direction is "on → off" (1→0), it means that the LED is on in the first half of the exposure window and goes off after the transition time τ. Therefore, the acquired grayscale value f is mainly contributed by the first half of the exposure window [t0, τ]. The following approximate relationship can be established: f ≈ (τ - t0) / T exp .

[0051] 6. High-precision inverse kinematics: In the above relationship, the absolute time of the jump is τ = (N+1) * T step It is known that the camera's exposure time T step This is also a known value preset as a shooting parameter, while the grayscale value f is measured from the image. Therefore, the start time t0 or end time t1 of the exposure can be solved from the above formula. For example, in the case of "off → on", t1 ≈ τ + f * T can be solved. step Therefore, we get t0 = t1 - T step Once high-precision t0 and t1 are obtained, the more precise exposure midpoint time (t0 + t1) / 2 can be calculated.

[0052] The sub-step interpolation method in this embodiment achieves even when using an coded step period T. step When using a 1μs time cursor device, the measurement resolution of exposure time can be effectively improved to the order of tens of nanoseconds, such as 20-50 nanoseconds, with the specific accuracy depending on the signal-to-noise ratio and linearity of the camera sensor. Applying this high-precision exposure time measurement result to the closed-loop calibration process described in Example 1 enables the final synchronization error of the multi-camera system to converge to the nanosecond level, thus meeting the more stringent requirements of high-precision synchronization applications.

[0053] Example 3 This embodiment illustrates how the technical solution of this application can be applied to a multi-camera system including a rolling shutter camera. Unlike a global shutter camera, which exposes all pixels simultaneously, a rolling shutter camera exposes and reads out line by line. This results in different lines of the same frame actually recording scene information at different times. This embodiment utilizes this characteristic to not only achieve synchronization between the rolling shutter camera and other cameras, but also to finely characterize its internal timing characteristics.

[0054] The system hardware structure is the same as in Embodiment 1. At least one or all of the cameras to be calibrated are rolling shutter cameras. The innovation of this embodiment also lies in the specific processing algorithm of the decoding and analysis module in the processing unit.

[0055] Reference Figure 8 This image illustrates the imaging effect of a rolling shutter camera. When a rolling shutter camera photographs a rapidly changing time-marker device, because the exposure process proceeds line by line along the scanning direction (indicated by a vertical downward arrow in the image), the pattern of the time-marker device in the image will exhibit a special "coded stripe" or "diagonal gradient" effect. Specifically, the line-first exposure in the top area of ​​the image records an earlier time code; while the line-later exposure in the bottom area of ​​the image records a later time code. For example... Figure 8 As shown, decoding the top region of the image may yield a time t. top Decode the central region of the image to obtain time t. mid Decode the bottom region of the image to obtain time t. bottom Due to the scanning delay, these moments satisfy t. top <t mid <t bottom .

[0056] In this embodiment, the decoding and analysis module is configured to perform the following special analysis process for rolling shutter images: 1. Image acquisition and positioning: Similar to Example 1, firstly, the image captured by the rolling shutter camera is acquired, and the imaging area of ​​the time cursor device is located in the image.

[0057] 2. Slicing Analysis Along the Scanning Direction: The algorithm slices the striped image of the time cursor along the camera's rolling scan direction (usually vertical, but this depends on the specific camera model). This slicing can be done row by row or grouped by several rows.

[0058] 3. Independent Decoding of Each Slice: For each slice (representing a row or a row region in the image), the algorithm independently executes the Gray code decoding process described in Example 1. That is, it identifies the on / off mode of the LEDs within the slice, decodes the corresponding Gray code value, and converts it into a time value.

[0059] 4. Generate a "row number-exposure time" mapping sequence: By decoding all rows or row groups, a mapping sequence will be obtained, in the form of: (row1, t1), (row2, t2), …, (row M , tM), where row i This is the row number, and ti is the exposure time corresponding to that row. This sequence completely describes the exposure time distribution within this frame of the image.

[0060] 5. Characteristic Parameter Calculation: Based on this "row number-exposure time" mapping sequence, the decoding and analysis module can further calculate several key timing characteristic parameters of the rolling shutter camera, including but not limited to: Frame start exposure time: the first time value t1 in the sequence.

[0061] End of frame exposure time: the last time value tM in the sequence.

[0062] Total scan time: tM - t1.

[0063] Interline scan delay (or line scan rate): The precise time interval between two line exposures can be obtained by calculating the time difference (t(i+1) - ti) between adjacent lines and averaging it.

[0064] In scenarios involving multi-camera simultaneous calibration, this embodiment provides a more flexible synchronization strategy. For example, the exposure center times of all cameras (whether global shutter or rolling shutter) can be aligned. For rolling shutter cameras, their exposure center time can be defined as the exposure time t of their center row (e.g., the central region of the image). mid The calibration control module can calculate the synchronization error based on the exposure center time of each camera and perform closed-loop compensation, ultimately ensuring that the center lines of all cameras are exposed at the same time.

[0065] Therefore, the technical solution of this embodiment not only enables high-precision synchronization between the rolling shutter camera and other cameras in the system, but also serves as a diagnostic tool, providing an additional detailed report of the camera's internal timing characteristics. This data, such as precise line scan rates, is crucial calibration input for advanced vision applications requiring line-by-line accuracy (e.g., distortion-free 3D reconstruction of high-speed moving objects).

[0066] Example 4 This embodiment demonstrates how the calibration system proposed in this application can adapt to the synchronous calibration requirements of ultra-high-speed cameras (e.g., exposure time less than 1 microsecond, frame rate up to hundreds of thousands or even millions of frames per second) through parameter configuration and hardware selection, reflecting the excellent scalability of this technical solution.

[0067] For ultra-high-speed cameras, the exposure time T exp Very short. According to the constraint T mentioned in Example 1... step ≥T exp In order to effectively calibrate this type of camera, the encoding step period T of the time cursor device is... step It must be shortened accordingly.

[0068] The system structure of this embodiment is basically the same as that of Embodiment 1, but its key time-related components are configured with high speed: 1. High-speed time cursor device: To achieve a shorter encoding step cycle T step To shorten the time cursor to 100 nanoseconds, for example, a high-speed design is required for the time cursor device. This typically necessitates the use of dedicated LEDs with high-speed switching characteristics, coupled with constant-current drive circuitry capable of providing fast rise and fall edge currents. Simultaneously, the logic for generating the drive signal—Gray code counting and encoding—must be implemented using a field-programmable gate array (FPGA) with precise timing control capabilities, rather than a standard microcontroller. Correspondingly, a higher frequency, such as 10MHz, is required for the reference clock to generate a 100ns step cycle.

[0069] 2. High-precision synchronous triggering device: Corresponding to the time cursor device, the delay compensation capability of the synchronous triggering device also needs to be improved. The delay step accuracy of its programmable delay channel should be consistent with T. step Or even higher precision, such as reaching the nanosecond level. This also requires the use of digital delay lines or high-frequency counters within the field-programmable gate array (FPGA) to ensure fine-tuning of the trigger signal.

[0070] In this embodiment, the entire closed-loop calibration process is exactly the same as that described in Embodiment 1, that is, it follows... Figure 5 The diagram illustrates a closed-loop logic of "trigger-acquisition-decoding-compensation-iteration". However, all time-related operations are performed on a finer time scale.

[0071] For example, when the decoding analysis module decodes the Gray code count value from an image captured by a high-speed camera, it multiplies the count value by a new, smaller encoding step period T. step (e.g., 100ns) to obtain the exposure time. Because T step=100ns, the system can directly perform effective exposure time measurement and synchronous calibration of ultra-high speed cameras with exposure times ranging from 100ns to hundreds of nanoseconds.

[0072] Furthermore, the high-speed configuration of this embodiment can be perfectly combined with the sub-stepping interpolation technique of Embodiment 2. When using a time cursor with a 100ns step period, by analyzing the grayscale information of the switching LEDs, the effective time measurement resolution can be further improved by an order of magnitude, reaching the level of 1-5 nanoseconds.

[0073] Using a 32-bit Gray code counter and a 100ns step period, the non-repeating range of the time cursor device (i.e., the time it takes for the counter to overflow and return to zero) is 2^32 * 100ns ≈ 429.5 seconds, which is approximately 7.1 minutes. This range is more than sufficient for automated calibration processes that can be completed in a few seconds to tens of seconds.

[0074] In summary, this embodiment demonstrates that the technical solution proposed in this application is not only applicable to conventional industrial cameras, but can also be seamlessly extended to the field of ultra-high-speed cameras with extremely demanding synchronization requirements, by configuring the time cursor device and the synchronization trigger device at high speed. This shows its strong vitality and wide applicability as a basic calibration tool.

[0075] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0076] In the description of this application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0077] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A multi-camera synchronization performance detection and calibration system, characterized in that, include: A time cursor device, the visual appearance of which is driven by a reference clock to change over time according to a predetermined encoding rule; The synchronous triggering device is configured to send trigger signals to multiple cameras respectively, and the synchronous triggering device provides each of the multiple cameras with an independent programmable trigger delay channel. The processing unit is configured as follows: Acquire images captured by multiple cameras, including the time cursor device; The visual appearance in each image is decoded to determine the exposure midpoint time of the camera corresponding to that image; based on the determined exposure midpoint times of each camera, the synchronization error between the multiple cameras is calculated; Based on the calculated synchronization error, the synchronization triggering device is controlled to adjust the triggering delay of at least one camera to reduce the synchronization error in a closed-loop manner. The time cursor device includes a counter composed of multiple light-emitting diodes, each light-emitting diode corresponding to a binary bit of Gray code; The count value corresponding to the Gray code is obtained by controlling the combination of the on / off states of the multiple light-emitting diodes; The encoding rule of the time cursor device has an encoding step period T. step The encoding stepping period is not less than the maximum exposure time set by the multiple cameras when shooting, and the count increment is crossed at most once within any exposure window; The processing unit is further configured to trigger a sub-step interpolation calculation step when the visual appearance presented in the image corresponds to a state transition intermediate state of the encoded rule, the sub-step interpolation calculation step including: The exposure midpoint time is calculated based on the grayscale value of the light-emitting diodes in the transition state in the visual appearance. Suppose a transition occurs between count values ​​N and N+1, where count values ​​N and N+1 are known and differ by only one bit. The direction of the transition can be determined by comparing the bit values ​​of these two count values ​​at the position of the LED in the transition state. The pixel grayscale value of the LED in the transition state is read and normalized to the [0, 1] interval to obtain the normalized grayscale value f. This normalized grayscale value f is proportional to the total exposure time T. exp The percentage of time that the LED in the transition state is in the "on" state; Based on the determined jump direction, different interpolation formulas are applied to calculate the position of the exposure window relative to the absolute moment τ at which the jump occurs: If the transition direction is "off → on", the collected normalized gray value f is mainly contributed by the latter half of the exposure window [τ, t1], and the following approximate relationship is established: f ≈ (t1 - τ) / T exp ; If the transition direction is "on → off", the collected normalized grayscale value f is mainly contributed by the first half of the exposure window [t0, τ], and the following approximate relationship is established: f ≈ (τ - t0) / T exp ; The absolute time of the transition τ = (N+1) T step It is known that the camera's exposure time T exp It is also a known value that is preset as the shooting parameter, and t1-t0=T exp The grayscale value f is measured from the image. Based on the above approximation relationship, the start time t0 or end time t1 of the exposure is solved, and the midpoint time of the exposure (t0 + t1) / 2 is calculated.

2. The multi-camera synchronization performance detection and calibration system according to claim 1, characterized in that, The processing unit is further configured to: When at least one of the plurality of cameras is a rolling shutter camera, for each of the at least one rolling shutter camera, the line exposure time of the rolling shutter camera is determined by analyzing the changes in the visual appearance presented along the rolling shutter scanning direction in a single frame of the image captured by it, and the line scan rate of the rolling shutter camera is calculated based on the line exposure time.

3. A method for detecting and calibrating the synchronization performance of multiple cameras, characterized in that, include: A time cursor device is provided, the visual appearance of which is driven by a reference clock to change over time according to a predetermined encoding rule; A synchronous triggering device capable of providing an independent programmable trigger delay for each of the multiple cameras sends trigger signals to the multiple cameras to capture the time cursor device; Acquire images captured by the multiple cameras; The visual appearance in each of the images is decoded to determine the exposure midpoint time of the camera corresponding to that image; Based on the determined exposure midpoint time of each camera, the synchronization error between the multiple cameras is calculated; Based on the calculated synchronization error, the triggering delay of the synchronization triggering device for at least one camera is adjusted in a closed-loop manner to reduce the synchronization error; The time cursor device includes a counter composed of multiple light-emitting diodes, each light-emitting diode corresponding to a binary bit of Gray code; The count value corresponding to the Gray code is obtained by controlling the combination of the on / off states of the multiple light-emitting diodes; The encoding rule of the time cursor device has an encoding step period T. step The encoding stepping period is not less than the maximum exposure time set by the multiple cameras when shooting, and the count increment is crossed at most once within any exposure window; It also includes a sub-step interpolation calculation step when the visual appearance presented in the image corresponds to a state transition intermediate state of the encoding rule, the sub-step interpolation calculation step including: The exposure midpoint time is calculated based on the grayscale value of the light-emitting diodes in the transition state in the visual appearance. Suppose a transition occurs between count values ​​N and N+1, where count values ​​N and N+1 are known and differ by only one bit. The direction of the transition can be determined by comparing the bit values ​​of these two count values ​​at the position of the LED in the transition state. The pixel grayscale value of the LED in the transition state is read and normalized to the [0, 1] interval to obtain the normalized grayscale value f. This normalized grayscale value f is proportional to the total exposure time T. exp Within this timeframe, the percentage of time that the LED in the transition state is in the "on" state; Based on the determined jump direction, different interpolation formulas are applied to calculate the position of the exposure window relative to the absolute moment τ at which the jump occurs: If the transition direction is "off → on", the collected normalized gray value f is mainly contributed by the latter half of the exposure window [τ, t1], and the following approximate relationship is established: f ≈ (t1 - τ) / T exp ; If the transition direction is "on → off", the collected normalized grayscale value f is mainly contributed by the first half of the exposure window [t0, τ], and the following approximate relationship is established: f ≈ (τ - t0) / T exp ; The absolute time of the transition τ = (N+1) T step It is known that the camera's exposure time T exp It is also a known value that is preset as the shooting parameter, and t1-t0=T exp The grayscale value f is measured from the image. Based on the above approximation relationship, the start time t0 or end time t1 of the exposure is solved, and the midpoint time of the exposure (t0 + t1) / 2 is calculated.

4. The method for detecting and calibrating multi-camera synchronization performance according to claim 3, characterized in that, When at least one of the plurality of cameras is a rolling shutter camera, for each of the at least one rolling shutter camera, the line exposure time of the rolling shutter camera is determined by analyzing the changes in the visual appearance presented along the rolling shutter scanning direction in a single frame of the image captured by it, and the line scan rate of the rolling shutter camera is calculated based on the line exposure time.

Images