Multi-camera time synchronization method and apparatus, computer device, and storage medium

Abstract

The application relates to the technical field of image acquisition, and discloses a multi-camera time synchronization method and device, computer equipment and a storage medium, the method comprising the following steps: acquiring a first image and a second image, the first image being obtained by a first camera performing image acquisition on a target screen, the second image being obtained by a second camera performing image acquisition on the target screen, the target screen being used for sequentially displaying different events, the target screen comprising a first region and a second region, and the gray scale of the image displayed by the first region being different from that of the image displayed by the second region in adjacent events; determining to-be-recognized regions corresponding to the first region and the second region in the first image and the second image; calculating a clock offset of the first camera and the second camera according to the gray scale values of the to-be-recognized regions; and debugging the first camera and / or the second camera according to the clock offset, so that the shooting time of the first camera and the second camera is synchronized. According to the application, the cost of multi-camera time synchronization is reduced.

Description

Technical Field

[0001] This invention relates to the field of image acquisition technology, and more specifically to a method, apparatus, computer equipment, and storage medium for multi-camera time synchronization. Background Technology

[0002] When acquiring image data, multiple cameras are typically used simultaneously to improve efficiency. However, different cameras have different hardware system clocks (referred to as camera time), necessitating measures to standardize their time. Current methods for camera time synchronization commonly employ a "high-frequency event + short camera exposure" approach. Multiple cameras capture the same high-frequency event during a short exposure, and this moment is considered the same time reference for all cameras, with an accuracy equal to the reciprocal of the frequency. The higher the event frequency, the more accurate the synchronization. Short exposures are used to ensure that each shot captures only a single event, avoiding skipping between consecutive events. In reality, generating high-frequency events is difficult; for example, high-frequency displays are expensive and often lack sufficient frequency, and high-frequency LED circuits are also costly. Summary of the Invention

[0003] In view of this, the present invention provides a method, apparatus, computer device and storage medium for multi-camera time synchronization, so as to solve the problem of high cost when synchronizing multiple cameras in the prior art.

[0004] In a first aspect, the present invention provides a multi-camera time synchronization method, the method comprising: acquiring a first image and a second image, wherein the first image is obtained by a first camera capturing an image of a target screen, and the second image is obtained by a second camera capturing an image of the target screen, the target screen being used to sequentially display different events, the target screen including a first region and a second region, wherein in adjacent events, the first region and the second region alternately display a first preset image and a second preset image, the order in which the first region and the second region display the first preset image and the second preset image is reversed, and the grayscale values ​​of the first preset image and the second preset image are different; determining a first region to be identified in the first image corresponding to the first region, and a second region to be identified in the second region; determining a third region to be identified in the second image corresponding to the first region, and a fourth region to be identified in the second image; calculating a clock offset between the first camera and the second camera based on a first grayscale value of the first region to be identified, a second grayscale value of the second region to be identified, a third grayscale value of the third region to be identified, and a fourth grayscale value of the fourth region to be identified; and adjusting the first camera and / or the second camera according to the clock offset to synchronize the shooting time of the first camera and the second camera.

[0005] The method provided in this invention involves a first camera and a second camera acquiring images of a target screen to obtain a first image and a second image. Since the target screen sequentially displays different events, and in different events, the first and second regions of the target screen alternately display a first preset image and a second preset image with different grayscale values, even if ghosting occurs in the images acquired by the first and second cameras when the exposure time of the first and second cameras is long and the target screen is a low refresh rate display, the time of acquiring the first image and the time of acquiring the second image can be determined based on the grayscale values ​​of each region to be identified in the first image and the second image. This allows for the determination of the clock deviation between the first and second cameras, and further time synchronization between the first and second cameras. This method does not require the cameras to have network interfaces or standard electrical interfaces, is applicable to all cameras, and does not require the generation of high-frequency events to achieve accurate time synchronization of multiple cameras, thus reducing costs.

[0006] In some optional implementations, the step of calculating the clock offset of the first camera and the second camera based on the gray values ​​of the first region to be identified, the second region to be identified, the third region to be identified, and the fourth region to be identified includes: calculating a first exposure time of a first event in the first image based on the first gray value and the second gray value; calculating a first start exposure time of the first camera when capturing the first image based on a first preset end time and the first exposure time of the first event; calculating a second exposure time of a second event in the second image based on the third gray value and the fourth gray value; calculating a second start exposure time of the second camera when capturing the second image based on a second preset end time and the second exposure time of the second event; and determining the clock offset of the first camera and the second camera based on the difference between the first start exposure time and the second start exposure time.

[0007] In some optional implementations, the step of calculating the first exposure time of the first event in the first image based on the first gray value and the second gray value includes: calculating the proportion of the first exposure time of the first event in the first image based on the proportion of the first gray value in the total gray value, wherein the total gray value is determined based on the sum of the first gray value and the second gray value; and calculating the first exposure time based on the proportion of the first exposure time and the total first exposure time, wherein the total first exposure time is the exposure time when the first camera acquires the first image.

[0008] In some optional implementations, the step of calculating the second exposure time of the second event in the second image based on the third gray value and the fourth gray value includes: calculating the proportion of the second exposure time of the second event in the second image based on the proportion of the third gray value and the total gray value, wherein the total gray value is determined based on the sum of the third gray value and the fourth gray value; and calculating the second exposure time based on the proportion of the second exposure time and the total exposure time, wherein the total exposure time is the exposure time when the second camera acquires the second image.

[0009] In some optional implementations, the target screen further includes a third region, the grayscale of which remains consistent across all events displayed on the target screen. The grayscale value of each region to be identified is calculated through the following steps: determining a first correction region corresponding to the third region in a first image; determining a first corrected grayscale value for the first correction region based on the first image; determining a first grayscale value based on the difference between the actual grayscale value of the first region to be identified and the first corrected grayscale value; determining a second grayscale value based on the difference between the actual grayscale value of the second region to be identified and the first corrected grayscale value; determining a second correction region corresponding to the third region in a second image; determining a second corrected grayscale value for the second correction region based on the second image; determining a third grayscale value based on the difference between the actual grayscale value of the third region to be identified and the second corrected grayscale value; and determining a fourth grayscale value based on the difference between the actual grayscale value of the fourth region to be identified and the second corrected grayscale value.

[0010] In practical applications, factors such as ambient backlight, screen light leakage, and camera background noise can cause differences in the grayscale of different areas in the images captured by the first and second cameras compared to the grayscale of different areas displayed on the target screen. Therefore, in order to improve accuracy, a third area is set in the target screen in this embodiment of the invention. The grayscale value of each area to be identified is verified by the grayscale value of the third area, thereby making the clock deviation calculated based on the grayscale value more accurate.

[0011] In some alternative implementations, when the target screen displays different events, the exposure time for each event is the same; the total first exposure time when the first camera captures the first image is the same as the exposure time for each event; and the total second exposure time when the second camera captures the second image is the same as the exposure time for each event.

[0012] In some alternative implementations, the first exposure time is calculated using the following formula:

[0013]

[0014] in, This indicates the first preset end time, and n1 represents the sequence number of the first event. Indicates the exposure time of each event. The camera exposure time is represented by N1 = N2, G1 represents the actual gray value of the first region to be identified, G2 represents the first corrected gray value, and G3 represents the actual gray value of the second region to be identified.

[0015] Secondly, the present invention provides a multi-camera time synchronization device, comprising: an image acquisition module for acquiring a first image and a second image, wherein the first image is obtained by the first camera capturing an image of a target screen, and the second image is obtained by the second camera capturing an image of the target screen, the target screen being used to sequentially display different events, the target screen including a first region and a second region, wherein in adjacent events, the first region and the second region alternately display a first preset image and a second preset image, the order in which the first region and the second region display the first preset image and the second preset image are reversed, and the grayscale of the first preset image and the second preset image are different; and a first image recognition module for determining the first image. The image includes a first region to be identified corresponding to a first region, and a second region to be identified corresponding to a second region; a second image recognition module for determining a third region to be identified corresponding to the first region and a fourth region to be identified corresponding to the second region in the second image; a clock offset calculation module for calculating the clock offset of the first camera and the second camera based on the first gray value of the first region to be identified, the second gray value of the second region to be identified, the third gray value of the third region to be identified, and the fourth gray value of the fourth region to be identified; and a debugging module for debugging the first camera and / or the second camera based on the clock offset to synchronize the shooting time of the first camera and the second camera.

[0016] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the multi-camera time synchronization method described in the first aspect or any corresponding embodiment thereof.

[0017] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the multi-camera time synchronization method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1This is a schematic diagram of the square wave signal sent by the signal generator in the hardware synchronization signal method;

[0020] Figure 2 This is a flowchart illustrating a multi-camera time synchronization method according to an embodiment of the present invention;

[0021] Figure 3 This is an event displayed on the target screen according to an embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram showing the overlap between a first preset image and a second preset image in the first region to be identified according to an embodiment of the present invention;

[0023] Figure 5 It is a target screen comprising four regions according to an embodiment of the present invention;

[0024] Figure 6 These are images of the target screen displaying 1 frame and 2 frames according to an embodiment of the present invention;

[0025] Figure 7 The image is taken when the camera exposure time is 10ms according to an embodiment of the present invention, and exactly between 0 and 10ms.

[0026] Figure 8 The image is captured by a camera with an exposure time of 10ms and a span of 5 to 15ms according to an embodiment of the present invention.

[0027] Figure 9 The image is captured by the camera during an exposure of 10ms, spanning from 7.55ms to 17.55ms, according to an embodiment of the present invention.

[0028] Figure 10 This is a schematic diagram illustrating the principle of calculating the camera's start exposure time according to an embodiment of the present invention;

[0029] Figure 11 This is a structural block diagram of a multi-camera time synchronization device according to an embodiment of the present invention;

[0030] Figure 12 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] In order to enable multiple cameras to shoot simultaneously, it is necessary to synchronize the time of multiple cameras. In related technologies, the methods commonly used for time synchronization of multiple cameras include: (1) network time synchronization method, (2) hardware synchronization signal method, and (3) event synchronization method.

[0033] The steps of the network time synchronization method include:

[0034] (1) The network switch acts as a unified clock server and connects to the terminal (the device to be synchronized with the time, such as a camera system) via a network cable;

[0035] (2) The terminal sends data to the server and records the terminal's time T1, which needs to be corrected.

[0036] (3) After receiving the time code, the server records the receiving time T2, which is the standard time;

[0037] (4) The server sends data to the terminal and records the sending time T3, which is the standard time.

[0038] (5) The terminal receives data from the server and records the reception time T4. This time needs to be corrected.

[0039] (6) The one-way time of the above process network transmission is △T=[(T4-T1)-(T3-T2)] / 2. This time is the standard time difference between T2 and T1. Then the standard time difference between the terminal and the server is T2-△T-T1, which is the correction amount of the terminal time relative to the standard time.

[0040] (7) Once different terminals have completed the above steps, the terminal time can be corrected to the server's standard time.

[0041] When synchronizing the time of each camera using the network time synchronization method, the timing accuracy is poor due to the large latency and susceptibility to interference of the wireless network. Therefore, it is necessary to connect the clock server to each camera via a network cable. This method requires the camera to support a wired network interface, which has poor versatility.

[0042] The steps of the hardware synchronization signal method include:

[0043] (1) Using the same signal generator, each camera is connected via a coaxial cable, and a square wave signal is sent to each camera. For example, the square wave signal is as follows: Figure 1 As shown;

[0044] (2) Different cameras start exposure at the same time when they receive the rising edge of the square wave voltage.

[0045] This method requires cameras to use a standard electrical interface; cameras without this standard interface cannot use this method.

[0046] Event synchronization methods include LED scrolling light solutions and high refresh rate display solutions.

[0047] The LED running light solution specifically includes:

[0048] (1) Design two 10*10 LED arrays, and light up each LED in the first array sequentially at an interval of 0.1ms;

[0049] (2) After the last LED in array 1 is lit, the first LED in another array is lit, which means 10ms have passed. At the same time, array 1 starts lighting up the first LED again from the beginning.

[0050] (3) Repeat the above steps. After each LED in array 1 is lit, light up the next LED in array 2.

[0051] (4) Different cameras use short exposure (<0.05ms) to photograph the above LED array. If camera A photographs the LED array 1 at position n1 and the LED array 2 at position n2, the corresponding time is T1 = n2 * 10ms + n1 * 0.1ms; if camera B photographs the LED array 1 at position n3 and the LED array 2 at position n4, the corresponding time is T2 = n4 * 10ms + n3 * 0.1ms.

[0052] (5) △T=T2-T1 is the clock offset of camera B relative to camera A.

[0053] LED running light solutions require specialized LED hardware, and the cost of custom-made equipment is relatively high.

[0054] High refresh rate display solutions specifically include:

[0055] (1) Use a 500Hz high refresh rate display, display different numbers in each frame, and the numbers are continuously accumulated;

[0056] (2) Different cameras use short exposure (<0.05ms) to photograph the above display. If camera A photographs the display and the number is n1, T1 = n1 * 2ms; if camera B photographs the display and the number is n2, T2 = n2 * 2ms.

[0057] (3) △T=T2-T1 is the clock offset of camera B relative to camera A.

[0058] The drawbacks of high refresh rate display solutions are that high refresh rate screens are expensive and the time synchronization accuracy can only reach 2ms.

[0059] To address the shortcomings of existing technologies that require wired network interfaces or standard electrical interfaces for time synchronization of multiple cameras, and to overcome the high costs associated with this approach, this invention proposes a "low-frequency event + long exposure" method for time synchronization. Using this method, such as screen refresh events with a refresh rate N of 60Hz or 120Hz, the display refreshes a different number each frame. Based on the numbers captured by different cameras, the time synchronization accuracy of different cameras can be controlled to 1 / N seconds. However, due to the use of long exposure, if two consecutive events are captured during the long exposure, two numbers from the two screens are exposed, resulting in ghosting in the photograph. While this reduces costs, it is not feasible to achieve true time synchronization of multiple cameras using existing technologies.

[0060] According to an embodiment of the present invention, a multi-camera time synchronization method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0061] This embodiment provides a multi-camera time synchronization method. Figure 2 This is a flowchart of a multi-camera time synchronization method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0062] Step S101: Acquire a first image and a second image. The first image is obtained by the first camera capturing an image of the target screen, and the second image is obtained by the second camera capturing an image of the target screen. The target screen is used to sequentially display different events. The target screen includes a first area and a second area. In adjacent events, the first area and the second area alternately display a first preset image and a second preset image. The order in which the first area and the second area display the first preset image and the second preset image is reversed, and the grayscale of the first preset image and the second preset image is different.

[0063] In this embodiment of the invention, the target screen can be a conventional low-frequency display, and the duration of a single frame of the target screen can be 1 / 60 second, 1 / 120 second, etc.

[0064] In this embodiment of the invention, the order in which the first and second regions display the first and second preset images is reversed. That is, when the first region displays the first preset image, the second region displays the second preset image, and vice versa. If in the first event, the first region displays the first preset image and the second region displays the second preset image, then in the second event, the first region displays the second preset image and the second region displays the first preset image; in the third event, the first region displays the first preset image and the second region displays the second preset image. The next event after the first event is the second event, the next event after the second event is the third event, and so on, with the target screen sequentially displaying different events.

[0065] In one optional embodiment, the first preset image can be a black rectangle, and the second preset image can be a white rectangle, with the black rectangle and the white rectangle having different gray levels.

[0066] In one optional embodiment, a first region alternately displays black and white rectangles, and a second region also alternately displays black and white rectangles. Within the same event, the images displayed in the first and second regions are different. For example, as shown... Figure 3 As shown, when the first area is a black rectangle, the second area is a white rectangle.

[0067] Step S102: Determine the first region to be identified in the first image corresponding to the first region, and the second region to be identified corresponding to the second region.

[0068] In an alternative embodiment, the first region to be identified and the second region to be identified can be determined in the first image based on the relative positions of the first region and the second region in the target screen.

[0069] Step S103: Determine the third region to be identified in the second image that corresponds to the first region, and the fourth region to be identified that corresponds to the second region.

[0070] In an alternative embodiment, a third and a fourth region to be identified can be determined in the second image based on the relative positions of the first and second regions in the target screen.

[0071] Step S104: Calculate the clock offset of the first camera and the second camera based on the first gray value of the first region to be identified, the second gray value of the second region to be identified, the third gray value of the third region to be identified, and the fourth gray value of the fourth region to be identified.

[0072] In this embodiment of the invention, when the first camera acquires the first image, due to the long exposure time, the exposure time may span two events displayed on the target screen. In this case, because the preceding and following events overlap in the first image, the first grayscale value of the first region to be identified is neither equal to the grayscale value of the first preset image nor equal to the grayscale value of the second preset image. Figure 4 As shown. If the subsequent event has a short occurrence time, the image displayed in the first region after the subsequent event has a smaller impact on the image displayed in the first region after the preceding event. That is, the grayscale value of the image displayed in the first region after the subsequent event has a smaller impact on the grayscale value of the image displayed in the first region after the preceding event. Conversely, if the subsequent event has a long occurrence time, the grayscale value of the image displayed in the first region after the subsequent event has a larger impact on the grayscale value of the image displayed in the first region after the preceding event. Similarly, the occurrence of the subsequent event will also have a certain impact on the grayscale value of the image displayed in the second region after the preceding event. Therefore, in this embodiment of the invention, the exposure duration of each of the two events during the exposure period of the first camera can be determined based on the first grayscale value of the first region to be identified and the second grayscale value of the second region to be identified. Thus, combined with the pre-set occurrence time of each event, the start time of the first camera exposure can be accurately obtained.

[0073] Similarly, in this embodiment of the invention, the exposure duration of the first event and the second event during the exposure of the second camera can be determined based on the third gray value of the third region to be identified and the fourth gray value of the fourth region to be identified, thereby accurately obtaining the time when the second camera begins to expose.

[0074] Furthermore, the clock offset between the first camera and the second camera can be determined based on the start time of the first camera's exposure and the start time of the second camera's exposure.

[0075] Step S105: Adjust the first camera and / or the second camera according to the clock offset to synchronize the shooting time of the first camera and the second camera.

[0076] The above steps are only an example of time synchronization between two cameras to explain the method provided by the present invention. In practical applications, the method provided by the present invention can be used to synchronize the time of multiple cameras.

[0077] The method provided in this invention involves a first camera and a second camera acquiring images of a target screen to obtain a first image and a second image. Since the target screen sequentially displays different events, and in different events, the first and second regions of the target screen alternately display a first preset image and a second preset image with different grayscale values, even if ghosting occurs in the images acquired by the first and second cameras when the exposure time of the first and second cameras is long and the target screen is a low refresh rate display, the time of acquiring the first image and the time of acquiring the second image can be determined based on the grayscale values ​​of each region to be identified in the first image and the second image. This allows for the determination of the clock deviation between the first and second cameras, and further time synchronization between the first and second cameras. This method does not require the cameras to have network interfaces or standard electrical interfaces, is applicable to all cameras, and does not require the generation of high-frequency events to achieve accurate time synchronization of multiple cameras, thus reducing costs.

[0078] In an optional embodiment, step S104 specifically includes the following steps:

[0079] Step a1: Calculate the first exposure time of the first event in the first image based on the first grayscale value and the second grayscale value.

[0080] In an optional embodiment, when calculating the first exposure time of the first event in the first image, firstly, the proportion of the first exposure time of the first event in the first image is calculated based on the proportion of the first gray value in the total gray value, wherein the total gray value is determined based on the sum of the first gray value and the second gray value; then, the first exposure time is calculated based on the proportion of the first exposure time and the total first exposure time, wherein the total first exposure time is the exposure time when the first camera acquires the first image.

[0081] In this embodiment of the invention, when the first image spans two events, the first event is the event that appears first in the first image.

[0082] In this embodiment of the invention, during the camera exposure time, the longer the first event occurs, the closer the first gray value is to the gray value in the first region of the first event; the shorter the first event occurs, the closer the first gray value is to the gray value in the first region of the next event. Therefore, the first exposure time of the first event in the first image can be determined based on the proportion of the first gray value in the total gray value.

[0083] Step a2: Calculate the first start exposure time when the first camera captures the first image based on the first preset end time and the first exposure time of the first event.

[0084] In one optional embodiment, the target screen displays each event sequentially, and the preset end time of the first event captured in the first image can be determined based on the duration of each event and the start time of the first event.

[0085] In an optional embodiment, the first start exposure time is the difference between the preset end time of the first event and the first exposure time.

[0086] Step a3: Based on the third and fourth grayscale values, calculate the second exposure time of the second event in the second image. The method for calculating the second exposure time is the same as that for calculating the first exposure time in step a1, and will not be repeated here.

[0087] In an optional embodiment, the second event and the first event can be the same event, that is, the two events spanned by the images captured by the first camera and the second camera are the same.

[0088] Step a4: Calculate the second start exposure time when the second camera captures the second image based on the second preset end time and the second exposure time of the second event.

[0089] Step a5: Determine the clock offset of the first camera and the second camera based on the difference between the first start exposure time and the second start exposure time.

[0090] In an optional embodiment, since calculating the start exposure time of each camera requires determining the preset end time of the first event appearing in the image, the target screen can also display the identifiers of each event when displaying the events. When the camera captures an image of the target screen, it will capture the identifier of the current event. Even if the captured image spans multiple events, the identifier of the first event appearing in the image can be determined through the image, thereby determining the preset end time of the first event and further calculating the start exposure time of the camera. For example, numbers "1, 2, 3, ..." can be used as the identifiers of each event, with different events corresponding to different identifiers.

[0091] In an optional embodiment, the target screen further includes a third region, the grayscale of which remains consistent across all events displayed on the target screen. The grayscale value of each region to be identified is calculated through the following steps:

[0092] Step b1: Determine the first correction region corresponding to the third region in the first image, and determine the first correction gray value of the first correction region based on the first image.

[0093] Step b2: Determine the first gray value based on the difference between the actual gray value of the first region to be identified and the first corrected gray value; determine the second gray value based on the difference between the actual gray value of the second region to be identified and the first corrected gray value.

[0094] Step b3: Determine the second correction region in the second image that corresponds to the third region, and determine the second correction gray value of the second correction region based on the second image.

[0095] Step b4: Determine the third gray value based on the difference between the actual gray value of the third region to be identified and the second corrected gray value; determine the fourth gray value based on the difference between the actual gray value of the fourth region to be identified and the second corrected gray value.

[0096] In this embodiment of the invention, although the third region does not change in each event, when the first camera and the second camera acquire images, the grayscale of the image displayed by the third region may be deviated in the first camera and the second camera due to reasons such as the camera background noise and shooting angle of the first camera and the second camera. Therefore, when calculating the grayscale of each region to be identified in the first image and the grayscale of each region to be identified in the second image, it is necessary to calculate the first corrected grayscale value of the third region in the first image and the second corrected grayscale value of the third region in the second image, so as to ensure the accuracy of the grayscale value of each region to be identified.

[0097] In practical applications, factors such as background light, screen light leakage, and camera background noise can cause differences in the grayscale of different areas in the images captured by the first and second cameras compared to the grayscale of different areas displayed on the target screen. Therefore, in order to improve accuracy, a third area is set in the target screen in this embodiment of the invention. The grayscale value of each area to be identified is verified by the grayscale value of the third area, thereby making the clock deviation calculated based on the grayscale value more accurate.

[0098] In one optional embodiment, when the target screen displays different events, the exposure time of each event is the same. The total first exposure time when the first camera captures the first image is the same as the exposure time of each event, and the total second exposure time when the second camera captures the second image is the same as the exposure time of each event.

[0099] For example, the duration of a single frame on a 60Hz display is 1 / 60 of a second, corresponding to a camera exposure time of 1 / 60 of a second; the duration of a single frame on a 120Hz display is 1 / 120 of a second, corresponding to a camera exposure time of 1 / 120 of a second. This ensures that the camera exposure will not span three events, in most cases span two events, and in very rare cases span only one event.

[0100] In an optional embodiment, the first exposure time is calculated using the following formula:

[0101]

[0102] in, This indicates the first preset end time, and n1 represents the sequence number of the first event. Indicates the exposure time of each event. The camera exposure time is represented by N1 = N2, G1 represents the actual gray value of the first region to be identified, G2 represents the first corrected gray value, and G3 represents the actual gray value of the second region to be identified.

[0103] The formula for calculating the second exposure time is the same as that for the first exposure time, and will not be repeated here.

[0104] As a specific application embodiment of the multi-camera time synchronization method provided in the above embodiments, for a target screen with 100FPS, the display time of each frame is 10ms, such as... Figure 5 As shown, the target screen includes four regions:

[0105] (1) Area A is displayed as alternating white and pure black rectangles;

[0106] (2) Area B, continuously displays pure black;

[0107] (3) Area C is displayed as alternating black and white rectangles;

[0108] (4) The number display area is incremented sequentially from 1, 2, 3 to ensure that the numbers displayed in each frame are different.

[0109] In this embodiment, the camera's exposure time is precisely set to 10ms. In rare cases, the captured image may be perfectly synchronized with the displayed image, resulting in a single digit in the captured image. In most cases, the 10ms exposure time will span two frames, displaying the digits from both frames simultaneously.

[0110] exist Figure 6 In the diagram, region A displays frame 1 in white (0-10ms) and frames 2 in black (10-20ms), while region C displays frame 1 in black (0-10ms) and frames 2 in white (10-20ms).

[0111] If the camera's exposure time is 10ms, and exactly between 0 and 10ms, then only frame 1 of the image can be captured, such as... Figure 7 As shown.

[0112] If the camera exposure is 10ms, with a time interval of 5-15ms, then frame 1 will be captured in exactly 5ms, and frame 2 in exactly 5ms. Furthermore, the brightness of regions A and C in the resulting photograph will be the same. Figure 8 As shown.

[0113] If the camera exposure is 10ms, spanning 7.55 to 17.55ms, then frame 1 will be captured in exactly 2.45ms, and frame 2 in exactly 7.55ms. In the resulting image, area A will be darker, and area C will be brighter, with a grayscale ratio of 2.45:7.55. Figure 9 As shown.

[0114] Therefore, by using the grayscale ratio of region A and region C, the precise time at which the camera begins exposure can be calculated.

[0115] For example, if the average grayscale of region A is 78.54 (since it is the average of tens of thousands of pixels in the entire region, the average grayscale can have a decimal point), and the average grayscale of region C is 187.32, then this means that the exposure time of frame 1 is shorter and the exposure time of frame 2 is longer, with a ratio of 78.54:187.32. This is better than the total exposure time of 10ms. Therefore, it is easy to calculate that the exposure time of frame 1 is 2.954ms and the exposure time of frame 2 is 7.046ms. The camera's exposure range is 7.046 to 17.046ms, with an error of less than 1ms.

[0116] like Figure 10 As shown, to ensure more accurate statistics and eliminate the influence of background noise, the grayscale value G1 of region A in frame 1 cannot be used directly. Instead, G2 is subtracted. Region G2 displays as pure black on the screen, but during photography, factors such as background light, camera noise, and screen light leakage (the screen cannot display pure black) can affect the actual grayscale value of region G2, causing it to be non-zero, for example, 2.337. Therefore, (G1-G2) is used as the grayscale value of region A. Similarly, (G3-G2) is used as the grayscale value of region C.

[0117] The gray level of region A is compared with the gray level of region C, and the ratio is (G1-G2):(G3-G2).

[0118] The total gray level of regions A and C is (G1-G2)+(G3-G2)=G1+G3-2*G2.

[0119] The proportion of gray level in region A to the total gray level above and below is (G1-G2) / (G1+G3-2*G2).

[0120] The camera frame rate and the screen frame rate are the same, both being N. Therefore, the screen display time is 1 / N seconds, and the camera exposure time is also 1 / N seconds. The grayscale ratio of region A represents the proportion of frame 1 in the entire camera exposure time (exposure percentage). So the exposure time of frame 1 is: exposure percentage * total exposure time, i.e.: (G1-G2) / (G1+G3-2*G2)*(1 / N) seconds.

[0121] The exposure time of frame 1 = the end time of frame 1 display - the time when the camera starts exposure, that is: the time when the camera starts exposure = the end time of frame 1 display - the exposure time of frame 1. The end time of frame 1 display = n1*(1 / N) seconds. n1 is the exposure sequence of frame 1 (0, 1, ..., 9527, ..., 9999), then the time when the camera starts exposure T = n1*(1 / N) - (G1-G2) / (G1+G3-2*G2)*(1 / N) = (n1-(G1-G2) / (G1+G3-2*G2)*(1 / N).

[0122] After calculating the exposure times T1 and T2 of the first and second cameras using the above method, ΔT = T2 - T1 represents the clock offset of the second camera relative to the first camera. Time synchronization between the first and second cameras can then be achieved based on this clock offset.

[0123] This embodiment also provides a multi-camera time synchronization device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0124] This embodiment provides a multi-camera time synchronization device, such as... Figure 11 As shown, it includes:

[0125] Image acquisition module 201 is used to acquire a first image and a second image. The first image is obtained by the first camera capturing an image of the target screen, and the second image is obtained by the second camera capturing an image of the target screen. The target screen is used to display different events in sequence. The target screen includes a first area and a second area. In adjacent events, the first area and the second area alternately display a first preset image and a second preset image. The order in which the first area and the second area display the first preset image and the second preset image is opposite, and the grayscale of the first preset image and the second preset image is different.

[0126] The first image recognition module 202 is used to determine a first region to be recognized in the first image corresponding to a first region, and a second region to be recognized corresponding to a second region.

[0127] The second image recognition module 203 is used to determine a third region to be recognized in the second image that corresponds to the first region, and a fourth region to be recognized that corresponds to the second region.

[0128] The clock bias calculation module 204 is used to calculate the clock bias of the first camera and the second camera based on the first gray value of the first region to be identified, the second gray value of the second region to be identified, the third gray value of the third region to be identified, and the fourth gray value of the fourth region to be identified.

[0129] The debugging module 205 is used to debug the first camera and / or the second camera according to the clock bias so that the shooting time of the first camera and the second camera is synchronized.

[0130] In some optional implementations, the clock bias calculation module 204 specifically includes:

[0131] The first exposure time calculation submodule is used to calculate the first exposure time of the first event in the first image based on the first gray value and the second gray value.

[0132] The first start exposure time calculation submodule is used to calculate the first start exposure time when the first camera captures the first image based on the first preset end time and the first exposure time of the first event.

[0133] The second exposure time calculation submodule is used to calculate the second exposure time of the second event in the second image based on the third gray value and the fourth gray value.

[0134] The second start exposure time calculation submodule is used to calculate the second start exposure time when the second camera captures the second image based on the second preset end time and the second exposure time of the second event.

[0135] The clock bias calculation submodule is used to determine the clock bias of the first camera and the second camera based on the difference between the first start exposure time and the second start exposure time.

[0136] In some optional implementations, the first exposure time calculation submodule specifically includes:

[0137] The first exposure time percentage calculation unit is used to calculate the first exposure time percentage of the first event in the first image based on the percentage of the first gray value in the total gray value. The total gray value is determined based on the sum of the first gray value and the second gray value.

[0138] The first exposure time calculation unit is used to calculate the first exposure time based on the proportion of the first exposure time and the total first exposure time. The total first exposure time is the exposure time when the first camera acquires the first image.

[0139] In some optional implementations, the second exposure time calculation submodule specifically includes:

[0140] The second exposure time percentage calculation unit is used to calculate the second exposure time percentage of the second event in the second image based on the percentage of the third gray value and the total gray value. The total gray value is determined based on the sum of the third gray value and the fourth gray value.

[0141] The second exposure time calculation unit is used to calculate the second exposure time based on the proportion of the second exposure time and the total second exposure time. The total second exposure time is the exposure time when the second camera acquires the second image.

[0142] In some optional implementations, the target screen further includes a third region, the grayscale of which remains consistent across all events displayed on the target screen. The clock offset calculation module 204 includes:

[0143] The first correction region calculation submodule is used to determine the first correction region in the first image that corresponds to the third region.

[0144] The first correction grayscale value calculation submodule is used to determine the first correction grayscale value of the first correction region based on the first image;

[0145] The first gray value calculation submodule is used to determine the first gray value based on the difference between the actual gray value of the first region to be identified and the first corrected gray value.

[0146] The second grayscale value calculation submodule is used to determine the second grayscale value based on the difference between the actual grayscale value of the second region to be identified and the first corrected grayscale value.

[0147] The second correction region calculation submodule is used to determine the second correction region in the second image that corresponds to the third region.

[0148] The second correction grayscale value calculation submodule is used to determine the second correction grayscale value of the second correction region based on the second image.

[0149] The third gray value calculation submodule is used to determine the third gray value based on the difference between the actual gray value of the third region to be identified and the second corrected gray value.

[0150] The fourth grayscale value calculation submodule is used to determine the fourth grayscale value based on the difference between the actual grayscale value of the fourth region to be identified and the second corrected grayscale value.

[0151] In some alternative implementations, when the target screen displays different events, the exposure time for each event is the same;

[0152] The total exposure time of the first exposure when the first camera captures the first image is the same as the exposure time of each event;

[0153] The total duration of the second exposure when the second camera acquires the second image is the same as the exposure time of each event.

[0154] In some alternative implementations, the first exposure time is calculated using the following formula:

[0155]

[0156] in, This indicates the first preset end time, and n1 represents the sequence number of the first event. Indicates the exposure time of each event. The camera exposure time is represented by N1 = N2, G1 represents the actual gray value of the first region to be identified, G2 represents the first corrected gray value, and G3 represents the actual gray value of the second region to be identified.

[0157] In this embodiment, the multi-camera time synchronization device is presented in the form of a functional unit. Here, a unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0158] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0159] This invention also provides a computer device having the above-described features. Figure 11 The multi-camera time synchronization device shown.

[0160] Please see Figure 12 , Figure 12 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 12 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 12 Take a processor 10 as an example.

[0161] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0162] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0163] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device as shown by a landing page for an app. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0164] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0165] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 11 Taking the example of a connection between China and Israel via a bus.

[0166] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0167] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0168] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for time synchronization of multiple cameras, characterized in that, The method includes: A first image and a second image are acquired. The first image is obtained by a first camera capturing an image of the target screen, and the second image is obtained by a second camera capturing an image of the target screen. The target screen is used to sequentially display different events. The target screen includes a first area and a second area. In adjacent events, the first area and the second area alternately display a first preset image and a second preset image. The order in which the first preset image and the second preset image are displayed in the first area and the second area are opposite. The grayscale of the first preset image and the second preset image are different. Determine a first region to be identified in the first image that corresponds to the first region, and a second region to be identified that corresponds to the second region; Determine a third region to be identified in the second image that corresponds to the first region, and a fourth region to be identified that corresponds to the second region; The clock offsets of the first camera and the second camera are calculated based on the first gray value of the first region to be identified, the second gray value of the second region to be identified, the third gray value of the third region to be identified, and the fourth gray value of the fourth region to be identified. The first camera and / or the second camera are adjusted according to the clock bias to synchronize the shooting time of the first camera and the second camera; The step of calculating the clock offset of the first camera and the second camera based on the gray values ​​of the first region to be identified, the second region to be identified, the third region to be identified, and the fourth region to be identified includes: Calculate the first exposure time of the first event in the first image based on the first gray value and the second gray value; Calculate the first start exposure time when the first camera captures the first image based on the first preset end time of the first event and the first exposure time; Based on the third grayscale value and the fourth grayscale value, calculate the second exposure time of the second event in the second image; Calculate the second start exposure time when the second camera captures the second image based on the second preset end time of the second event and the second exposure time; The clock offsets of the first camera and the second camera are determined based on the difference between the first start exposure time and the second start exposure time.

2. The method according to claim 1, characterized in that, The step of calculating the first exposure time of the first event in the first image based on the first gray value and the second gray value includes: The proportion of the first exposure time of the first event in the first image is calculated based on the proportion of the first gray value in the total gray value. The total gray value is determined based on the sum of the first gray value and the second gray value. The first exposure time is calculated based on the first exposure time percentage and the first total exposure time, where the first total exposure time is the exposure time when the first camera captures the first image.

3. The method according to claim 1, characterized in that, The step of calculating the second exposure time of the second event in the second image based on the third gray value and the fourth gray value includes: The proportion of the second exposure time of the second event in the second image is calculated based on the ratio of the third gray value to the total gray value, wherein the total gray value is determined based on the sum of the third gray value and the fourth gray value; The second exposure time is calculated based on the second exposure time percentage and the second total exposure time. The second total exposure time is the exposure time when the second camera captures the second image.

4. The method according to claim 2 or 3, characterized in that, The target screen also includes a third region. The grayscale of this third region remains consistent across all events displayed on the target screen. The grayscale value of each region to be identified is calculated using the following steps: In the first image, a first correction region corresponding to the third region is determined; Determine the first corrected gray value of the first corrected region based on the first image; The first gray value is determined based on the difference between the actual gray value of the first region to be identified and the first corrected gray value. The second gray value is determined based on the difference between the actual gray value of the second region to be identified and the first corrected gray value; In the second image, a second correction region corresponding to the third region is determined; Determine the second corrected grayscale value of the second corrected region based on the second image; The third gray value is determined based on the difference between the actual gray value of the third region to be identified and the second corrected gray value; The fourth gray value is determined based on the difference between the actual gray value of the fourth region to be identified and the second corrected gray value.

5. The method according to claim 4, characterized in that, When the target screen displays different events, the exposure time for each event is the same; The total exposure time of the first camera when acquiring the first image is the same as the exposure time of each event; The total exposure time of the second camera when acquiring the second image is the same as the exposure time of each event.

6. The method according to claim 5, characterized in that, The first exposure time is calculated using the following formula: in,( () indicates the first preset end time. Indicates the sequence number of the first event. Indicates the exposure time of each event. Indicates the camera exposure time, where, = , This represents the actual grayscale value of the first region to be identified. This represents the first corrected grayscale value. This represents the actual grayscale value of the second region to be identified.

7. A multi-camera time synchronization device, characterized in that, The device includes: An image acquisition module is used to acquire a first image and a second image. The first image is obtained by a first camera capturing an image of the target screen, and the second image is obtained by a second camera capturing an image of the target screen. The target screen is used to sequentially display different events. The target screen includes a first area and a second area. In adjacent events, the first area and the second area alternately display a first preset image and a second preset image. The order in which the first preset image and the second preset image are displayed in the first area and the second area is reversed. The grayscale of the first preset image and the second preset image is different. The first image recognition module is used to determine a first region to be recognized in the first image corresponding to the first region, and a second region to be recognized corresponding to the second region. The second image recognition module is used to determine a third region to be recognized in the second image that corresponds to the first region, and a fourth region to be recognized that corresponds to the second region. The clock bias calculation module is used to calculate the clock bias of the first camera and the second camera based on the first gray value of the first region to be identified, the second gray value of the second region to be identified, the third gray value of the third region to be identified, and the fourth gray value of the fourth region to be identified. A debugging module is used to debug the first camera and / or the second camera according to the clock bias so that the shooting time of the first camera and the second camera is synchronized. The step of calculating the clock offset of the first camera and the second camera based on the gray values ​​of the first region to be identified, the second region to be identified, the third region to be identified, and the fourth region to be identified includes: Calculate the first exposure time of the first event in the first image based on the first gray value and the second gray value; Calculate the first start exposure time when the first camera captures the first image based on the first preset end time of the first event and the first exposure time; Based on the third grayscale value and the fourth grayscale value, calculate the second exposure time of the second event in the second image; Calculate the second start exposure time when the second camera captures the second image based on the second preset end time of the second event and the second exposure time; The clock offsets of the first camera and the second camera are determined based on the difference between the first start exposure time and the second start exposure time.

8. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the multi-camera time synchronization method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the multi-camera time synchronization method according to any one of claims 1 to 6.

Images