Multi-camera time synchronization method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of image acquisition, and discloses a multi-camera time synchronization method and device, computer equipment and a storage medium, the method comprises the following steps: obtaining a first image and a second image, the first image is obtained by carrying out image acquisition on a target screen by a first camera, and the second image is obtained by carrying out image acquisition on the target screen by a second camera; the target screen is used for displaying different events in sequence, the target screen comprises a first area and a second area, in adjacent events, the gray levels of images displayed in the first area are different, and the gray levels of images displayed in the second area are different; determining to-be-identified areas respectively corresponding to the first area and the second area in the first image and the second image; calculating clock offsets of the first camera and the second camera according to the gray value of each to-be-identified area; and debugging the first camera and / or the second camera according to the clock offset so as to synchronize the shooting time of the first camera and the shooting time of the second camera. According to the invention, the cost of multi-camera time synchronization is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image acquisition technology, and in particular to a multi-camera time synchronization method, device, computer equipment and storage medium. Background Art

[0002] When performing image data acquisition, in order to improve data acquisition efficiency, multiple cameras are usually used for "simultaneous" shooting. However, the hardware system clocks of different cameras (referred to as camera time, the same below) are different, and certain measures need to be taken to unify the time standards of different cameras. When synchronizing camera time in existing methods, the conventional approach is to use the "high-frequency event + camera short exposure" method. When multiple cameras with short exposure capture the same high-frequency event, it is considered that this moment is the same time base for multiple cameras, and the accuracy is the inverse of the frequency. The higher the event frequency, the more accurate the synchronization time accuracy. The reason for using short exposure is that only a single event can be captured each time, and it cannot span two events before and after. In reality, it is difficult to create high-frequency events. For example, high-frequency displays are expensive and not high enough, and the cost of high-frequency LED circuits is also relatively high. Summary of the invention

[0003] In view of this, the present invention provides a multi-camera time synchronization method, apparatus, computer equipment and storage medium to solve the problem of high cost when performing time synchronization on multiple cameras in the prior art.

[0004] In a first aspect, the present invention provides a multi-camera time synchronization method, the method comprising: acquiring a first image and a second image, the first image being obtained by performing image acquisition on a target screen by a first camera, the second image being obtained by performing image acquisition on the target screen by a second camera, the target screen being used to sequentially display different events, the target screen comprising a first area and a second area, in adjacent events, the first area and the second area both alternately display a first preset image and a second preset image, the first area and the second area display the first preset image and the second preset image in opposite orders, and the first preset image and the second preset image have different grayscales; determining a first area to be identified corresponding to the first area in the first image, and a second area to be identified corresponding to the second area; determining a third area to be identified corresponding to the first area in the second image, and a fourth area to be identified corresponding to the second area; calculating clock offsets of the first camera and the second camera according to a first grayscale value of the first area to be identified, a second grayscale value of the second area to be identified, a third grayscale value of the third area to be identified, and a fourth grayscale value of the fourth area to be identified; debugging the first camera and / or the second camera according to the clock offset to synchronize the shooting time of the first camera and the second camera.

[0005] In the method provided by the embodiment of the present invention, the first camera and the second camera capture images of the target screen to obtain the first image and the second image. Since the target screen displays different events in sequence, in different events, the first area and the second area of ​​the target screen alternately display the first preset image and the second preset image with different grayscales. Therefore, when the exposure time of the first camera and the second camera is long and the target screen is a low refresh display screen, even if ghosting occurs in the images captured by the first camera and the second camera, the time when the first image is captured and the time when the second image is captured can be determined according to the grayscale value of each area to be identified in the first image and the grayscale value of each area to be identified in the second image, thereby determining the clock deviation between the first camera and the second camera, and further performing time synchronization on the first camera and the second camera. This method does not require the camera to have a network interface or a standard electrical interface, is applicable to all cameras, and does not require the creation of high-frequency events to achieve accurate time synchronization of multiple cameras, thereby reducing costs.

[0006] In some optional embodiments, the step of calculating the clock offset of the first camera and the second camera according to the grayscale value of the first area to be identified, the grayscale value of the second area to be identified, the grayscale value of the third area to be identified, and the grayscale value of the fourth area to be identified includes: calculating a first exposure time of a first event in the first image according to the first grayscale value and the second grayscale value; calculating a first start exposure time when the first camera shoots the first image according to the first preset end time and the first exposure time of the first event; calculating a second exposure time of the second event in the second image according to the third grayscale value and the fourth grayscale value; calculating a second start exposure time when the second camera shoots the second image according to the second preset end time and the second exposure time of the second event; and determining the clock offset of the first camera and the second camera according to the difference between the first start exposure time and the second start exposure time.

[0007] In some optional embodiments, the step of calculating the first exposure time of the first event in the first image according to the first grayscale value and the second grayscale value includes: calculating the first exposure time ratio of the first event in the first image according to the ratio of the first grayscale value in the total grayscale value, the total grayscale value is determined according to the sum of the first grayscale value and the second grayscale value; calculating the first exposure time according to the first exposure time ratio and the total first exposure time, the total first exposure time being the exposure time when the first camera captures the first image.

[0008] In some optional embodiments, the step of calculating the second exposure time of the second event in the second image according to the third grayscale value and the fourth grayscale value includes: calculating the second exposure time ratio of the second event in the second image according to the ratio of the third grayscale value and the total grayscale value, and the total grayscale value is determined according to the sum of the third grayscale value and the fourth grayscale value; calculating the second exposure time according to the second exposure time ratio and the second total exposure time, and the second total exposure time is the exposure time when the second camera captures the second image.

[0009] In some optional embodiments, the target screen also includes a third area. In each event displayed on the target screen, the grayscale of the third area remains consistent, and the grayscale value of each area to be identified is calculated by the following steps: determining a first correction area corresponding to the third area in the first image; determining a first corrected grayscale value of the first correction area based on the first image; determining the first grayscale value based on the difference between an actual grayscale value of the first area to be identified and the first corrected grayscale value; determining the second grayscale value based on the difference between an actual grayscale value of the second area to be identified and the first corrected grayscale value; determining a second correction area corresponding to the third area in the second image; determining the second corrected grayscale value of the second correction area based on the second image; determining the third grayscale value based on the difference between the actual grayscale value of the third area to be identified and the second corrected grayscale value; determining the fourth grayscale value based on the difference between the actual grayscale value of the fourth area to be identified and the second corrected grayscale value.

[0010] In practical applications, there are influences such as ambient background light, screen light leakage, and camera background noise, which may cause the grayscale of each area of ​​the image captured by the first camera and the second camera to be different from the grayscale of each area displayed on the target screen. Therefore, in order to improve the accuracy, in an embodiment of the present invention, a third area is set in the target screen, and the grayscale value of each area to be identified is verified by the grayscale value of the third area, so that the clock deviation calculated according to the grayscale value is more accurate.

[0011] In some optional embodiments, when the target screen displays different events, the exposure time of each event is the same; the total first exposure time when the first camera captures the first image is the same as the exposure time of each event; the total second exposure time when the second camera captures the second image is the same as the exposure time of each event.

[0012] In some optional implementations, the first exposure time is calculated by the following formula:

[0013]

[0014] in, represents the first preset end time, n1 represents the sequence number of the first event, represents the exposure time of each event, represents the camera exposure time, wherein N1=N2, G1 represents the actual gray value of the first area to be identified, G2 represents the first corrected gray value, and G3 represents the actual gray value of the second area to be identified.

[0015] In a second aspect, the present invention provides a multi-camera time synchronization device, the device comprising: an image acquisition module, used to acquire a first image and a second image, the first image is obtained by a first camera performing image acquisition on a target screen, the second image is obtained by a second camera performing image acquisition on the target screen, the target screen is used to sequentially display different events, the target screen comprises a first area and a second area, in adjacent events, the first area and the second area both alternately display a first preset image and a second preset image, the first area and the second area display the first preset image and the second preset image in opposite orders, and the grayscales of the first preset image and the second preset image are different; a first image recognition module, used to determine a first A first area to be identified corresponding to the first area in the image, and a second area to be identified corresponding to the second area; a second image recognition module, used to determine a third area to be identified corresponding to the first area in the second image, and a fourth area to be identified corresponding to the second area; a clock offset calculation module, used to calculate the clock offset of the first camera and the second camera according to the first grayscale value of the first area to be identified, the second grayscale value of the second area to be identified, the third grayscale value of the third area to be identified, and the fourth grayscale value of the fourth area to be identified; a debugging module, used to debug the first camera and / or the second camera according to the clock offset to synchronize the shooting time of the first camera and the second camera.

[0016] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the multi-camera time synchronization method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0017] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the multi-camera time synchronization method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1It is a schematic diagram of the square wave signal sent by the signal generator in the hardware synchronization signal method;

[0020] Figure 2 is a schematic flow chart of a multi-camera time synchronization method according to an embodiment of the present invention;

[0021] Figure 3 is an event displayed on a target screen according to an embodiment of the present invention;

[0022] Figure 4 is a schematic diagram of a case where a first preset image and a second preset image overlap in a first to-be-recognized area according to an embodiment of the present invention;

[0023] Figure 5 is a target screen including four areas according to an embodiment of the present invention;

[0024] Figure 6 are images when the target screen displays 1 frame and 2 frames according to an embodiment of the present invention;

[0025] Figure 7 is an image captured when the exposure time of the camera according to the embodiment of the present invention is 10 ms and is exactly 0 to 10 ms;

[0026] Figure 8 is an image captured by a camera according to an embodiment of the present invention when the exposure is 10 ms and the span is 5 to 15 ms;

[0027] Fig. 9 is an image captured by a camera according to an embodiment of the present invention when the exposure time is 10 ms and the span is 7.55 to 17.55 ms;

[0028] Fig.10 is a schematic diagram of calculating the camera start exposure time according to an embodiment of the present invention;

[0029] Fig.11 is a structural block diagram of a multi-camera time synchronization device according to an embodiment of the present invention;

[0030] Fig.12 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0032] In order to enable multiple cameras to achieve true simultaneous shooting, the time of multiple cameras needs to be synchronized. In the related art, the methods commonly used for time synchronization of multiple cameras include: (1) network timing method, (2) hardware synchronization signal method, and (3) event synchronization method.

[0033] The steps of the network timing method include:

[0034] (1) The network switch acts as a unified clock server and is connected to the terminal (device to be timed, such as a camera system) via a network cable;

[0035] (2) The terminal sends data to the server and records the terminal time T1, which needs to be corrected;

[0036] (3) After receiving the time code, the server records the receiving time T2, which is the standard time;

[0037] (4) The server sends data to the terminal and records the sending time T3, which is the standard time;

[0038] (5) The terminal receives the server data and records the receiving time T4, which needs to be corrected;

[0039] (6) The one-way time of the network transmission in the above process is △T = [(T4-T1)-(T3-T2)] / 2, which is the standard time difference between T2 and T1. The standard time difference between the terminal and the server is T2-△T-T1, which is the correction amount of the terminal time relative to the standard time.

[0040] (7) After completing the above steps on different terminals, the terminal time can be corrected to the standard time of the server.

[0041] When synchronizing the time of each camera through the network timing method, the wireless network has a large delay and is easily interfered with, resulting in poor timing accuracy. Therefore, it is necessary to connect the clock server to each camera through a network cable. This method requires the camera to support a wired network interface and has poor universality.

[0042] The steps of the hardware synchronization signal method include:

[0043] (1) The same signal generator is connected to each camera through a coaxial line, and sends square wave signals to different cameras respectively. For example, the square wave signal is as follows: Figure 1 As shown;

[0044] (2) When different cameras receive the rising edge of the square wave voltage, they all start to expose simultaneously.

[0045] This method requires that all cameras use a standard electrical interface. Cameras without this standard interface cannot use this method.

[0046] The event synchronization method includes the LED marquee solution and the high refresh rate display solution.

[0047] The LED marquee solution specifically includes:

[0048] (1) Design two 10*10 LED arrays and light up each LED in the first array in turn at an interval of 0.1ms;

[0049] (2) After lighting up the last lamp bead in array 1, the first lamp bead in another array is lit, which means that 10ms have passed, and array 1 lights up the first lamp bead again from the beginning;

[0050] (3) Repeat the above steps. After lighting up the LED in array 1, light up the next LED in array 2.

[0051] (4) Different cameras use short exposure (<0.05ms) to shoot the above lamp bead arrays. If camera A shoots the lighting position of array 1 as n1 and the lighting position of array 2 as n2, the corresponding time is T1=n2*10ms+n1*0.1ms; if camera B shoots the lighting position of array 1 as n3 and the lighting position of array 2 as n4, the corresponding time is T2=n4*10ms+n3*0.1ms;

[0052] (5)△T=T2-T1 is the clock offset of camera B relative to camera A.

[0053] The LED marquee solution requires dedicated LED hardware equipment, and the cost of customized equipment is relatively high.

[0054] The high refresh rate display solution specifically includes:

[0055] (1) Using a 500Hz high refresh rate display, each frame displays a different number, and the numbers continue to accumulate;

[0056] (2) Different cameras use short exposure (<0.05ms) to shoot the above display. If camera A shoots, the number is n1, T1 = n1*2ms; if camera B shoots, the number is n2, T2 = n2*2ms;

[0057] (3) △T = T2-T1 is the clock offset of camera B relative to camera A.

[0058] The drawbacks of high refresh rate display solutions are that high refresh rate screens are expensive and the time synchronization accuracy can only reach 2ms.

[0059] In order to solve the problem that the time synchronization of multiple cameras in the prior art requires the cameras to have a wired network interface or a standard electrical interface, and to solve the defect of high cost in the time synchronization of multiple cameras in the prior art, the embodiment of the present invention considers using the "low-frequency event + long exposure" method to perform time synchronization on multiple cameras. If the "low-frequency event + long exposure" method is used, such as a screen refresh event, the refresh rate N is 60Hz or 120Hz, and the display refreshes a different number for each frame. According to the numbers captured by different cameras, the time synchronization accuracy of different cameras can be controlled to 1 / N seconds. However, due to the use of long exposure, if the long exposure captures the two previous and next events, the two numbers of the two previous and next screens are exposed, and double images appear in the photo. Although the cost is reduced, the time synchronization of multiple cameras cannot be achieved through the prior art.

[0060] According to an embodiment of the present invention, an embodiment of a multi-camera time synchronization method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0061] In this embodiment, a multi-camera time synchronization method is provided. Figure 2 is a flowchart of a multi-camera time synchronization method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0062] Step S101, obtaining a first image and a second image, wherein the first image is obtained by capturing an image of a target screen by a first camera, and the second image is obtained by capturing an image of the target screen by a second camera. The target screen is used to sequentially display different events, and the target screen includes a first area and a second area. In adjacent events, the first area and the second area both alternately display a first preset image and a second preset image, and the first area and the second area display the first preset image and the second preset image in opposite orders, and the grayscales of the first preset image and the second preset image are different.

[0063] In the embodiment of the present invention, the target screen may be a conventional low-frequency display, and the single frame duration of the target screen may be 1 / 60 second, 1 / 120 second, etc.

[0064] In the embodiment of the present invention, the order in which the first area and the second area display the first preset image and the second preset image is opposite, that is, when the first area displays the first preset image, the second area displays the second preset image, and when the first area displays the second preset image, the second area displays the first preset image. If in the first event, the first area displays the first preset image and the second area displays the second preset image, then in the second event, the first area displays the second preset image and the second area displays the first preset image, and in the third event, the first area displays the first preset image and the second area displays the second preset image, wherein the next event of the first event is the second event, and the next event of the second event is the third event, and so on, the target screen displays different events in sequence.

[0065] In an optional embodiment, the first preset image may be a black rectangular frame, and the second preset image may be a white rectangular frame, and the black rectangular frame and the white rectangular frame have different grayscales.

[0066] In an optional embodiment, the first area alternately displays a black rectangular frame and a white rectangular frame, and the second area also alternately displays a black rectangular frame and a white rectangular frame. In the same event, the first area and the second area display different images. For example, Figure 3 As shown, when the first area is a black rectangular frame, the second area is a white rectangular frame.

[0067] Step S102, determining a first area to be identified corresponding to the first area and a second area to be identified corresponding to the second area in the first image.

[0068] In an optional embodiment, the first area to be identified and the second area to be identified may be determined in the first image according to the relative positions of the first area and the second area in the target screen.

[0069] Step S103, determining a third area to be identified corresponding to the first area and a fourth area to be identified corresponding to the second area in the second image.

[0070] In an optional embodiment, the third area to be identified and the fourth area to be identified may be determined in the second image according to the relative positions of the first area and the second area in the target screen.

[0071] Step S104 , calculating clock offsets of the first camera and the second camera according to the first grayscale value of the first to-be-recognized area, the second grayscale value of the second to-be-recognized area, the third grayscale value of the third to-be-recognized area, and the fourth grayscale value of the fourth to-be-recognized area.

[0072] In the embodiment of the present invention, when the first camera acquires the first image, due to the long exposure time, the exposure time may span two events displayed on the target screen. At this time, due to the overlap of the previous event and the next event in the first image, the first grayscale value of the first area to be identified is neither equal to the grayscale value of the first preset image nor equal to the grayscale value of the second preset image. Figure 4 As shown. If the subsequent event occurs for a shorter time, the image displayed by the subsequent event in the first area has less influence on the image displayed by the previous event in the first area, that is, the grayscale value of the image displayed by the subsequent event in the first area has less influence on the grayscale value of the image displayed by the previous event in the first area. On the contrary, if the subsequent event occurs for a longer time, the grayscale value of the image displayed by the subsequent event in the first area has a greater influence on the grayscale value of the image displayed by the previous event in the first area. Similarly, the occurrence of the subsequent event will also have a certain influence on the grayscale value of the image displayed by the previous event in the second area. Therefore, in an embodiment of the present invention, the exposure duration of each of the two events during the exposure of the first camera can be determined according to the first grayscale value of the first area to be identified and the second grayscale value of the second area to be identified, so that the time when the first camera starts to expose can be accurately obtained in combination with the pre-set time of occurrence of each event.

[0073] Similarly, in an embodiment of the present invention, the exposure duration of the first event and the second event during the exposure of the second camera can be determined according to the third grayscale value of the third area to be identified and the fourth grayscale value of the fourth area to be identified, so as to accurately obtain the time when the second camera starts exposing.

[0074] Further, the clock offset between the first camera and the second camera may be determined according to the time when the first camera starts to expose the image and the time when the second camera starts to expose the image.

[0075] Step S105 , debugging the first camera and / or the second camera according to the clock offset, so as to synchronize the shooting time of the first camera and the second camera.

[0076] The above steps are only used to explain the method provided by the present invention by taking time synchronization of two cameras as an example. In practical applications, the method provided by the present invention can be used to time synchronize multiple cameras.

[0077] In the method provided by the embodiment of the present invention, the first camera and the second camera capture images of the target screen to obtain the first image and the second image. Since the target screen displays different events in sequence, in different events, the first area and the second area of ​​the target screen alternately display the first preset image and the second preset image with different grayscales. Therefore, when the exposure time of the first camera and the second camera is long and the target screen is a low refresh display screen, even if ghosting occurs in the images captured by the first camera and the second camera, the time when the first image is captured and the time when the second image is captured can be determined according to the grayscale value of each area to be identified in the first image and the grayscale value of each area to be identified in the second image, thereby determining the clock deviation between the first camera and the second camera, and further performing time synchronization on the first camera and the second camera. This method does not require the camera to have a network interface or a standard electrical interface, is applicable to all cameras, and does not require the creation of high-frequency events to achieve accurate time synchronization of multiple cameras, thereby reducing costs.

[0078] In an optional embodiment, the above step S104 specifically includes the following steps:

[0079] Step a1: calculating a first exposure time of a first event in a first image according to the first grayscale value and the second grayscale value.

[0080] In an optional embodiment, when calculating the first exposure time of the first event in the first image, first, the first exposure time ratio of the first event in the first image is calculated according to the ratio of the first grayscale value in the total grayscale value, wherein the total grayscale value is determined according to the sum of the first grayscale value and the second grayscale value; then, the first exposure time is calculated according to the first exposure time ratio and the total first exposure time, wherein the total first exposure time is the exposure time when the first camera captures the first image.

[0081] In the embodiment of the present invention, when the first image spans two events, the first event is the event that appears first in the first image.

[0082] In an embodiment of the present invention, within the camera exposure time, the longer the first event occurs, the closer the first grayscale value is to the grayscale value in the first area of ​​the first event, and the shorter the first event occurs, the closer the first grayscale value is to the grayscale value in the first area of ​​the next event of the first event. Therefore, the first exposure time of the first event in the first image can be determined according to the proportion of the first grayscale value in the total grayscale value.

[0083] Step a2: calculating a first start exposure time when the first camera captures the first image according to the first preset end time of the first event and the first exposure time.

[0084] In an optional embodiment, the target screen displays each event in sequence, and the preset end time of the first event captured in the first image can be determined according to the duration of each event and the start time of the first event.

[0085] In an optional embodiment, the first start exposure time is the difference between a preset end time of the first event and the first exposure time.

[0086] Step a3, calculating the second exposure time of the second event in the second image according to the third grayscale value and the fourth grayscale value. The method of calculating the second exposure time is the same as the method of calculating the first exposure time in step a1, which will not be repeated here.

[0087] In an optional embodiment, the second event and the first event may be the same event, that is, the images captured by the first camera and the second camera span the same two events.

[0088] Step a4: calculating a second start exposure time when the second camera captures the second image according to the second preset end time of the second event and the second exposure time.

[0089] Step a5: determining the clock offset of the first camera and the second camera according to the difference between the first exposure start time and the second exposure start time.

[0090] In an optional embodiment, since it is necessary to determine the preset end time of the event that first appears in the image when calculating the start exposure time of each camera, the target screen can also display the identifier of each event when displaying each event. When the camera collects images of the target screen, it will collect the identifier of the current event. Even if the collected image spans multiple events, the identifier of the event that first appears in the image can be determined through the image, thereby determining the preset end time of the event that first appears, and further calculating the start exposure time of the camera. For example, the numbers "1, 2, 3, ..." can be used as the identifiers of each event, and different events correspond to different identifiers.

[0091] In an optional embodiment, the target screen further includes a third area. In each event displayed on the target screen, the grayscale of the third area remains consistent. The grayscale value of each area to be identified is calculated by the following steps:

[0092] Step b1, determining a first correction area corresponding to the third area in the first image, and determining a first correction grayscale value of the first correction area according to the first image.

[0093] Step b2, determining the first grayscale value according to the difference between the actual grayscale value of the first area to be identified and the first corrected grayscale value, and determining the second grayscale value according to the difference between the actual grayscale value of the second area to be identified and the first corrected grayscale value.

[0094] Step b3, determining a second correction area corresponding to the third area in the second image, and determining a second correction grayscale value of the second correction area according to the second image.

[0095] Step b4, determining the third grayscale value according to the difference between the actual grayscale value of the third area to be identified and the second corrected grayscale value, and determining the fourth grayscale value according to the difference between the actual grayscale value of the fourth area to be identified and the second corrected grayscale value.

[0096] In the embodiment of the present invention, although the third area does not change in each event, when the first camera and the second camera capture images, the grayscale of the image displayed by the third area in the first camera and the second camera may deviate due to the camera background noise, shooting angle, etc. of the first camera and the second camera respectively. Therefore, when calculating the grayscale of each area to be identified in the first image and the grayscale of each area to be identified in the second image, it is necessary to calculate the first corrected grayscale value of the third area in the first image and the second corrected grayscale value of the third area in the second image respectively, so as to ensure the accuracy of the grayscale value of each area to be identified.

[0097] In practical applications, there are influences such as background light, screen light leakage, and camera background noise, which result in differences in the grayscale of each area of ​​the image captured by the first camera and the second camera and the grayscale of each area displayed on the target screen. Therefore, in order to improve the accuracy, a third area is set in the target screen in an embodiment of the present invention, and the grayscale value of each area to be identified is verified by the grayscale value of the third area, so that the clock deviation calculated according to the grayscale value is more accurate.

[0098] In an optional embodiment, when the target screen displays different events, the exposure time of each event is the same, the total first exposure time when the first camera captures the first image is the same as the exposure time of each event, and the total second exposure time when the second camera captures the second image is the same as the exposure time of each event.

[0099] For example, a single frame of a 60Hz display lasts for 1 / 60 seconds, and the corresponding camera exposure time is 1 / 60 seconds. A single frame of a 120Hz display lasts for 1 / 120 seconds, and the corresponding camera exposure time is 1 / 120 seconds. This ensures that the camera exposure will not span three events, and in most cases will span two events, with a very small probability of exclusively covering one event.

[0100] In an optional embodiment, the first exposure time is calculated by the following formula:

[0101]

[0102] in, represents the first preset end time, n1 represents the sequence number of the first event, represents the exposure time of each event, represents the camera exposure time, wherein N1=N2, G1 represents the actual gray value of the first area to be identified, G2 represents the first corrected gray value, and G3 represents the actual gray value of the second area to be identified.

[0103] The calculation formula for the second exposure time is the same as the calculation formula for the first exposure time, and will not be repeated here.

[0104] As a specific application example of the multi-camera time synchronization method provided in the above embodiment, for a target screen with a frame rate of 100 FPS, the display time of each frame is 10 ms. Figure 5 As shown, the target screen includes four areas:

[0105] (1) Area A, displayed as a white rectangular frame and a pure black rectangular frame alternately;

[0106] (2) Area B, continuously displays pure black;

[0107] (3) Area C, displayed as alternating black and white rectangular frames;

[0108] (4) Digital display area: The numbers are accumulated from 1, 2, and 3 in sequence to ensure that the numbers displayed in each frame are different.

[0109] In this embodiment, the exposure time of the camera is precisely set to 10ms. In rare cases, the photographed photo is synchronized with the displayed image, and the photographed photo has exactly one number. In most cases, the exposure time of 10ms will span the previous and next frames, and the numbers of the previous and next frames will be displayed simultaneously.

[0110] exist Figure 6 In the figure, area A is white when displaying frame 1 (0-10ms) and turns black when displaying frame 2 (10-20ms). Area C is black when displaying frame 1 (0-10ms) and turns white when displaying frame 2 (10-20ms).

[0111] If the camera exposure time is 10ms and it is exactly 0-10ms, then only frame 1 can be captured. Figure 7 shown.

[0112] If the camera exposure is 10ms and the span is 5 to 15ms, then frame 1 is captured in exactly 5ms and frame 2 in exactly 5ms, and the brightness of area A and area C captured in the image is the same, such as Figure 8 shown.

[0113] If the camera exposure is 10ms, with a span of 7.55-17.55ms, then frame 1 is captured in exactly 2.45ms, and frame 2 is captured in 7.55ms. In the image, area A is darker, area C is brighter, and the upper and lower grayscale ratio is 2.45:7.55. Fig. 9 shown.

[0114] Therefore, the exact time when the camera starts exposing can be calculated by the grayscale ratio of area A and area C.

[0115] For example, the average grayscale of area A is 78.54 (because it is the average of tens of thousands of pixels in the entire area, the average grayscale can have decimal points), and the average grayscale of area C is 187.32, which means that frame 1 is exposed for a shorter time and frame 2 is exposed for a longer time, and the ratio is 78.54:187.32, which is better than the total exposure time of 10ms. It can be easily calculated that the exposure time of frame 1 is 2.954ms, the exposure time of frame 2 is 7.046ms, and the camera exposure range is 7.046~17.046ms, with an error of less than 1ms.

[0116] like Fig.10 As shown, in order to make statistics more accurate and remove the influence of background noise, the grayscale G1 of area A in frame 1 cannot be used directly, but G2 is subtracted. The G2 area is displayed as pure black on the screen, but when taking pictures, there will be background light, camera background noise, and screen light leakage (the screen cannot display pure black), resulting in the actual grayscale of the G2 area not being 0, for example, it may be 2.337. Therefore, (G1-G2) is used as the grayscale of area A. Similarly, (G3-G2) is used as the grayscale of area C.

[0117] The grayscale of region A is compared with the grayscale of region C, and the ratio is (G1-G2):(G3-G2).

[0118] The total grayscale of region A and region C is (G1-G2)+(G3-G2)=G1+G3-2*G2.

[0119] The ratio of the grayscale of area A to the total grayscale of the upper and lower parts is (G1-G2) / (G1+G3-2*G2).

[0120] The camera frame rate and screen frame rate are the same, both N, then the screen display time is 1 / N seconds, and the camera exposure time is also 1 / N seconds; the grayscale ratio of area A represents the proportion of frame 1 in the entire camera exposure time (exposure ratio), then the exposure time of frame 1 is: exposure ratio * total exposure time, that is: (G1-G2) / (G1+G3-2*G2)*(1 / N) seconds.

[0121] The exposure time of frame 1 = the time when frame 1 ends displaying - the time when the camera starts exposure, that is, the time when the camera starts exposure = the time when frame 1 ends displaying - the exposure time of frame 1. The time when frame 1 ends displaying = n1*(1 / N) seconds. n1 is the exposure sequence of frame 1 (0, 1, ..., 9527, ..., 9999), then the time when the camera starts exposure T = n1*(1 / N)-(G1-G2) / (G1+G3-2*G2)*(1 / N) = (n1-(G1-G2) / (G1+G3-2*G2)*(1 / N).

[0122] After calculating the time T1 when the first camera starts to expose and the time T2 when the second camera starts to expose using the above method, ΔT=T2-T1 is the clock offset of the second camera relative to the first camera. The time synchronization of the first camera and the second camera can be achieved according to the clock offset.

[0123] In this embodiment, a multi-camera time synchronization device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0124] This embodiment provides a multi-camera time synchronization device, such as Fig.11 As shown, including:

[0125] An image acquisition module 201 is used to acquire a first image and a second image, wherein the first image is obtained by acquiring an image of a target screen by a first camera, and the second image is obtained by acquiring an image of a target screen by a second camera, wherein the target screen is used to sequentially display different events, and the target screen includes a first area and a second area, wherein in adjacent events, the first area and the second area both alternately display a first preset image and a second preset image, and the first area and the second area display the first preset image and the second preset image in opposite orders, and the grayscales of the first preset image and the second preset image are different;

[0126] A first image recognition module 202, used to determine a first area to be recognized corresponding to the first area, and a second area to be recognized corresponding to the second area in the first image;

[0127] The second image recognition module 203 is used to determine a third area to be recognized corresponding to the first area in the second image, and a fourth area to be recognized corresponding to the second area;

[0128] A clock offset calculation module 204, configured to calculate clock offsets of the first camera and the second camera according to a first grayscale value of the first to-be-recognized area, a second grayscale value of the second to-be-recognized area, a third grayscale value of the third to-be-recognized area, and a fourth grayscale value of the fourth to-be-recognized area;

[0129] The debugging module 205 is used to debug the first camera and / or the second camera according to the clock offset so as to synchronize the shooting time of the first camera and the second camera.

[0130] In some optional implementations, the clock offset calculation module 204 specifically includes:

[0131] A first exposure time calculation submodule, used for calculating a first exposure time of a first event in a first image according to a first gray value and a second gray value;

[0132] A first start exposure time calculation submodule, used to calculate a first start exposure time when the first camera captures the first image according to a first preset end time and a first exposure time of the first event;

[0133] A second exposure time calculation submodule, used for calculating a second exposure time of the second event in the second image according to the third gray value and the fourth gray value;

[0134] A second start exposure time calculation submodule, used to calculate a second start exposure time when the second camera captures the second image according to a second preset end time of the second event and the second exposure time;

[0135] The clock offset calculation submodule is used to determine the clock offset of the first camera and the second camera according to the difference between the first exposure start time and the second exposure start time.

[0136] In some optional implementations, the first exposure time calculation submodule specifically includes:

[0137] A first exposure time proportion calculation unit, configured to calculate a first exposure time proportion of the first event in the first image according to a proportion of the first gray value in the total gray value, wherein the total gray value is determined according to a sum of the first gray value and the second gray value;

[0138] The first exposure time calculation unit is used to calculate the first exposure time according to the first exposure time proportion and the first exposure total time, where the first exposure total time is the exposure time when the first camera captures the first image.

[0139] In some optional implementations, the second exposure time calculation submodule specifically includes:

[0140] A second exposure time proportion calculation unit, configured to calculate a second exposure time proportion of the second event in the second image according to a proportion of the third grayscale value and the total grayscale value, wherein the total grayscale value is determined according to a sum of the third grayscale value and the fourth grayscale value;

[0141] The second exposure time calculation unit is used to calculate the second exposure time according to the second exposure time proportion and the second exposure total time, where the second exposure total time is the exposure time when the second camera captures the second image.

[0142] In some optional implementations, the target screen further includes a third area, and in each event displayed on the target screen, the grayscale of the third area remains consistent, and the clock offset calculation module 204 includes:

[0143] A first correction region calculation submodule, used to determine a first correction region corresponding to the third region in the first image;

[0144] A first correction gray value calculation submodule, used to determine a first correction gray value of a first correction area according to the first image;

[0145] A first grayscale value calculation submodule, used to determine a first grayscale value according to a difference between an actual grayscale value of the first to-be-identified area and a first corrected grayscale value;

[0146] A second grayscale value calculation submodule, used to determine a second grayscale value according to a difference between an actual grayscale value of the second to-be-identified area and the first corrected grayscale value;

[0147] A second correction region calculation submodule, used to determine a second correction region corresponding to the third region in the second image;

[0148] A second correction gray value calculation submodule, used to determine a second correction gray value of a second correction area according to the second image;

[0149] A third grayscale value calculation submodule, used to determine a third grayscale value according to a difference between an actual grayscale value of a third to-be-identified area and the second corrected grayscale value;

[0150] The fourth grayscale value calculation submodule is used to determine the fourth grayscale value according to the difference between the actual grayscale value of the fourth to-be-identified area and the second corrected grayscale value.

[0151] In some optional implementations, when the target screen displays different events, the exposure time of each event is the same;

[0152] The first total exposure time when the first camera captures the first image is the same as the exposure time of each event;

[0153] The second total exposure time when the second camera captures the second image is the same as the exposure time of each event.

[0154] In some optional implementations, the first exposure time is calculated by the following formula:

[0155]

[0156] in, represents the first preset end time, n1 represents the sequence number of the first event, represents the exposure time of each event, represents the camera exposure time, wherein N1=N2, G1 represents the actual gray value of the first area to be identified, G2 represents the first corrected gray value, and G3 represents the actual gray value of the second area to be identified.

[0157] The multi-camera time synchronization device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0158] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0159] The embodiment of the present invention also provides a computer device having the above Fig.11 The multi-camera time synchronization device shown.

[0160] See also Fig.12 , Fig.12 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig.12 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.12 A processor 10 is taken as an example.

[0161] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0162] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0163] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0164] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0165] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Fig.11 The example of connecting through bus is taken in the following.

[0166] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0167] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0168] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A multi-camera time synchronization method, characterized in that: The method comprises: Acquire a first image and a second image, wherein the first image is obtained by performing image acquisition on a target screen by a first camera, and the second image is obtained by performing image acquisition on the target screen by a second camera, the target screen is used to sequentially display different events, the target screen includes a first area and a second area, in adjacent events, the first area and the second area both alternately display a first preset image and a second preset image, the first area and the second area display the first preset image and the second preset image in opposite orders, and the first preset image and the second preset image have different grayscales; Determine a first area to be identified corresponding to the first area, and a second area to be identified corresponding to the second area in the first image; Determine a third area to be identified corresponding to the first area, and a fourth area to be identified corresponding to the second area in the second image; Calculating clock offsets of the first camera and the second camera according to a first grayscale value of the first to-be-recognized area, a second grayscale value of the second to-be-recognized area, a third grayscale value of the third to-be-recognized area, and a fourth grayscale value of the fourth to-be-recognized area; The first camera and / or the second camera are debugged according to the clock offset to synchronize the shooting times of the first camera and the second camera.

2. The method according to claim 1, characterized in that The step of calculating the clock offset of the first camera and the second camera according to the grayscale value of the first area to be identified, the grayscale value of the second area to be identified, the grayscale value of the third area to be identified, and the grayscale value of the fourth area to be identified includes: Calculating a first exposure time of a first event in the first image according to the first grayscale value and the second grayscale value; Calculating a first start exposure time when the first camera captures the first image according to a first preset end time and the first exposure time of the first event; Calculating a second exposure time of a second event in the second image according to the third grayscale value and the fourth grayscale value; Calculating a second start exposure time when the second camera captures the second image according to a second preset end time of the second event and the second exposure time; The clock offsets of the first camera and the second camera are determined according to the difference between the first exposure start time and the second exposure start time.

3. The method according to claim 2, characterized in that The step of calculating a first exposure time of a first event in the first image according to the first grayscale value and the second grayscale value comprises: Calculating a first exposure time ratio of the first event in the first image according to a ratio of the first gray value in a total gray value, wherein the total gray value is determined according to a sum of the first gray value and the second gray value; The first exposure time is calculated according to the first exposure time proportion and the first exposure total time, where the first exposure total time is the exposure time when the first camera captures the first image.

4. The method according to claim 2, characterized in that: The step of calculating a second exposure time of a second event in the second image according to the third grayscale value and the fourth grayscale value comprises: Calculating a second exposure time ratio of the second event in the second image according to a ratio of the third grayscale value to the total grayscale value, wherein the total grayscale value is determined according to a sum of the third grayscale value and the fourth grayscale value; The second exposure time is calculated according to the second exposure time proportion and the second exposure total time, where the second exposure total time is the exposure time when the second camera captures the second image.

5. The method according to claim 3 or 4, characterized in that: The target screen also includes a third area. In each event displayed on the target screen, the grayscale of the third area remains consistent. The grayscale value of each area to be identified is calculated by the following steps: determining a first correction area corresponding to the third area in the first image; determining a first correction grayscale value of the first correction area according to the first image; Determine the first grayscale value according to the difference between the actual grayscale value of the first to-be-identified area and the first corrected grayscale value; Determine the second grayscale value according to the difference between the actual grayscale value of the second to-be-identified area and the first corrected grayscale value; determining a second correction area corresponding to the third area in the second image; determining a second correction grayscale value of the second correction area according to the second image; Determine the third grayscale value according to the difference between the actual grayscale value of the third to-be-identified area and the second corrected grayscale value; The fourth grayscale value is determined according to a difference between an actual grayscale value of the fourth to-be-identified area and the second corrected grayscale value.

6. The method according to claim 5, characterized in that When the target screen displays different events, the exposure time of each event is the same; The first total exposure time when the first camera acquires the first image is the same as the exposure time of each event; The second total exposure time when the second camera captures the second image is the same as the exposure time of each event.

7. The method according to claim 6, characterized in that The first exposure time is calculated by the following formula: in, represents the first preset end time, n1 represents the sequence number of the first event, represents the exposure time of each event, represents the camera exposure time, wherein N1=N2, G1 represents the actual gray value of the first area to be identified, G2 represents the first corrected gray value, and G3 represents the actual gray value of the second area to be identified.

8. A multi-camera time synchronization device, characterized in that: The device comprises: An image acquisition module, used to acquire a first image and a second image, wherein the first image is obtained by acquiring an image of a target screen by a first camera, and the second image is obtained by acquiring an image of the target screen by a second camera, wherein the target screen is used to sequentially display different events, wherein the target screen includes a first area and a second area, wherein in adjacent events, the first area and the second area both alternately display a first preset image and a second preset image, wherein the first area and the second area display the first preset image and the second preset image in opposite orders, and wherein the first preset image and the second preset image have different grayscales; A first image recognition module, used to determine a first area to be recognized corresponding to the first area, and a second area to be recognized corresponding to the second area in the first image; A second image recognition module, used to determine a third area to be recognized corresponding to the first area, and a fourth area to be recognized corresponding to the second area in the second image; a clock offset calculation module, configured to calculate clock offsets of the first camera and the second camera according to a first grayscale value of the first area to be identified, a second grayscale value of the second area to be identified, a third grayscale value of the third area to be identified, and a fourth grayscale value of the fourth area to be identified; A debugging module is used to debug the first camera and / or the second camera according to the clock offset so as to synchronize the shooting time of the first camera and the second camera.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the multi-camera time synchronization method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the multi-camera time synchronization method according to any one of claims 1 to 7.

Images