A method and system for automated testing of multi-camera synchronization error

Abstract

The specification discloses a kind of multi-camera synchronization error automated testing method and system, the method comprises: control display screen device plays the FPS target video of transmission frame number per second made in advance, to make each frame target image in the FPS target video sequentially switch display according to playing interval time and target number;Wherein, each frame target image in the FPS target video is designed with respective relative time stamp according to target number;Control different camera in multi-camera device synchronously continuous shooting the FPS target video played, obtain respective actual imaging image set;Wherein, each actual imaging image carries corresponding trigger time stamp;Control processing device carries out image matching according to trigger time stamp, and determines the relative time stamp of actual imaging image of different camera under same trigger time stamp, calculates multi-camera synchronization error according to the relative time stamp of actual imaging image.

Description

TECHNICAL FIELD

[0001] The present specification relates to the field of vehicle-mounted technology, in particular to a multi-camera synchronization error automatic testing method and system. BACKGROUND

[0002] For a multi-camera acquisition system, high-precision synchronization triggering between multiple cameras has become a more and more common requirement. Whether it is binocular stereo vision, surround stitching, or fusion perception, large error synchronization image information will seriously affect the normal function of the upper application from the root. Taking a vehicle-mounted fisheye camera as an example, four fisheye cameras are generally used as a group in a vehicle-mounted system to combine and stitch a BEV (Bird’s Eye View) perspective, so this group of multi-cameras has a strong demand for synchronization of exposure time. If the multi-cameras in this group are not synchronized in exposure time, resulting in errors in the stitched BEV perspective, it will pose a hidden danger to safe driving.

[0003] In order to reduce the error in multi-camera synchronization, the multi-camera acquisition system will be inspected before leaving the factory. However, the current multi-camera synchronization inspection mechanism is not perfect, and the existing inspection mechanism relies more on manual spot-checking. The multiple cameras in the same system are aligned to the same real-time updating clock screen, and the synchronization error of the multiple cameras is judged by the naked eye. However, this method not only consumes a lot of manpower, but also is difficult to achieve a unified quantitative standard, and cannot be widely promoted. SUMMARY

[0004] The present specification provides a multi-camera synchronization error automatic testing method and system, which detects synchronization error from the perspective of actual imaging error, has the characteristics of low cost but high precision and efficiency, standardizes the detection process and standard of multi-camera synchronization while saving cost, and has the condition of large-scale promotion to the industrial field.

[0005] To solve the above technical problems, the first aspect of the present specification provides a multi-camera synchronization error automatic testing method, which comprises:

[0006] controlling a display screen device to play a pre-made FPS (Frames Per Second) target video, so that each frame of target image in the FPS target video is displayed in sequence according to a playing interval time and a target number; wherein each frame of target image in the FPS target video is designed with a respective relative timestamp according to the target number;

[0007] controlling different cameras in a multi-camera device to synchronously and continuously capture the played FPS target video to obtain respective actual imaging image sets; wherein each actual imaging image carries a corresponding trigger timestamp;

[0008] The control processing device performs image matching according to the trigger timestamp, and determines relative timestamps of actual imaging images of different cameras at the same trigger timestamp, and calculates a multi-camera synchronization error according to the relative timestamps of the actual imaging images.

[0009] Preferably, the target image in the FPS target video comprises N pixel blocks, N≥4 and is a positive integer;

[0010] A pixel block number is drawn in a first fixed area of each pixel block; the pixel block numbers of the N pixel blocks are sequentially combined into a target number of the target image;

[0011] The second fixed area of each pixel block has a respective VFT encoding pattern; in the same pixel block, the decoded number after decoding of the VFT encoding pattern is consistent with the pixel block number, or the VFT encoding pattern is fixed and consistent in all target images and does not change with switching of the target image;

[0012] The third fixed area of each pixel block is drawn with a gray scale encoding area, and the gray scale value in the gray scale encoding area is used to represent the pixel block number.

[0013] Preferably, the target image in the FPS target video is designed according to the following steps:

[0014] A blank image used to make the target image is obtained, and the blank image is equally divided into N pixel blocks; wherein the target image has a target number, N≥4 and is a positive integer;

[0015] A digital number is drawn in a first fixed area of each pixel block to represent the pixel block number of each pixel block; wherein the pixel block numbers of the N pixel blocks are sequentially combined into the target number of the target image;

[0016] A visual fiducial tag (VFT) encoding pattern is used to replace the second fixed area of each pixel block; in the same pixel block, the decoded number after decoding of the VFT encoding pattern is consistent with the pixel block number, or the VFT encoding pattern is fixed and consistent in all target images and does not change with switching of the target image;

[0017] A gray scale encoding area is drawn in the third fixed area of each pixel block; in the same pixel block, the gray scale value in the gray scale encoding area is used to represent the pixel block number.

[0018] Preferably, the drawing of the digital number in the first fixed area of each pixel block comprises:

[0019] A corresponding digital number is selected from 0 to 9 according to the target number of the target image;

[0020] The selected digital number is sequentially drawn in the first fixed area of each pixel block.

[0021] Preferably, the VFT encoding pattern replaces the second fixed area of each pixel block, including:

[0022] According to the target number of the target image or the pixel block number of each pixel block, a corresponding VFT encoding pattern is selected from the pre-prepared VFT encoding pattern set, and the selected VFT encoding pattern is sequentially replaced in the second fixed area of each pixel block; or

[0023] A randomly selected VFT encoding pattern is selected from the pre-prepared VFT encoding pattern set to replace the second fixed area of each pixel block, and the VFT encoding pattern is fixed and consistent in all target images, and does not change with the switching of the target image.

[0024] Preferably, the gray scale encoding area is drawn in the third fixed area of each pixel block, including:

[0025] An arithmetic gray scale value queue is designed, and a pixel value corresponding to each gray scale value in the arithmetic gray scale value is obtained; in the arithmetic gray scale value queue, one gray scale value is associated with one digital number;

[0026] According to the target number of the target image or the pixel block number of each pixel block, a corresponding pixel value is selected to replace the third fixed area of each pixel block to form a corresponding gray scale encoding area.

[0027] Preferably, the method further comprises:

[0028] According to the number of target images and the preset refresh rate, the playing interval time of adjacent target images is calculated;

[0029] According to the target number of each target image and the playing interval time, the FPS target video is made.

[0030] Preferably, if at each time of synchronous shooting, due to the short camera exposure time, only a single target image is obtained as an actual imaging image within a single exposure time, the control processing device matches the images according to the trigger time stamp, and determines the relative time stamp of the actual imaging image of different cameras at the same trigger time stamp, and calculates the multi-camera synchronization error according to the relative time stamp of the actual imaging image, including:

[0031] The processing device is controlled to match according to the trigger time stamp, and a single target image of different cameras at the same trigger time stamp is obtained;

[0032] Resolving the VFT encoding pattern in the single target image of any two target cameras with co-viewing field of view, obtaining the target number of the single target image of the two target cameras;

[0033] Determining the relative time stamp of each target camera according to the target number of the single target image of the two target cameras;

[0034] Taking the difference of the relative time stamps of the two target cameras as the synchronization error of the two target cameras.

[0035] Preferably, if multiple target images are superimposed and exposed as actual imaging images within a single frame exposure time due to the long exposure time of the camera during each synchronization shooting, the control display screen device plays a pre-made FPS target video, specifically including:

[0036] Controlling the display screen device to play the FPS target video with the fixed and consistent VFT encoding pattern in all target images.

[0037] Preferably, if multiple target images are superimposed and exposed as actual imaging images within a single frame exposure time due to the long exposure time of the camera during each synchronization shooting, the control processing device performs image matching according to the trigger time stamp and determines the relative time stamp of the actual imaging images of different cameras at the same trigger time stamp, and calculates the multi-camera synchronization error according to the relative time stamp of the actual imaging images, including:

[0038] Controlling the processing device to perform matching according to the trigger time stamp to obtain the actual imaging images of different cameras at the same trigger time stamp;

[0039] For a single camera, the corresponding actual imaging image is located and searched through the VFT encoding pattern to obtain the corresponding gray value; the average value of the gray value corresponding to the single camera in the gray encoding area is processed to obtain the final gray value corresponding to the single camera in the gray encoding area; the final gray value is used to search the gray value of the multiple superimposed and exposed target images, determine the target image with the closest gray value, and take the relative time stamp of the target image with the closest gray value as the relative time stamp of the actual imaging image;

[0040] Calculating the multi-camera synchronization error according to the relative time stamp of the actual imaging images of different cameras.

[0041] In a second aspect of the present specification, a multi-camera synchronization error automatic testing system is provided, and the system includes:

[0042] A display device is configured to play a pre-produced frame per second (FPS) target video, so that each frame of target image in the FPS target video is sequentially displayed according to a display interval time and a target number; wherein each frame of target image in the FPS target video is designed with a respective relative time stamp according to the target number.

[0043] Different cameras in a multi-camera device are configured to synchronously and continuously capture the played FPS target video to obtain a respective set of actual imaging images; wherein each actual imaging image carries a corresponding trigger time stamp.

[0044] A processing device is configured to perform image matching according to the trigger time stamp, and determine relative time stamps of actual imaging images of different cameras at the same trigger time stamp, and calculate a multi-camera synchronization error according to the relative time stamps of the actual imaging images.

[0045] Preferably, each target image in the FPS target video comprises N pixel blocks, wherein N is a positive integer greater than or equal to 4.

[0046] A pixel block number is drawn in a first fixed area of each pixel block; and the pixel block numbers of the N pixel blocks are sequentially combined to form the target number of the target image.

[0047] A second fixed area of each pixel block has a respective VFT encoding pattern; in the same pixel block, a decoded number decoded from the VFT encoding pattern is consistent with the pixel block number, or the VFT encoding pattern is fixed and consistent in all target images and does not change with the switching of the target image.

[0048] A third fixed area of each pixel block is drawn with a gray scale encoding area, and a gray scale value in the gray scale encoding area is used to represent the pixel block number.

[0049] Preferably, the system further comprises a target image design device configured to design the target image in the FPS target video according to the following steps:

[0050] A blank image used to produce the target image is obtained, and the blank image is equally divided into N pixel blocks; wherein the target image has a target number, and N is a positive integer greater than or equal to 4.

[0051] A digital number is drawn in a first fixed area of each pixel block to represent a pixel block number of each pixel block; wherein the pixel block numbers of the N pixel blocks are sequentially combined to form the target number of the target image.

[0052] A visual fiducial tag (VFT) encoding pattern is used to replace the second fixed area of each pixel block; in the same pixel block, the decoded number of the VFT encoding pattern after decoding is consistent with the pixel block number, or the VFT encoding pattern is fixed and consistent in all target images and does not change with the switching of target images.

[0053] A gray scale encoding area is drawn in the third fixed area of each pixel block; in the same pixel block, the gray scale value in the gray scale encoding area is used to represent the pixel block number.

[0054] Preferably, the target image design device is specifically used for:

[0055] According to the target number of the target image, a corresponding digital number is selected from 0 to 9;

[0056] The selected digital number is sequentially drawn in the first fixed area of each pixel block.

[0057] Preferably, the target image design device is specifically used for:

[0058] According to the target number of the target image or the pixel block number of each pixel block, a corresponding VFT encoding pattern is selected from a set of VFT encoding patterns prepared in advance, and the selected VFT encoding pattern is used to sequentially replace the second fixed area of each pixel block; or

[0059] A randomly selected VFT encoding pattern is selected from a set of VFT encoding patterns prepared in advance to replace the second fixed area of each pixel block, and the VFT encoding pattern is fixed and consistent in all target images and does not change with the switching of target images.

[0060] Preferably, the target image design device is specifically used for:

[0061] A sequence of equal-difference gray scale values is designed, and a pixel value corresponding to each gray scale value in the equal-difference gray scale value is obtained; in the sequence of equal-difference gray scale values, one gray scale value is associated with one digital number;

[0062] According to the target number of the target image or the pixel block number of each pixel block, a corresponding pixel value is selected to replace the third fixed area of each pixel block, forming a corresponding gray scale encoding area.

[0063] Preferably, the system further comprises a video design device for:

[0064] According to the number of target images and the preset refresh rate, the playing interval time of adjacent target images is calculated;

[0065] The FPS target video is produced according to the target numbers and the playing interval time of each target image.

[0066] Preferably, if, at each time of synchronous shooting, only a single target image is obtained as an actual imaging image within a single frame exposure time due to a short camera exposure time, the processing device is specifically configured to:

[0067] The processing device is controlled to match according to the trigger time stamp, so as to obtain a single target image of each target camera at the same trigger time stamp;

[0068] The VFT coding pattern in the single target image of any two target cameras with a common view field is analyzed, so as to obtain the target numbers of the single target images of the two target cameras;

[0069] The relative time stamps of the two target cameras are determined according to the target numbers of the single target images of the two target cameras;

[0070] The difference between the relative time stamps of the two target cameras is taken as the synchronization error of the two target cameras.

[0071] Preferably, if, at each time of synchronous shooting, multiple target images are superimposed and exposed as an actual imaging image within a single frame exposure time due to a long camera exposure time, the display screen device is specifically configured to control the display screen device to play the FPS target video in which the VFT coding pattern is fixed and consistent in all target images.

[0072] Preferably, if, at each time of synchronous shooting, multiple target images are superimposed and exposed as an actual imaging image within a single frame exposure time due to a long camera exposure time, the processing device is specifically configured to:

[0073] The processing device is controlled to match according to the trigger time stamp, so as to obtain an actual imaging image of each target camera at the same trigger time stamp;

[0074] For a single camera, a gray scale coding area of a corresponding actual imaging image is located and searched through the VFT coding pattern, so as to obtain a corresponding gray scale value; the gray scale value corresponding to the gray scale coding area of the single camera is processed by averaging, so as to obtain a final gray scale value corresponding to the gray scale coding area of the single camera; the final gray scale value is used to search the gray scale values of the multiple superimposed and exposed target images, so as to determine a target image with a closest gray scale value, and the relative time stamp of the target image with the closest gray scale value is taken as the relative time stamp of the actual imaging image;

[0075] The multi-camera synchronization error is calculated according to the relative time stamps of the actual imaging images of the different cameras.

[0076] In a third aspect of the present specification, a target image is provided, which is used to perform the test method described in the above technical solution; the target image comprises N pixel blocks, N is a positive integer and N≥4;

[0077] A pixel block number is drawn in the first fixed area of each pixel block; the pixel block numbers of the N pixel blocks are sequentially combined as a target number of the target image;

[0078] The second fixed area of each pixel block has a respective VFT encoding pattern; in the same pixel block, the decoded number after decoding of the VFT encoding pattern is consistent with the pixel block number, or the VFT encoding pattern is fixed and consistent in all target images and does not change with the switching of the target image;

[0079] The third fixed area of each pixel block is drawn with a gray scale encoding area, and the gray scale value in the gray scale encoding area is used to represent the pixel block number.

[0080] In a fourth aspect of the present specification, an FPS target video is provided, which is made by the target pattern described in the above technical solution.

[0081] Through one or more embodiments of the present specification, the present specification has the following beneficial effects or advantages:

[0082] The present specification discloses a kind of multi-camera synchronization error automatic test method and system, in the method, control display screen device plays the frames per second FPS target video made in advance, to make each frame target image in the FPS target video sequentially switch display according to playing interval time and target number;Wherein, each frame target image in the FPS target video is designed with respective relative time stamp according to target number;Different camera in control multi-camera device is synchronously and continuously photographed the FPS target video played, and respective actual imaging image set is obtained;Wherein, each actual imaging image carries corresponding trigger time stamp;Control processing device carries out image matching according to trigger time stamp, and determines the relative time stamp of actual imaging image of different camera under the same trigger time stamp, calculates multi-camera synchronization error according to the relative time stamp of actual imaging image.The scheme of the present specification starts from the actual imaging error angle, under the condition of synchronous shooting, the relative time stamp carried by each target image is detected to detect synchronization error, with the characteristics of low cost but high precision and high efficiency.In addition, no more preparation is needed in test stage, and there is no special requirement for site, only need to play the FPS target video made in advance on screen, the multi-camera synchronization error test in (a set or even multiple sets) multi-camera acquisition device can be completed at the same time, save cost at the same time standardize the detection process and standard of multi-camera synchronization, with the condition of large-scale promotion to industrial field.

[0083] The above description is only a summary of the technical solutions of the present specification. In order to enable a more thorough understanding of the technical means of the present specification, the content of the present specification can be implemented, and in order to enable the above and other purposes, features and advantages of the present specification to be more obvious and easy to understand, the following specific embodiments of the present specification are described. BRIEF DESCRIPTION OF DRAWINGS

[0084] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not meant to limit the present specification. Moreover, the same reference numerals are used throughout the several views to designate the same or similar parts. In the drawings:

[0085] Figure 1 A schematic diagram of a target image of a target numbered 9527 is shown according to an embodiment of the present specification;

[0086] Figure 2 A design process diagram of a target image is shown according to an embodiment of the present specification;

[0087] Figure 3 A process diagram of an automated testing method of multi-camera synchronization error is shown according to an embodiment of the present specification;

[0088] Figure 4 A schematic diagram of an automated testing system of multi-camera synchronization error is shown according to an embodiment of the present specification. DETAILED DESCRIPTION

[0089] Exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0090] The embodiments of the present specification provide an automated testing method and system of multi-camera synchronization error, to solve the technical problems that the prior art is difficult to achieve unified quantitative standard while consuming a large amount of labor, and cannot be popularized on a large scale.

[0091] Specifically, this manual offers improvements in both target images and testing. Firstly, it designs target images and FPS target videos for testing. Secondly, it improves the testing process. This solution leverages machine vision's ability to rapidly detect and locate image tags. It establishes a complete end-to-end process, from multi-camera acquisition, image frame matching, target image decoding including actual exposure imaging, and camera synchronization error analysis. This provides automated camera synchronization error testing capabilities for all industrial multi-camera acquisition systems.

[0092] The following embodiments will first be introduced with respect to target images and FPS target videos.

[0093] In this embodiment, the FPS target video is composed of several frames of target images combined according to the target number, and the playback interval time of adjacent frames of target images is calculated by the number of target images and the preset refresh rate. For example, 10,000 target images with target numbers ranging from 0000 to 9999 are made into a specified FPS video. Then, in a 100FPS target video, the playback interval time of adjacent frames of target images is 100s / 10000=10ms. In addition, each frame of target image is designed with its own relative timestamp according to the target number. For example, the target number of the first frame target image (A) is 0000, and its relative timestamp is 0ms. The target number of the second frame target image (B) is 0001, and its relative timestamp is 10ms, and so on. Furthermore, the playback interval between adjacent target images is equal to the time error between the relative timestamps of adjacent target images. For example, if the time error between target image B and target image A is 10ms, the playback interval between target image B and target image A is also 10ms, making the two equal. This embodiment standardizes the switching timing of target images during video playback by designing the relationship between the playback interval, target number, and relative timestamp, thereby ensuring the accuracy of the subsequent synchronization error calculation using relative timestamps. Furthermore, in this embodiment, the playback interval between all adjacent target images is equal, for example, 10ms.

[0094] In this embodiment, the target image in the FPS target video includes N pixel blocks, where N≥4 and is a positive integer. Figure 1 , which is a schematic diagram of the structure of a target image. This target image contains 4 pixel blocks. It is worth noting that the structure of all target images in the FPS target video is the same, so this embodiment only uses a single target image as an example for description.

[0095] The pixel block number is drawn in the first fixed area of ​​each pixel block. The pixel block numbers of N pixel blocks are sequentially combined into the target number of the target image, which plays a role in distinguishing the target image when storing. Figure 1For example, the upper left corners of four pixel blocks are marked with the numbers 9, 5, 2, and 7, respectively, from left to right and from top to bottom. The pixel block numbers are then combined in sequence from left to right and from top to bottom to form 9527, which serves as the target number for the target image. Of course, the first fixed area is not limited to the upper left corner, and the combination order is not limited to the order from left to right and from top to bottom.

[0096] The second fixed area of ​​each pixel block has its own VFT coding pattern. VFT (Visual Fiducial Tags) is a commonly used labeling tool for identification and positioning through camera images. In this embodiment, VFT coding patterns corresponding to the digital numbers 0 to 9 are drawn in advance. The VFT coding patterns for different numbers are different. By parsing the VFT coding patterns, the corresponding decoding numbers can be obtained. In addition to parsing to obtain the decoding numbers, the VFT coding patterns also play a positioning role. For example, after obtaining the VFT coding pattern, the target number in the first fixed area and the grayscale coding area in the third fixed area can be located and searched.

[0097] In this embodiment, two different types of target images can be drawn using a VFT coded pattern. These two types of target images constitute their own corresponding FPS target videos and cannot appear in the same FPS target video. To distinguish them, this embodiment names the two types of target images as first-type target images and second-type target images. For target images with the same target number, the first-type target image and the second-type target image are identical except for the VFT coded pattern and / or target number drawn in the second fixed area.

[0098] In an optional embodiment, the first type target image is in the same pixel block, and the decoding number after the VFT coding pattern is decoded is consistent with the pixel block number. Figure 1 As shown, the VFT coding pattern of each pixel block remains consistent with the numerical code corresponding to its upper left corner after decoding, which is 9, 5, 2, and 7. In this embodiment, the VFT coding pattern in the target image changes with the change of the target number. In the same target image, the number sequence formed by the sequential combination of the decoded numbers after the VFT coding pattern is decoded is consistent with the target number. The first type of target image is suitable for calculating the synchronization error when only a single target image is obtained as the actual imaging image within a single frame exposure time due to a short camera exposure time. If the camera exposure time is long and multiple target images are superimposed and exposed within a single frame exposure time as the actual imaging image, it is necessary to use an FPS target video composed of the second type of target image.

[0099] In an alternative embodiment, the VFT encoding pattern in the second type of target image is fixed and consistent in all target images, and does not change with the switching of the target images. The other aspects are the same as the first type of target image. For example, the target number formed by the combination of the first fixed pixel regions changes with the switching of the target images, regardless of whether it is the first type of target image or the second type of target image. In this embodiment, the camera may overlap multiple target images into one actual imaging image due to the long exposure time during single-frame exposure imaging, and at this time, multiple VFT encoding patterns overlap, and the actual imaging image cannot be located and searched by using the VFT encoding pattern to locate the target number and the gray code area. Therefore, in this embodiment, the fixed VFT encoding pattern is filled into each target image, and does not change with the switching of the target images. Of course, the fixed VFT encoding pattern still has the positioning function, for example, after identifying the VFT encoding pattern, the gray code area in the third fixed region is located and searched to the left, and the gray code area drawn in the third fixed region is used to calculate the target number of the actual imaging image.

[0100] Generally, the first type of target image is suitable for the case where the camera exposure time is short, and only one target image is obtained as the actual imaging image during single-frame exposure time. The second type of target image is suitable for the case where the camera exposure time is long, and multiple target images are overlapped and exposed as the actual imaging image during single-frame exposure time. If two types of target images cannot be provided during actual use, the second type of target image can be used to replace the first type of target image, and the gray code area is used to replace the VFT encoding pattern to calculate the synchronization error.

[0101] The third fixed region of each pixel block is drawn with a gray code area, and the gray value in the gray code area is used to represent the pixel block number. It is worth noting that in the same pixel block, all the pixel points in the gray code area have the same gray value, for example, one of 0, 28, 56, and the like. The gray code area is used in the case where multiple target images are exposed and imaged into an actual imaging image by each camera during single-frame exposure imaging. Figure 2 In the embodiment, the gray code area is located to the left of the VFT encoding pattern and below the digital number.

[0102] It is worth noting that in the same pixel block, the first fixed region, the second fixed region, and the third fixed region do not overlap.

[0103] The above is a specific introduction of the target image. When testing is required, the above target image can be made into an FPS target video for playing, which has the characteristics of low cost but high precision and high efficiency compared with other detection methods. Since each target image is provided with a digital number associated with the target number, a VFT coding pattern and a gray coding area, the target image of the embodiment is applicable under various conditions when multiple cameras are exposed, and can be promoted to the industrial field on a large scale.

[0104] To further illustrate and explain the target image, the design method of the target image in the FPS target video is introduced in detail in the following embodiment. Since the design methods of several target images in the FPS target video are consistent, the embodiment takes the design of a single target image as an example for illustration, and the others are similar.

[0105] Reference Figure 2 is a design flowchart of the target image. The target image is designed according to the following steps:

[0106] Step 201, obtaining a blank image for making a target image, and dividing the blank image into N pixel blocks.

[0107] Among them, the target image has a target number, N≥4 and is a positive integer. The number of target numbers is determined according to the number of pixel blocks. For example Figure 1 If the pixel block in has four, the target number of the target image is four digits, including 0000-9999. Of course, the number of target images required is made according to actual demand, for example, 1000 target images are required, then 1000 target images with target numbers of 000-999 can be made.

[0108] Step 202, drawing a digital number in the first fixed area of each pixel block to represent the pixel block number of each pixel block.

[0109] Among them, the pixel block numbers of the N pixel blocks are sequentially combined into the target number of the target image. The pixel block number is used for storage, which will be described in detail later, and will not be described here.

[0110] In the specific implementation process, the corresponding digital number is selected from 0-9 according to the target number of the target image; the selected digital number is sequentially drawn in the first fixed area of each pixel block. For example, a target image with a target number of 9527 is required, then the digital numbers 9, 5, 2 and 7 are drawn in the first fixed area of each pixel block in the order from top to bottom and from left to right.

[0111] Step 203, replacing the second fixed area of each pixel block with a visual fiducial tag VFT coding pattern.

[0112] In the same pixel block, the decoded number of the VFT encoding pattern after decoding and the pixel block number are consistent, or the VFT encoding pattern is fixed and consistent in all target images and does not change with the switching of the target images.

[0113] Since the VFT encoding patterns drawn by different types of target images are different, their drawing methods are also different. If the decoded number of the VFT encoding pattern after decoding and the pixel block number are consistent in the same pixel block, during drawing, the corresponding VFT encoding pattern is selected from the set of VFT encoding patterns prepared in advance according to the target number of the target image or the pixel block number of each pixel block, and the second fixed area of each pixel block is replaced in sequence with the selected VFT encoding pattern. The VFT encoding pattern of the present embodiment has 10 patterns, corresponding to the VFT encoding patterns of the digital numbers 0-9. Therefore, the VFT encoding pattern with the corresponding number can be selected according to the target number of the target image or the pixel block number of each pixel block to be drawn in the second fixed area of the corresponding pixel block.

[0114] If the VFT encoding pattern is fixed and consistent in all target images and does not change with the switching of the target images, a randomly selected VFT encoding pattern is selected from the set of VFT encoding patterns prepared in advance to replace the second fixed area of each pixel block, and the VFT encoding pattern is fixed and consistent in all target images and does not change with the switching of the target images. The selected VFT encoding pattern is one or N, and the maximum value is equal to the number of pixel blocks. Taking the selection of one VFT encoding pattern as an example, one VFT encoding pattern can be randomly selected from the VFT encoding patterns corresponding to the digital numbers 0-9 to be drawn in the second fixed area of all target images, for example, the VFT encoding pattern corresponding to the digital number 0 is used in all target images, so that the VFT encoding pattern of all target images becomes 0000. Taking the selection of four different VFT encoding patterns as an example, the VFT encoding pattern corresponding to 9257 can also be randomly selected from the VFT encoding patterns corresponding to the digital numbers 0-9 to fill the second fixed area of each pixel block of all target images, so that the VFT encoding pattern in all target images is fixed as 9527 and does not change with the switching of the target images.

[0115] Step 204, drawing a gray scale encoding area in the third fixed area of each pixel block.

[0116] In the same pixel block, the gray scale value in the gray scale encoding area is used to represent the pixel block number.

[0117] In the process of drawing the gray scale coding area, a gray scale value queue with equal difference is designed, and the pixel value corresponding to each gray scale value in the equal difference gray scale value queue is obtained. For example, the equal difference gray scale value queue is 0, 28, 56, …, and the number of gray scale values in the equal difference gray scale value queue is 10, which are associated with the number 0-9. In the equal difference gray scale value queue, one gray scale value is associated with one number, for example, the gray scale value 0 is associated with the number 0, the gray scale value 28 is associated with the number 1, and so on. Further, each gray scale value in each equal difference gray scale value is expressed by a group of BGR pixel values. Since there are multiple groups of BGR pixel values expressing the same gray scale value, a group of BGR pixel values can be randomly selected. When drawing the gray scale coding area, according to the target number of the target image or the pixel block number of each pixel block, the corresponding pixel value is selected to replace the third fixed area of each pixel block to form the corresponding gray scale coding area. For example, in each pixel block, the third fixed area at the left end of the VFT coding pattern is replaced by the corresponding pixel value. Thus, the gray scale value of the gray scale coding area in the target image is calculated, and the target number of the target image is correspondingly obtained. Figure 1

[0118] The above is the production process of the target image. On this basis, the playing interval time of adjacent target images is calculated according to the number of target images and the preset refresh rate, and then the FPS target video is produced according to the target number of each target image and the playing interval time, which is used for subsequent test.

[0119] The present specification designs a target image containing a time stamp based on the recognition ability of machine vision to VFT, which can replace the traditional stopwatch (or similar functional tool) to record the camera exposure time with high precision. The synchronization error test of multiple cameras can be completed by simply playing the FPS target video with a screen tool, which has the condition of large-scale promotion to the industrial field.

[0120] Please refer to Figure 3 is a flowchart of a kind of automated test method for synchronization error of multiple cameras in the embodiment of the present specification, which comprises the following steps:

[0121] Step 301, control the display screen device to play the FPS target video produced in advance, so that each frame of target image in the FPS target video is displayed in sequence according to the playing interval time and the target number.

[0122] In the present embodiment, since the playing interval time of adjacent two frames of target images is consistent, each frame of target pattern is displayed with fixed playing interval time when playing.

[0123] ​In actual application, since the exposure of the multi-camera device is not known in advance, the FPS target video composed of the first type of target images is played first. If during the synchronous shooting, there is a case that multiple target images are superimposed and exposed as actual imaging images within a single frame exposure time due to a long camera exposure time, the played FPS target video needs to be switched. Specifically, the display screen device is controlled to play the FPS target video with the VFT coding pattern fixed and consistent in all target images, i.e., the FPS target video composed of the second type of target images is played, to adapt to the case that multiple target images are superimposed and exposed as actual imaging images within a single frame exposure time due to a long camera exposure time.

[0124] In step 302, the different cameras in the multi-camera device are controlled to synchronously and continuously shoot the played FPS target video to obtain respective actual imaging image sets.

[0125] In this embodiment, the view angle of each camera in the multi-camera device is controlled to be turned to the display screen playing the target pattern for shooting. During shooting, a trigger signal is generated to control the synchronous shooting of each camera, to ensure the synchronous action of the multi-camera. After the synchronous shooting, the multi-camera transmits the actual imaging images obtained by shooting to the processing device. If continuous shooting is performed, the actual imaging image sets corresponding to each camera are obtained and sequentially sent to the processing device. Of course, in order to improve the processing efficiency, each actual imaging image obtained by shooting can be sent to the processing device for synchronous error judgment. In this embodiment, each actual imaging image of each camera is transmitted through ROS (Robot Operating System) in the system, and the image corresponding trigger timestamp is written in the ros message (a message format that can be transmitted in the ROS system), so that each actual imaging image obtained by shooting carries the corresponding trigger timestamp, which represents the trigger time of the trigger signal triggering the shooting of each camera, to match the target images obtained by synchronous shooting. Since the exposure time of different cameras is different, under the same trigger timestamp, each camera can shoot target images of different target numbers.

[0126] In step 303, the processing device is controlled to perform image matching according to the trigger timestamp, and determine the relative timestamp of the actual imaging images of different cameras under the same trigger timestamp, and calculate the multi-camera synchronous error according to the relative timestamp of the actual imaging images.

[0127] In the embodiment, the processing device filters out unqualified images after receiving the actual imaging images taken by each camera. Specifically, a minimum threshold is set for the data bit intensity value of the VFT encoding pattern and the Hamming value of the target image, and the target image is filtered according to the minimum threshold to obtain a qualified target image. In the embodiment, qualified means that the data bit intensity value of the VFT encoding pattern and the Hamming value of the target image are respectively higher than the minimum threshold.

[0128] In the embodiment, if a single target image is obtained as the actual imaging image within a single frame exposure time due to a short camera exposure time at each time of synchronous shooting, the processing device matches the trigger time stamp to obtain a single target image of different cameras at the same trigger time stamp when calculating the synchronization error of the multiple cameras. Since the exposure time of different cameras is different, a single target image of different target numbers can be taken by each camera at the same trigger time stamp. Further, any two target cameras with a common view are selected from the multiple cameras, and the VFT encoding pattern in the single target images of the two target cameras is analyzed to obtain the target numbers of the single target images of the two target cameras; the relative time stamps of the two target cameras are determined according to the target numbers of the single target images of the two target cameras; and the difference between the relative time stamps of the two target cameras is taken as the synchronization error of the two target cameras. For example, the actual imaging images taken by two cameras at the time of synchronous shooting correspond to numbers 9526 and 9527, and the relative time stamps are 95260 ms and 95270 ms, respectively. The synchronization error of the two cameras is 10 ms, which is obtained by calculating the difference between the relative time stamps.

[0129] In the embodiment, if multiple target images are superimposed and exposed as the actual imaging image within a single frame exposure time due to a long camera exposure time at each time of synchronous shooting, the second type of target image is used to calculate the synchronization error of the multiple cameras. When calculating, the processing device matches the trigger time stamp to obtain the actual imaging images of different cameras at the same trigger time stamp. At this time, the actual imaging image is obtained by superimposing and exposing multiple target images.

[0130] In this case, since the actual imaging image is obtained by superimposed exposure of multiple target images, the relative time stamp cannot be obtained from the target image, and needs to be calculated. In this embodiment, the processing device has stored the second type target image in advance. The calculation method of the relative time stamp is described below taking a single camera as an example. For a single camera, the corresponding gray code area of the actual imaging image is located and searched through the VFT coding pattern, and the corresponding gray value is obtained. The average value of the gray value of the single camera in the gray code area is calculated to obtain the final gray value of the single camera in the gray code area. The final gray value is used to search the gray value of the multiple superimposed exposure target images, the target image with the closest gray value is determined, and the relative time stamp of the target image with the closest gray value is taken as the relative time stamp of the actual imaging image.

[0131] Specifically, since the actual imaging image of the embodiment is obtained by superimposed exposure of multiple target images, when the camera captures the actual imaging image, the gray code area corresponding to the actual imaging image is also the gray code area formed by the superimposed gray code areas of the multiple target images, and the gray code area corresponding to the actual imaging image can be regarded as the average value of the gray code areas of the multiple target images. Further, since the gray values of the gray code area obtained by the camera when capturing the actual imaging image may have certain errors due to environmental light or other factors, the average value of the gray values of the gray code area needs to be calculated. The multiple superimposed exposure target images are searched according to the average value of the gray values of the gray code area, the target image with the closest gray value is determined, and the relative time stamp of the target image with the closest gray value is taken as the relative time stamp of the actual imaging image. For example, taking the equal difference gray values 0, 28, 56, … as an example, if the camera superimposes three target images numbered 9 (252) 5 (140) 2 (56) 6 (168), 9 (252) 5 (140) 2 (56) 7 (196), and 9 (252) 5 (140) 2 (56) 8 (224) to become an actual imaging image A, wherein the gray values of the gray code areas are in the brackets. The average value of the gray values of the gray code areas of the actual imaging image A is calculated. Specifically, the average value of the gray values of the gray code areas corresponding to each pixel block of the actual imaging image A is calculated. Since the gray values of the actual imaging image A have certain differences with the original target image gray values due to external environmental factors (for example, the actual imaging image A is slightly brighter, and the gray values of each pixel point are larger), the average values of the gray values of each gray code area of the actual imaging image A are calculated, which are (from top to bottom, left to right): 255, 150, 72, and 200. The gray values of the gray code areas of the three target images 9526, 9527, and 9528 are searched according to the average values of the gray values of the gray code areas, and the target image closest to 9527 is determined, and the relative time stamp 9527 ms of the target image is taken as the relative time stamp of the actual imaging image.

[0132] Further, after determining the target image with the closest gray value, the size of the gray value difference between the actual imaging image and the target image with the closest gray value and the preset difference threshold value is calculated. If it is higher than the preset difference threshold value, it indicates that the actual imaging image and the target image with the closest gray value are too far apart, the actual imaging image is directly filtered out, and the actual imaging image obtained by the next trigger is used for synchronization error calculation. It is worth noting that as long as the actual imaging image of one camera is filtered out, the actual imaging images of other cameras at the same trigger time are also filtered out, and all cameras use the actual imaging image obtained by the next trigger for synchronization error calculation.

[0133] It is worth noting that since each target image contains N gray scale encoding areas, in the process of obtaining the final gray value of a single camera corresponding to the gray scale encoding area by averaging the gray values of the N gray scale encoding areas of the single camera, the final gray value of the single camera corresponding to the N gray scale encoding areas is obtained by averaging the gray values of the N gray scale encoding areas of the single camera.

[0134] Similarly, since each target image contains N gray scale encoding areas, in the process of searching the gray values of the superimposed exposure target images using the final gray value, the target image with the closest gray value is determined, the final gray value corresponding to the N gray scale encoding areas of the actual imaging image is used to sequentially search the gray values of each target image, and the target image with the most closest gray values is determined as the closest target image. According to the above example, among the three target images 9526, 9527, and 9528, the target images with target numbers 9526 and 9528 have three closest gray values with the actual imaging image, and the target image with target number 9527 has four closest gray values with the actual imaging image. Therefore, the closest target image is determined to be 9527, and the relative timestamp 9527 ms is used as the relative timestamp of the actual imaging image.

[0135] The multi-camera synchronization error is calculated according to the relative timestamps of the actual imaging images of different cameras. Specifically, the difference between the relative timestamps of the actual imaging images of two cameras is used as the synchronization error of the two cameras.

[0136] The embodiment can overcome the defect that the multi-camera synchronization error cannot be calculated when the exposure time is too long by using the relative timestamp corresponding to the target image with the closest gray value as the relative timestamp of the actual imaging image. The multi-camera synchronization error can be accurately calculated even when the exposure time is too long and multiple target images are superimposed.

[0137] In this embodiment, the digital code in the target image can be used as a reference when storing the name, thereby playing a distinguishing role. Specifically, the two target images with the largest synchronization error are calculated and stored respectively, and the image storage name can be "target number.jpg", and of course can also be "relative timestamp.jpg", which is used to mark the target image calculated by the gray code area to calculate the synchronization error, so as to facilitate subsequent manual review. Of course, this storage method can also be used when calculating the synchronization error by using the VFT coding pattern.

[0138] The above is the implementation process of the automatic test of the multi-camera synchronization error in this embodiment. The multi-camera synchronization self-checking can be efficiently and automatically performed, and a multi-camera synchronization error detection report can be output, thereby providing a feasible digital solution for multi-camera acquisition devices in large-scale industrialization. In the test, the samples that do not meet the conditions can be automatically filtered, the scheme is highly intelligent (no one is needed during testing), and the test efficiency and accuracy are greatly improved.

[0139] The scheme in this specification has strong anti-interference ability in the test process, and the disturbance in the environment does not affect the positioning and identification of the VFT. The target coding of this specification has strong expandability, can be made into high-radix bit coding, has no manufacturing cost, and only needs to be played on a regular display screen, does not need a large number of LEDs to form an array, and can be accurately positioned and accurately time without relying on manual operation.

[0140] The scheme in this specification has a very high recognition rate when facing self-developed targets, and can filter out images that do not meet the quality through multiple indicators. The test success rate is much higher than that of the original scheme.

[0141] The scheme in this specification does not have the prerequisite of "requiring stable camera frequency", and has no special requirements for the test space and environmental brightness. The switching state of the self-developed target is controllable.

[0142] The scheme in this specification does not need much preparation work in the test stage, and has no special requirements for the site. Only the pre-generated FPS target video needs to be played on the screen, and the multi-camera synchronization error test of the multi-camera acquisition device (one set or even multiple sets) can be completed at the same time. The influence of bad samples is effectively excluded, and the precision can reach the precision of the interval of the switching target pattern in the used screen.

[0143] Based on the same inventive concept, the following embodiment introduces an automatic test system for multi-camera synchronization error. Referring to Figure 4 , the system includes a display screen device 401, a multi-camera device 402, and a processing device 403. The field of view of each camera (camera 1, camera 2, …) in the multi-camera device 402 is focused on the display screen device 401. The multi-camera device 402 and the processing device 403 can exchange information depending on ROS.

[0144] A display device 401 is configured to play a pre-produced FPS target video, so that each frame of target image in the FPS target video is displayed in sequence according to a playing interval time and a target number; wherein each frame of target image in the FPS target video is designed with a respective relative timestamp according to the target number;

[0145] Different cameras in a multi-camera device 402 are configured to synchronously and continuously capture the played FPS target video to obtain a respective set of actual imaging images; wherein each actual imaging image carries a corresponding trigger timestamp;

[0146] A processing device 403 is configured to perform image matching according to the trigger timestamp, and determine the relative timestamps of the actual imaging images of different cameras at the same trigger timestamp, and calculate a multi-camera synchronization error according to the relative timestamps of the actual imaging images.

[0147] In an optional embodiment, each target image in the FPS target video comprises N pixel blocks, where N is a positive integer greater than or equal to 4;

[0148] A first fixed area of each pixel block is drawn with a pixel block number; and the pixel block numbers of the N pixel blocks are combined in sequence to form the target number of the target image;

[0149] A second fixed area of each pixel block is drawn with a respective VFT encoding pattern; in the same pixel block, the decoded number decoded from the VFT encoding pattern is consistent with the pixel block number, or the VFT encoding pattern is fixed and consistent in all target images and does not change with the switching of the target image;

[0150] A third fixed area of each pixel block is drawn with a gray scale encoding area, and the gray scale value in the gray scale encoding area is used to represent the pixel block number.

[0151] In an optional embodiment, the system further comprises a target image design device configured to design the target image in the FPS target video according to the following steps:

[0152] A blank image used to produce the target image is obtained, and the blank image is equally divided into N pixel blocks; wherein the target image has a target number, and N is a positive integer greater than or equal to 4;

[0153] A digital number is drawn in a first fixed area of each pixel block to represent the pixel block number of each pixel block; wherein the pixel block numbers of the N pixel blocks are combined in sequence to form the target number of the target image;

[0154] replacing the second fixed area of each pixel block with a visual fiducial tag (VFT) encoding pattern; in the same pixel block, the decoded number of the VFT encoding pattern is consistent with the pixel block number, or the VFT encoding pattern is fixed and consistent in all target images and does not change with the switching of target images;

[0155] drawing a gray scale encoding area in the third fixed area of each pixel block; in the same pixel block, the gray scale value in the gray scale encoding area is used to represent the pixel block number.

[0156] In an alternative embodiment, the target image design device is particularly used for:

[0157] selecting a corresponding digital number from 0 to 9 according to the target number of the target image;

[0158] sequentially drawing the selected digital number in the first fixed area of each pixel block.

[0159] In an alternative embodiment, the target image design device is particularly used for:

[0160] selecting a corresponding VFT encoding pattern from a set of VFT encoding patterns prepared in advance according to the target number of the target image or the pixel block number of each pixel block, and sequentially replacing the second fixed area of each pixel block with the selected VFT encoding pattern; or

[0161] selecting a randomly selected VFT encoding pattern from a set of VFT encoding patterns prepared in advance to replace the second fixed area of each pixel block, and making the VFT encoding pattern fixed and consistent in all target images and not changing with the switching of target images.

[0162] In an alternative embodiment, the target image design device is particularly used for:

[0163] designing an arithmetic gray scale value queue and obtaining a pixel value corresponding to each gray scale value in the arithmetic gray scale value queue; in the arithmetic gray scale value queue, one gray scale value is associated with one digital number;

[0164] selecting a corresponding pixel value to replace the third fixed area of each pixel block according to the target number of the target image or the pixel block number of each pixel block, to form a corresponding gray scale encoding area.

[0165] In an alternative embodiment, the system further comprises a video design device for:

[0166] calculating the playing interval time of adjacent target images according to the number of target images and a preset refresh rate;

[0167] The FPS target video is produced according to the target numbers and the playing interval time of each target image.

[0168] In an alternative embodiment, if, at each time of synchronous shooting, only a single target image is obtained as an actual imaging image within a single frame exposure time due to a short camera exposure time, the processing device 403 is specifically configured to:

[0169] The processing device 403 is controlled to match according to the trigger time stamp, so as to obtain a single target image of different cameras at the same trigger time stamp;

[0170] The VFT encoding pattern in the single target image of any two target cameras with a common view field is analyzed, so as to obtain the target numbers of the single target images of the two target cameras;

[0171] The relative time stamps of the two target cameras are determined according to the target numbers of the single target images of the two target cameras;

[0172] The difference between the relative time stamps of the two target cameras is taken as the synchronization error of the two target cameras.

[0173] In an alternative embodiment, if, at each time of synchronous shooting, multiple target images are superimposed and exposed as an actual imaging image within a single frame exposure time due to a long camera exposure time, the display screen device 401 is specifically configured to control the display screen device 401 to play the FPS target video in which the VFT encoding pattern is fixed and consistent in all target images.

[0174] In an alternative embodiment, if, at each time of synchronous shooting, multiple target images are superimposed and exposed as an actual imaging image within a single frame exposure time due to a long camera exposure time, the processing device 403 is specifically configured to:

[0175] The processing device 403 is controlled to match according to the trigger time stamp, so as to obtain an actual imaging image of different cameras at the same trigger time stamp;

[0176] For a single camera, a corresponding gray scale encoding area of the actual imaging image is located and searched through the VFT encoding pattern, so as to obtain a corresponding gray scale value; the single camera is subjected to mean value processing on the gray scale value corresponding to the gray scale encoding area, so as to obtain a final gray scale value corresponding to the gray scale encoding area of the single camera; the final gray scale value is used to search the gray scale values of the multiple superimposed and exposed target images, so as to determine a target image with a closest gray scale value, and the relative time stamp of the target image with the closest gray scale value is taken as the relative time stamp of the actual imaging image;

[0177] The multi-camera synchronization error is calculated according to the relative time stamps of the actual imaging images of the different cameras.

[0178] Based on the same inventive concept, the following embodiment introduces a target image for the test method described in the foregoing embodiments; and the specific structure of the target image is described in detail in the foregoing embodiments. Therefore, it will not be repeated here.

[0179] Based on the same inventive concept, the following embodiment introduces an FPS target video made by the target pattern of the foregoing embodiments. The specific structure of the FPS target video is described in detail in the foregoing embodiments. Therefore, it will not be repeated here.

[0180] Through one or more embodiments of the present specification, the present specification has the following beneficial effects or advantages:

[0181] The present specification discloses a kind of multi-camera synchronization error automated test method and system, in the method, control display screen device plays the frame per second FPS target video made in advance, to make each frame target image in the FPS target video sequentially switch display according to playing interval time and target number;Wherein, each frame target image in the FPS target video is designed with respective relative time stamp according to target number;Control different camera in multi-camera device synchronously continuously photograph the FPS target video played, obtain respective actual imaging image set;Wherein, each actual imaging image carries corresponding trigger time stamp;Control processing device carries out image matching according to trigger time stamp, and determine the relative time stamp of actual imaging image of different camera under the same trigger time stamp, calculate multi-camera synchronization error according to the relative time stamp of actual imaging image.The scheme of the present specification starts from the actual imaging error angle, under the condition of synchronous shooting, the relative time stamp carried by each target image is detected to synchronize error, with the characteristics of low cost but high precision and high efficiency.In addition, there is no need for more preparation work in the test stage, and there is no special requirement for site, only need to play the FPS target video made in advance on screen, it can complete (even multiple sets) multi-camera synchronization error test in multi-camera acquisition device at the same time, save cost at the same time standardize the detection process and standard of multi-camera synchronization, with the condition of large-scale promotion to industrial field.

[0182] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems can be used with these teachings, based on the description as provided herein. In addition, the present specification is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages can be used to implement the teachings of the present specification as described herein, and any references below to specific languages are provided for disclosure of enablement only.

[0183] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the description can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been described in detail in order not to obscure the understanding of this description.

[0184] Similarly, it is to be understood that the embodiments of the description can be used in other ways, and that the individual features of the description can be claimed in combinations other than the combinations explicitly stated herein. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0185] Those skilled in the art will appreciate that the modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and further can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination of all the features disclosed in the description, including the accompanying claims, abstract and drawings, and all the processes or units of any method or apparatus disclosed so far can be adopted. Unless explicitly stated otherwise, each feature disclosed in the description, including the accompanying claims, abstract and drawings, can be replaced by an alternative feature providing the same, equivalent or similar function.

[0186] Further, those skilled in the art will appreciate that, although some embodiments herein include certain features of other embodiments but not others, combinations of the features of the different embodiments are to be expected and are within the scope of the description and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0187] Various component embodiments of the present specification can be implemented in hardware, or as software modules running in one or more processors, or combinations thereof. Those skilled in the art will appreciate that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functionality of some or all of the components in the gateway, the proxy server, the system according to the embodiments of the present specification. The present specification can also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for performing part or all of the methods described herein. Such a program implementing the present specification can be stored on a computer readable medium or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or in any other form.

[0188] It should be noted that the above-mentioned embodiments illustrate rather than limit the present specification, and that one skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps not listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The specification can be implemented by means of both hardware and software, and any combinations thereof. In a unit claim, several devices can be listed with a conjunction like 'or', and a unit can be claimed even if several of the devices are merely optional. Individual components of the specification can be identified by the use of the word 'first','second', etc., but these words are not intended to have any order or sequence meaning.

Claims

1. An automated testing method for multi-camera synchronization errors, the method comprising: Controlling the display screen device to play a pre-made FPS target video, so that each target image frame in the FPS target video is switched and displayed in sequence according to the playback interval and the target number; wherein each target image frame in the FPS target video is designed with a respective relative timestamp according to the target number; the target image in the FPS target video includes N pixel blocks, N ≥ 4 and is a positive integer; a pixel block number is drawn in a first fixed area of ​​each pixel block; the pixel block numbers of the N pixel blocks are sequentially combined to form the target number of the target image; a second fixed area of ​​each pixel block has a respective VFT coding pattern; in the same pixel block, the decoded number of the VFT coding pattern after decoding is consistent with the pixel block number, or the VFT coding pattern is fixed and consistent in all target images and does not change with the switching of the target image; a grayscale coding area is drawn in a third fixed area of ​​each pixel block, and the grayscale value in the grayscale coding area is used to represent the pixel block number; Controlling different cameras in the multi-camera device to synchronously and continuously shoot and play the FPS target video to obtain respective actual imaging image sets; wherein each actual imaging image carries a corresponding trigger timestamp; The control processing device performs image matching according to the trigger timestamp, determines the relative timestamps of actual imaging images of different cameras at the same trigger timestamp, and calculates the multi-camera synchronization error according to the relative timestamps of the actual imaging images.

2. The method according to claim 1, wherein the target image in the FPS target video is designed according to the following steps: Obtain a blank image for producing the target image, and divide the blank image into N pixel blocks; wherein, The target image has a target number, N ≥ 4 and is a positive integer; Drawing a digital number in the first fixed area of ​​each pixel block to represent the pixel block number of each pixel block; wherein the pixel block numbers of the N pixel blocks are sequentially combined to form the target number of the target image; The second fixed area of ​​each pixel block is replaced by a visual reference tag (VFT) coding pattern; in the same pixel block, the decoded number of the VFT coding pattern after decoding remains consistent with the pixel block number, or the VFT coding pattern is fixed and consistent in all target images and does not change with the switching of target images; A grayscale coding area is drawn in the third fixed area of ​​each pixel block; in the same pixel block, the grayscale value in the grayscale coding area is used to represent the pixel block number.

3. The method according to claim 2, wherein the step of drawing the digital number in the first fixed area of ​​each pixel block comprises: Select a corresponding digital number from 0 to 9 according to the target number of the target image; The selected digital numbers are sequentially drawn in the first fixed area of ​​each pixel block.

4. The method of claim 2, wherein replacing the second fixed region of each pixel block with a visual reference tag (VFT) encoding pattern comprises: selecting a corresponding VFT coding pattern from a pre-prepared VFT coding pattern set according to the target number of the target image or the pixel block number of each pixel block, and sequentially replacing the second fixed area of ​​each pixel block with the selected VFT coding pattern; or A randomly selected VFT coding pattern is selected from a pre-prepared VFT coding pattern set to replace the second fixed area of ​​each pixel block, and the VFT coding pattern is fixed and consistent in all target images and does not change with the switching of the target image.

5. The method according to claim 2, wherein drawing the grayscale coding area in the third fixed area of ​​each pixel block comprises: Designing an equidistant grayscale value array and obtaining the pixel value corresponding to each grayscale value in the equidistant grayscale value array; In the equidistant gray value queue, one gray value is associated with one digital number; According to the target number of the target image or the pixel block number of each pixel block, a corresponding pixel value is selected to replace the third fixed area of ​​each pixel block to form a corresponding grayscale coding area.

6. The method of claim 2, further comprising: The playback interval between adjacent target images is calculated based on the number of target images and the preset refresh rate; The FPS target video is produced according to the target number and playback interval of each target image.

7. The method of claim 1, wherein if, during each synchronous shooting, due to a short camera exposure time, only a single target image is obtained within a single frame exposure time as the actual imaged image, the control processing device performs image matching according to the trigger timestamp, determines relative timestamps of actual imaged images of different cameras at the same trigger timestamp, and calculates the multi-camera synchronization error based on the relative timestamps of the actual imaged images, comprising: Controlling the processing device to perform matching according to the trigger timestamp to obtain single target images taken by different cameras at the same trigger timestamp; Parsing the VFT coding pattern in the single target images of any two target cameras with a common field of view to obtain the target numbers of the single target images of the two target cameras; Determining respective relative timestamps according to target numbers of the single target images of the two target cameras; The difference between the relative time stamps of the two target cameras is used as the synchronization error of the two target cameras.

8. The method of claim 1, wherein, if during each synchronous shooting, due to a long camera exposure time, multiple target images are superimposed within a single frame exposure time to be exposed as the actual imaging image, the controlling display device to play a pre-made target video with a transmission frame rate of FPS specifically comprises: The display screen device is controlled to play the FPS target video in which the VFT coding pattern is fixed and consistent in all target images.

9. The method of claim 1, wherein if, during each synchronized shooting, due to a long camera exposure time, multiple target images are superimposed within a single frame exposure time as an actual image, the control processing device performs image matching according to the trigger timestamps, determines relative timestamps of actual images of different cameras at the same trigger timestamp, and calculates the multi-camera synchronization error based on the relative timestamps of the actual images, including: Controlling the processing device to perform matching according to the trigger timestamp to obtain actual imaging images of different cameras at the same trigger timestamp; For a single camera, locate and search the grayscale coding area corresponding to the actual imaging image using the VFT coding pattern to obtain the corresponding grayscale value; average the grayscale values ​​corresponding to the grayscale coding area of ​​the single camera to obtain the final grayscale value corresponding to the grayscale coding area of ​​the single camera; use the final grayscale value to traverse and search the grayscale values ​​of multiple target images with superimposed exposures to determine the target image with the closest grayscale value, and use the relative timestamp corresponding to the target image with the closest grayscale value as the relative timestamp of the actual imaging image; The multi-camera synchronization error is calculated according to the relative time stamps of the actual imaging images of the different cameras.

10. An automated testing system for multi-camera synchronization errors, the system comprising: A display screen device, for playing a pre-produced frame-per-second (FPS) target video, so that each target image frame in the FPS target video is switched and displayed in sequence according to the playback interval time and the target number; wherein each target image frame in the FPS target video is designed with a respective relative timestamp according to the target number; the target image in the FPS target video includes N pixel blocks, N ≥ 4 and is a positive integer; a pixel block number is drawn in a first fixed area of ​​each pixel block; the pixel block numbers of the N pixel blocks are sequentially combined to form the target number of the target image; a second fixed area of ​​each pixel block has a respective VFT coding pattern; in the same pixel block, the decoded number of the VFT coding pattern after decoding is consistent with the pixel block number, or the VFT coding pattern is fixed and consistent in all target images and does not change with the switching of the target image; a grayscale coding area is drawn in a third fixed area of ​​each pixel block, and the grayscale value in the grayscale coding area is used to represent the pixel block number; Different cameras in the multi-camera device are used to synchronously and continuously shoot the FPS target video to obtain respective sets of actual imaging images; wherein each actual imaging image carries a corresponding trigger timestamp; The processing device is used to perform image matching according to the trigger timestamp, determine the relative timestamps of the actual imaging images of different cameras under the same trigger timestamp, and calculate the multi-camera synchronization error based on the relative timestamps of the actual imaging images.

11. The system of claim 10, further comprising: The target image design device is used to design the target image in the FPS target video according to the following steps: Obtaining a blank image for producing the target image, and dividing the blank image into N pixel blocks; wherein the target image has a target number, N ≥ 4 and is a positive integer; Drawing a digital number in the first fixed area of ​​each pixel block to represent the pixel block number of each pixel block; wherein the pixel block numbers of the N pixel blocks are sequentially combined to form the target number of the target image; The second fixed area of ​​each pixel block is replaced by a visual reference tag (VFT) coding pattern; in the same pixel block, the decoded number of the VFT coding pattern after decoding remains consistent with the pixel block number, or the VFT coding pattern is fixed and consistent in all target images and does not change with the switching of target images; A grayscale coding area is drawn in the third fixed area of ​​each pixel block; in the same pixel block, the grayscale value in the grayscale coding area is used to represent the pixel block number.

12. The system according to claim 11, wherein the target image design device is specifically configured to: Select a corresponding digital number from 0 to 9 according to the target number of the target image; The selected digital numbers are sequentially drawn in the first fixed area of ​​each pixel block.

13. The system according to claim 11, wherein the target image design device is specifically configured to: selecting a corresponding VFT coding pattern from a pre-prepared VFT coding pattern set according to the target number of the target image or the pixel block number of each pixel block, and sequentially replacing the second fixed area of ​​each pixel block with the selected VFT coding pattern; or A randomly selected VFT coding pattern is selected from a pre-prepared VFT coding pattern set to replace the second fixed area of ​​each pixel block, and the VFT coding pattern is fixed and consistent in all target images and does not change with the switching of the target image.

14. The system according to claim 11, wherein the target image design device is specifically configured to: Designing an equidistant grayscale value queue and obtaining a pixel value corresponding to each grayscale value in the equidistant grayscale value queue; in the equidistant grayscale value queue, each grayscale value is associated with a digital number; According to the target number of the target image or the pixel block number of each pixel block, a corresponding pixel value is selected to replace the third fixed area of ​​each pixel block to form a corresponding grayscale coding area.

15. The system according to claim 11, further comprising a video design device for: The playback interval between adjacent target images is calculated based on the number of target images and the preset refresh rate; The FPS target video is produced according to the target number and playback interval of each target image.

16. The system of claim 10, wherein if, during each synchronous shooting, the camera exposure time is short and only a single target image is obtained as the actual imaging image within the single-frame exposure time, the processing device is specifically configured to: Controlling the processing device to perform matching according to the trigger timestamp to obtain single target images taken by different cameras at the same trigger timestamp; Parsing the VFT coding pattern in the single target images of any two target cameras with a common field of view to obtain the target numbers of the single target images of the two target cameras; Determining respective relative timestamps according to target numbers of the single target images of the two target cameras; The difference between the relative time stamps of the two target cameras is used as the synchronization error of the two target cameras.

17. The system as described in claim 10, if during each synchronous shooting, due to the long exposure time of the camera, multiple target images are superimposed and exposed within a single frame exposure time as the actual imaging image, the display screen device is specifically used to control the display screen device to play the FPS target video in which the VFT coding pattern is fixed and consistent in all target images.

18. The system of claim 10, wherein if, during each synchronous shooting, the camera exposure time is long, multiple target images are superimposed and exposed within a single frame exposure time as the actual imaging image, the processing device is specifically configured to: Controlling the processing device to perform matching according to the trigger timestamp to obtain actual imaging images of different cameras at the same trigger timestamp; For a single camera, locate and search the grayscale coding area corresponding to the actual imaging image using the VFT coding pattern to obtain the corresponding grayscale value; average the grayscale values ​​corresponding to the grayscale coding area of ​​the single camera to obtain the final grayscale value corresponding to the grayscale coding area of ​​the single camera; use the final grayscale value to traverse and search the grayscale values ​​of multiple target images with superimposed exposures to determine the target image with the closest grayscale value, and use the relative timestamp corresponding to the target image with the closest grayscale value as the relative timestamp of the actual imaging image; The multi-camera synchronization error is calculated according to the relative time stamps of the actual imaging images of the different cameras.

19. A target image, the target image being used to perform the test method according to any one of claims 1 to 9; the target image comprising N pixel blocks, where N is a positive integer and is greater than or equal to 4; A pixel block number is drawn in the first fixed area of ​​each pixel block; the pixel block numbers of the N pixel blocks are sequentially combined to form a target number of the target image; The second fixed area of ​​each pixel block has its own VFT coding pattern; in the same pixel block, the decoded number of the VFT coding pattern after decoding remains consistent with the pixel block number, or the VFT coding pattern is fixed and consistent in all target images and does not change with the switching of target images; A grayscale coding area is drawn in the third fixed area of ​​each pixel block, and the grayscale value in the grayscale coding area is used to represent the pixel block number.

20. An FPS target video, wherein the FPS target video is produced by using the target image according to claim 19.

Images