Multi-angle camera video picture synchronization method and computer readable storage medium

Abstract

The application relates to the technical field of multi-angle camera video picture synchronization, and discloses a multi-angle camera video picture synchronization method and a computer readable storage medium, the method is as follows: acquiring a picture library, pictures in the picture library have time stamps; traversing the picture library, establishing an original coordinate list according to the maximum time stamp and the minimum time stamp; mapping the pictures in the picture library to the original coordinate list to generate a reference coordinate list and a plurality of synchronization coordinate lists; synchronously comparing the original coordinate list and the synchronization coordinate list, extracting all the pictures successfully synchronized from the picture library according to a synchronous comparison result, and storing the pictures in a result file; and uploading and storing the result file to a synchronization library. The application is used to solve the problem that in the prior art, only time stamps are used to synchronize the video frame pictures of multiple angles, due to the excessively large picture data, when synchronization is performed, each reference picture needs to traverse all the pictures once, and the time cost of traversing all the pictures multiple times is large.

Description

Technical Field

[0001] This invention relates to the field of multi-angle camera video and image synchronization technology, and in particular to a multi-angle camera video and image synchronization method and a computer-readable storage medium. Background Technology

[0002] In the current technology, the relevant algorithms for autonomous driving are developing rapidly. Among them, Level 3 autonomous driving requires the support of machine vision algorithms, and the performance of machine vision algorithms depends to a large extent on the quantity and quality of data in the dataset.

[0003] Currently, major automakers are using their own data collection vehicles to gather massive datasets for algorithm training. These vehicles are equipped with cameras at multiple angles to collect data, and the resulting videos are ultimately extracted into frames for training. Because the number of images extracted from high-definition cameras is enormous, it's necessary to synchronize these multi-view images. Relying on a single traversal matching algorithm for image synchronization results in a large number of loops, high time complexity, and is extremely time-consuming and labor-intensive.

[0004] Chinese Invention Patent Publication No. [CN112017216A] discloses an image processing method, apparatus, computer-readable storage medium, and computer device. For the synchronization module of the image processing method, it proposes to synchronize multiple angle video frame sequences according to the shooting timestamp.

[0005] This method relies solely on timestamps to synchronize video frames from multiple angles. However, due to the large amount of image data, each reference image needs to be traversed once during synchronization, resulting in significant time overhead from multiple traversals. Furthermore, due to limitations in the camera's hardware, timestamps from videos captured by different cameras may not perfectly match at the millisecond level in scenarios requiring high precision, leading to image synchronization failures. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a method for synchronizing video images from multiple angle cameras and a computer-readable storage medium, which solves the problem in the prior art that relies solely on timestamps to synchronize video frame images from multiple angles. Due to the large amount of image data, each reference image needs to be traversed once during synchronization, resulting in a large time overhead from traversing all images multiple times.

[0007] The present invention solves the above-mentioned technical problems through the following technical means:

[0008] In a first aspect, embodiments of this application provide a method for synchronizing video and images from a multi-angle camera, the method being:

[0009] Obtain an image library, wherein the images in the image library have timestamps;

[0010] Iterate through the image library and build a list of original coordinates based on the maximum and minimum timestamps;

[0011] Map the images in the image library to the original coordinate list to generate a baseline coordinate list and multiple synchronized coordinate lists;

[0012] The original coordinate list is compared with the synchronized coordinate list. Based on the comparison results, all successfully synchronized images are extracted from the image library and stored in the result file.

[0013] Upload and store the result files to the synchronization library.

[0014] In conjunction with the first aspect, in some optional implementations, the specific method for obtaining the image library is as follows:

[0015] Frame extraction is performed on the video stream from each camera to obtain images with timestamps, and all images are uploaded to the library to form an image library.

[0016] In conjunction with the first aspect, in some optional implementations, the specific method for traversing the image library and establishing the original coordinate list based on the maximum and minimum timestamps is as follows:

[0017] Iterate through all images in the image library and obtain the maximum and minimum timestamps;

[0018] The maximum and minimum timestamps are integerized to obtain the maximum integerized timestamp and the minimum integerized timestamp;

[0019] A list of original coordinates of length m is created based on the maximum and minimum integer timestamps, where each element in the list is "0", and m = maximum integer timestamp - minimum integer timestamp.

[0020] In conjunction with the first aspect, in some optional implementations, the specific method for mapping images in the image library to the original coordinate list to generate a reference coordinate list and multiple synchronized coordinate lists is as follows:

[0021] Obtain a baseline file and multiple synchronization files, wherein the baseline file includes several baseline images and the synchronization files include several synchronization images;

[0022] Map several of the aforementioned reference images onto the original coordinate list to generate a reference coordinate list;

[0023] Map the synchronized images in each synchronized file to the original coordinate list to generate multiple synchronized coordinate lists.

[0024] In conjunction with the first aspect, in some optional implementations, the specific method for obtaining the base file and multiple synchronization files is as follows:

[0025] Based on the number of cameras, create image files corresponding to the number of cameras in the image library, and store all images in the image library in the corresponding image files.

[0026] Designate one of multiple cameras as the reference camera, and the rest as synchronization cameras;

[0027] From the image files corresponding to the reference camera, select the images that need to be synchronized as reference images, and store all the reference images in the same file to obtain the reference file;

[0028] The image file corresponding to the synchronized camera is used as the synchronization file, and the images in the synchronization file are used as the synchronized images.

[0029] In conjunction with the first aspect, in some optional implementations, a plurality of the aforementioned reference images are mapped onto an original coordinate list to generate a reference coordinate list; the specific method for mapping the synchronization images in each synchronization file to the original coordinate list to generate multiple synchronization coordinate lists is as follows:

[0030] Iterate through the baseline images in the baseline file, map the baseline image to the corresponding position in the original coordinate list according to the timestamp of each baseline image, and change the element "0" at that position to element "1", generating a baseline coordinate list containing elements "0" and "1".

[0031] Iterate through the synchronized images in the synchronized files. Based on the timestamp of each synchronized image, map the synchronized image to the corresponding position in the original coordinate list, and change the element "0" at that position to element "1". Generate a synchronized coordinate list containing elements "0" and "1". Iterate through all synchronized files in the image library in turn to generate multiple synchronized coordinate lists containing elements "0" and "1".

[0032] In conjunction with the first aspect, in some optional implementations, the original coordinate list and the synchronized coordinate list are compared synchronously. Based on the synchronization comparison results, all successfully synchronized images are extracted from the image library and stored in the result file. The specific method is as follows:

[0033] Perform a bitwise AND operation between the reference coordinate list and multiple synchronized coordinate lists to obtain a fully synchronized coordinate list.

[0034] Based on the position of element "1" in the fully synchronized coordinate list, the corresponding fully synchronized timestamp is parsed out;

[0035] Based on the fully synchronized timestamp, the corresponding synchronized images are extracted from each synchronized file as the synchronized result images, and all the base images and synchronized result images are stored in the result file.

[0036] In conjunction with the first aspect, in some optional implementations, the original coordinate list and the synchronized coordinate list are compared synchronously. Based on the synchronization comparison results, all successfully synchronized images are extracted from the image library and stored in the result file. The specific method is as follows:

[0037] Based on the position of element "1" in the reference coordinate list, each synchronous coordinate list is sliced ​​according to a preset slicing range to obtain several coordinate list fragments.

[0038] If the coordinate list fragment contains an element "1", then parse the non-fully synchronized timestamp corresponding to the element "1".

[0039] Determine the synchronization file corresponding to the coordinate list segment, extract the corresponding synchronization image from the synchronization file based on the incomplete synchronization timestamp, and use the image as the incomplete synchronization result image;

[0040] All baseline images and non-fully synchronized result images are stored in the result file.

[0041] Thirdly, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is run on a computer, the computer causes the computer to perform the method of the first aspect.

[0042] The beneficial effects of this invention are:

[0043] 1. This invention establishes an original coordinate list using the maximum and minimum timestamps, and maps the reference image and synchronized images to the original coordinate list respectively, generating a reference coordinate list and multiple synchronized coordinate lists. Simply compare the reference coordinate list and the multiple synchronized coordinate lists during synchronization, and based on the comparison results, extract all successfully synchronized images from the image library and store them in the result file. Compared to existing technologies, synchronization does not require multiple traversals of all images, effectively reducing the time overhead of image synchronization.

[0044] 2. This invention obtains a fully synchronized image by performing a bitwise AND operation between a reference coordinate list and multiple synchronized coordinate lists. By analyzing the position of the element "1" in the reference coordinate list, each synchronized coordinate list is sliced ​​according to a preset slicing range to obtain several coordinate list fragments. If an element "1" exists within a coordinate list fragment, a partially synchronized image is obtained. Compared to previous technologies, this avoids situations where timestamps at the millisecond level cannot be perfectly matched, leading to image synchronization failures in scenarios with high precision requirements. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating a method for synchronizing video and images from a multi-angle camera according to the present invention.

[0046] Figure 2 This is a flowchart illustrating step 400 in Embodiment 1 of the multi-angle camera video and image synchronization method of the present invention;

[0047] Figure 3 This is a flowchart illustrating step 400 in Embodiment 2 of the multi-angle camera video and image synchronization method of the present invention; Detailed Implementation

[0048] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can understand the advantages and effects of the present invention from the content disclosed in this specification. It should be noted that the illustrations provided in the following embodiments are for illustrative purposes only and represent schematic diagrams, not actual pictures, and should not be construed as limiting the invention. In order to better illustrate the embodiments of the present invention, some parts in the figures may be omitted, enlarged, or reduced, and do not represent the actual product size; it is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the figures.

[0049] In the figures of this invention, the same or similar reference numerals correspond to the same or similar components. In the description of this invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the figure, they are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the figures are only for illustrative purposes and should not be construed as limiting this invention. For those skilled in the art, the specific meaning of the above-mentioned terms can be understood according to the specific circumstances.

[0050] Example 1:

[0051] like Figures 1-2 As shown, the present invention provides a method for synchronizing video and images from a multi-angle camera, the method of which is as follows:

[0052] Step 100: Obtain the image library, where the images have timestamps;

[0053] Step 200: Traverse the image library and build an original coordinate list based on the maximum and minimum timestamps;

[0054] Step 300: Map the images in the image library to the original coordinate list to generate a baseline coordinate list and multiple synchronized coordinate lists;

[0055] Step 400: Compare the original coordinate list with the synchronized coordinate list. Based on the comparison results, extract all successfully synchronized images from the image library and store them in the result file.

[0056] Step 500: Upload and store the result file in the synchronization library.

[0057] In the above implementation, by generating a reference coordinate list and multiple synchronization coordinate lists, the original coordinate list is compared with the synchronization coordinate lists. Based on the synchronization comparison results, all successfully synchronized images are extracted from the image library and stored in the result file. This design significantly reduces the time overhead of comparing the original coordinate list with the synchronization coordinate lists compared to iterate through all images multiple times, effectively lowering the time cost of image synchronization and improving image synchronization efficiency.

[0058] In step 100, the specific method for obtaining the image library is as follows:

[0059] Frames are extracted from the video stream from each camera to obtain timestamps, and all images are uploaded to the library to form an image library.

[0060] It should be noted that this embodiment uses five cameras, each corresponding to a different shooting angle.

[0061] In step 200, the specific method for traversing the image library and building the original coordinate list based on the maximum and minimum timestamps is as follows:

[0062] Iterate through all images in the image library and get the maximum and minimum timestamps;

[0063] Integerize the maximum and minimum timestamps to obtain the maximum integerized timestamp and the minimum integerized timestamp;

[0064] Create a raw coordinate list of length m based on the maximum and minimum integer timestamps. All elements in the raw coordinate list are "0". Here, m = maximum integer timestamp - minimum integer timestamp.

[0065] It should be noted that since the timestamp ends in decimals, it needs to be converted to an integer in order to calculate the length of the original coordinate list.

[0066] In step 300, the specific method for mapping the images in the image library to the original coordinate list and generating the reference coordinate list and multiple synchronized coordinate lists is as follows:

[0067] Step 310: Obtain the baseline file and multiple synchronization files. The baseline file contains several baseline images, and the synchronization files contain several synchronization images.

[0068] Step 320: Map several reference images onto the original coordinate list to generate a reference coordinate list; map the synchronized images in each synchronized file onto the original coordinate list to generate multiple synchronized coordinate lists.

[0069] It should be noted that, based on a scenario with five cameras, there is one baseline file and five synchronized files. Each baseline file and each of the multiple synchronized files corresponds to one camera.

[0070] In step 310, the specific method for obtaining the benchmark file and multiple synchronization files is as follows:

[0071] Based on the number of cameras, create image files in the image library corresponding to the number of cameras, and store all images in the image library in the corresponding image files.

[0072] Designate one of multiple cameras as the reference camera, and the rest as synchronization cameras;

[0073] From the image files corresponding to the reference camera, select the images that need to be synchronized as reference images, store all the reference images in the same file, and obtain the reference file.

[0074] The corresponding image file from the synchronized camera is used as the synchronization file, and the images in the synchronization file are used as the synchronization images.

[0075] It should be noted that in this embodiment, five image files are created, four cameras are synchronized, and correspondingly, four files are synchronized.

[0076] In step 320, several reference images are mapped onto the original coordinate list to generate a reference coordinate list; the specific method for mapping the synchronization images in each synchronization file to the original coordinate list to generate multiple synchronization coordinate lists is as follows:

[0077] Iterate through the baseline images in the baseline file, map the baseline image to the corresponding position in the original coordinate list according to the timestamp of each baseline image, and change the element "0" at that position to element "1", generating a baseline coordinate list containing elements "0" and "1".

[0078] Iterate through the synchronized images in the synchronized files. Based on the timestamp of each synchronized image, map the synchronized image to the corresponding position in the original coordinate list, and change the element "0" at that position to element "1". Generate a synchronized coordinate list containing elements "0" and "1". Iterate through all synchronized files in the image library in turn to generate multiple synchronized coordinate lists containing elements "0" and "1".

[0079] In step 400, the original coordinate list and the synchronized coordinate list are compared. Based on the comparison results, all successfully synchronized images are extracted from the image library and stored in the result file. The specific method for this is as follows:

[0080] Step 411: Perform a bitwise AND operation between the reference coordinate list and multiple synchronized coordinate lists to obtain a fully synchronized coordinate list;

[0081] Step 412: Based on the position of element "1" in the fully synchronized coordinate list, parse out the corresponding fully synchronized timestamp;

[0082] Step 413: Based on the full synchronization timestamp, extract the corresponding synchronization images from each synchronization file as the full synchronization result images, and store all the baseline images and full synchronization result images in the result file.

[0083] Example 2:

[0084] like Figure 3 As shown, in this embodiment, the remaining steps are the same, and the only difference from Embodiment 1 is:

[0085] In step 400, the original coordinate list and the synchronized coordinate list are compared. Based on the comparison results, all successfully synchronized images are extracted from the image library and stored in the result file. The specific method for this is as follows:

[0086] Step 421: Based on the position of element "1" in the reference coordinate list, slice each synchronous coordinate list according to the preset slicing range to obtain several coordinate list fragments;

[0087] Step 422: If the coordinate list fragment contains an element "1", then parse the incompletely synchronized timestamp corresponding to the element "1";

[0088] Step 423: Determine the synchronization file of the corresponding coordinate list segment, extract the corresponding synchronization image from the synchronization file based on the incomplete synchronization timestamp, and use the image as the incomplete synchronization result image;

[0089] Step 424: Store all baseline images and non-fully synchronized result images into a result file.

[0090] Example 3:

[0091] A computer-readable storage medium is provided, which stores a computer program that, when executed on a computer, causes the computer to perform a method for synchronizing video and images from a multi-angle camera, as well as the computer-readable storage medium.

[0092] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications and substitutions should be covered within the scope of the claims of the present invention. Technical aspects, shapes, and structures not described in detail in this invention are all well-known technologies.

Claims

1. A multi-angle camera video picture synchronization method, characterized in that, The method is: Obtain a picture library, pictures in the picture library have timestamps; Traverse the picture library to establish an original coordinate list according to the maximum timestamp and the minimum timestamp; Map the pictures in the picture library to the original coordinate list to generate a reference coordinate list and a plurality of synchronization coordinate lists; Synchronize and compare the original coordinate list with the synchronization coordinate list, and according to the synchronization comparison result, extract all synchronization successful pictures from the picture library and store them in a result file; Upload and store the result file to a synchronization library; The specific method for mapping the pictures in the picture library to the original coordinate list to generate the reference coordinate list and the plurality of synchronization coordinate lists is: Obtain a reference file and a plurality of synchronization files, the reference file includes a plurality of reference pictures, and the synchronization files include a plurality of synchronization pictures; Map the plurality of reference pictures in the original coordinate list to generate a reference coordinate list; Map the synchronization pictures in each synchronization file to the original coordinate list to generate a plurality of synchronization coordinate lists; Map the plurality of reference pictures in the original coordinate list to generate a reference coordinate list; The specific method for mapping the synchronization pictures in each synchronization file to the original coordinate list to generate a plurality of synchronization coordinate lists is: Traverse the reference pictures in the reference file, map the reference pictures to the corresponding positions of the original coordinate list according to the timestamps of each reference picture, modify the element "0" at the position to the element "1", and generate a reference coordinate list containing the elements "0" and "1"; Traverse the synchronization pictures in the synchronization file, map the synchronization pictures to the corresponding positions of the original coordinate list according to the timestamps of each synchronization picture, modify the element "0" at the position to the element "1", and generate a synchronization coordinate list containing the elements "0" and "1", and traverse all the synchronization files in the picture library in turn to generate a plurality of synchronization coordinate lists containing the elements "0" and "1".

2. The multi-angle camera video picture synchronization method of claim 1, wherein, The specific method for obtaining the picture library is: Frame extraction processing is performed on the video stream under each camera to obtain pictures with timestamps, and all the pictures are uploaded into the library to form a picture library.

3. The multi-angle camera video picture synchronization method of claim 1, wherein, The specific method for traversing the picture library to establish an original coordinate list according to the maximum timestamp and the minimum timestamp is: Traverse all the pictures in the picture library to obtain the maximum timestamp and the minimum timestamp; Integerize the maximum timestamp and the minimum timestamp to obtain a maximum integer timestamp and a minimum integer timestamp; Establish an original coordinate list with a length of m according to the maximum integer timestamp and the minimum integer timestamp, and each element in the original coordinate list is "0", wherein m = maximum integer timestamp - minimum integer timestamp.

4. The multi-angle camera video picture synchronization method of claim 1, wherein, The specific method for obtaining the reference file and the plurality of synchronization files is: According to the number of cameras, establish picture files corresponding to the number of cameras in the picture library, and store all the pictures in the picture library in the corresponding picture files; Specify one of the plurality of cameras as a reference camera, and the remaining cameras as synchronization cameras; Filtering out pictures needed to be synchronized as reference pictures from picture files corresponding to the reference camera, storing all the reference pictures in a same file to obtain a reference file; Picture files corresponding to the synchronization camera are used as synchronization files, and pictures in the synchronization files are used as synchronization pictures.

5. The multi-angle camera video picture synchronization method of claim 1, wherein, The specific method of synchronously comparing the original coordinate list with the synchronization coordinate list, extracting all the successfully synchronized pictures from the picture library according to the synchronization comparison result, and storing the pictures in a result file is as follows: Performing an AND operation on the reference coordinate list and the plurality of synchronization coordinate lists to obtain a completely synchronized coordinate list; According to the position of the element "1" in the completely synchronized coordinate list, a corresponding completely synchronized timestamp is parsed; According to the completely synchronized timestamp, corresponding synchronization pictures are extracted from each synchronization file as completely synchronized result pictures, and all the reference pictures and the completely synchronized result pictures are stored in a result file.

6. The multi-angle camera video picture synchronization method of claim 1, wherein, The specific method of synchronously comparing the original coordinate list with the synchronization coordinate list, extracting all the successfully synchronized pictures from the picture library according to the synchronization comparison result, and storing the pictures in a result file is as follows: According to the position of the element "1" in the reference coordinate list, each synchronization coordinate list is sliced according to a preset slice range to obtain a plurality of coordinate list segments; If the element "1" exists in the coordinate list segment, a non-completely synchronized timestamp corresponding to the element "1" is parsed; A synchronization file corresponding to the coordinate list segment is determined, and according to the non-completely synchronized timestamp, a corresponding synchronization picture is extracted from the synchronization file, and the picture is used as a non-completely synchronized result picture; All the reference pictures and the non-completely synchronized result pictures are stored in a result file.

7. A computer readable storage medium, the computer readable storage medium storing a computer program, when the computer program is run on a computer, the computer program causes the computer to execute the method of any one of claims 1-6.

Images