Image data synchronization method, storage medium and electronic equipment

Abstract

The invention relates to the technical field of image processing, and particularly provides an image data synchronization method, a storage medium and electronic equipment, and the method can comprise the steps: marking a plurality of frames of images collected by a plurality of cameras, and obtaining marked images; putting each frame of marked image in the marked image into a queue according to a time sequence to obtain an image queue; and traversing the image queue, and extracting an optimal synchronous image group from the image queue. According to some embodiments of the invention, on-demand accurate synchronization of image data can be realized.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image data synchronization method, storage medium and electronic device. Background Art

[0002] A multi-camera system can provide images from different perspectives in the same scene to make up for the limitations of some spatial occlusions. Therefore, in triangulation, the synchronous acquisition of multi-camera images is crucial.

[0003] At present, the existing technology uses hardware control strategies to trigger the exposure operation of multiple cameras to achieve time synchronization; or uses the camera's own video recording time code function to achieve alignment synchronization, or uses the main camera as a reference to achieve data synchronization of multiple cameras. However, due to the differences in camera models and hardware configurations, the existing technology needs to adopt different synchronization strategies for different cameras, and the image data synchronization method has poor universality.

[0004] Therefore, how to provide a technical solution for a method for synchronizing image data with wide versatility has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of some embodiments of the present application is to provide a method, storage medium and electronic device for synchronizing image data. The technical solutions of the embodiments of the present application can realize image data synchronization of multiple cameras without being restricted by camera models and hardware configurations, and have wide versatility and high accuracy.

[0006] In a first aspect, some embodiments of the present application provide a method for synchronizing image data, including: marking multiple frames of images captured by multiple cameras to obtain marked images; placing each frame of the marked images in the marked images into a queue in chronological order to obtain an image queue; traversing the image queue to extract the optimal synchronized image group from the image queue.

[0007] Some embodiments of the present application obtain an image queue by marking multiple frames of images captured by multiple cameras and then placing them into a queue, and then traversing the image queue to extract the optimal synchronized image group to complete image synchronization. The embodiments of the present application can achieve image data synchronization of multiple cameras without being constrained by camera models and hardware configurations, have wide versatility, high accuracy, and meet real-time data acquisition requirements.

[0008] In some embodiments, marking multiple frames of images captured by multiple cameras to obtain marked images includes: marking each frame of the multiple frames with a timestamp and a camera number to obtain each marked image, wherein the camera number represents the source camera of each frame of the image.

[0009] Some embodiments of the present application can clearly record the time and source of image acquisition by marking each frame of image with a timestamp and a camera number, thereby providing support for subsequent image data synchronization.

[0010] In some embodiments, extracting the optimal synchronized image group from the image queue includes: determining the number of cameras in the multiple cameras that are in normal working condition; screening out at least one image group to be synchronized from the image queue, wherein the number of frames of images in each image group to be synchronized in the at least one image group to be synchronized is the same as the number of cameras, and each frame of the image in each image group to be synchronized corresponds one-to-one to a camera in normal working condition; determining the optimal synchronized image group based on the time difference of each image group to be synchronized and a preset synchronization time threshold.

[0011] Some embodiments of the present application can select at least one image group to be synchronized from the image queue by determining the number of cameras in normal working state, and then determine the final synchronized image group from the at least one image group to be synchronized, thereby ensuring the accuracy of image data synchronization.

[0012] In some embodiments, filtering out at least one image group to be synchronized from the image queue includes: determining the first frame camera number and the first frame timestamp of the first frame image in each image group to be synchronized; taking the first frame image as the starting point, searching the image queue for the image that is closest to the first frame timestamp and matches other cameras to obtain each image group to be synchronized; wherein the other cameras are cameras in normal working state except the camera corresponding to the first frame camera number.

[0013] Some embodiments of the present application obtain each image group to be synchronized by determining the first frame image in each image group to be synchronized, searching for images that meet the time and camera number conditions from the image queue, and providing accurate data support for the subsequent selection of the final synchronized image group.

[0014] In some embodiments, the determining of the optimal synchronized image group based on the time difference of each image group to be synchronized and a preset synchronization time threshold includes: obtaining the time difference of each image group to be synchronized, wherein the time difference is the difference between the first frame timestamp and the last frame timestamp of the last frame image in each image group to be synchronized; obtaining the image group to be synchronized corresponding to the minimum value of the time difference of each image group to be synchronized, and when it is confirmed that the image group to be synchronized corresponding to the minimum value satisfies the preset synchronization time threshold, taking the image group to be synchronized corresponding to the minimum value as the optimal synchronized image group.

[0015] Some embodiments of the present application determine the synchronization image group through the time difference of each image group to be synchronized, thereby ensuring the accuracy of image data synchronization.

[0016] In some embodiments, after extracting the optimal synchronous image group from the image queue, the method further includes: deleting expired and invalid images in the image queue according to a preset synchronous update strategy to obtain the latest image queue.

[0017] Some embodiments of the present application prevent invalid data in the image queue from occupying memory and interfering with the extraction of the optimal synchronous image group by deleting expired invalid images.

[0018] In some embodiments, the method further includes: when the length of the image queue exceeds a preset length threshold, deleting frame images in the image queue that exceed the preset length threshold according to a first-in-first-out principle.

[0019] Some embodiments of the present application ensure that the latest image data can be stored in the image queue by deleting frame images that exceed a preset length threshold.

[0020] In some embodiments, the first frame image is determined by dynamically adjusting according to the nearest clustering principle within the group based on the camera numbers corresponding to the images in each image group to be synchronized during the process of selecting the images in the group.

[0021] In a second aspect, some embodiments of the present application provide a device for synchronizing image data, comprising: an image marking module, used to mark multiple frames of images captured by multiple cameras to obtain marked images; an image processing module, used to put each frame of the marked images into a queue in chronological order to obtain an image queue; an image synchronization module, used to traverse the image queue and extract the optimal synchronized image group from the image queue.

[0022] In a third aspect, some embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the method described in any embodiment of the first aspect.

[0023] In a fourth aspect, some embodiments of the present application provide an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement a method as described in any embodiment of the first aspect when executing the program.

[0024] In a fifth aspect, some embodiments of the present application provide a computer program product, wherein the computer program product comprises a computer program, wherein the computer program, when executed by a processor, can implement the method described in any embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of some embodiments of the present application, the drawings required for use in some embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A system diagram of image data synchronization provided for some embodiments of the present application;

[0027] Figure 2 One of the flow charts of the method for synchronizing image data provided in some embodiments of the present application;

[0028] Figure 3 One of the image queue schematic diagrams provided for some embodiments of the present application;

[0029] Figure 4 A second schematic diagram of an image queue provided for some embodiments of the present application;

[0030] Figure 5 A second flowchart of a method for synchronizing image data provided in some embodiments of the present application;

[0031] Figure 6 A flow chart of a method for synchronous image group selection provided for some embodiments of the present application;

[0032] Figure 7 A block diagram of an apparatus for synchronizing image data provided in some embodiments of the present application;

[0033] Figure 8 A schematic diagram of an electronic device is provided for some embodiments of the present application. DETAILED DESCRIPTION

[0034] The technical solutions in some embodiments of the present application will be described below in conjunction with the drawings in some embodiments of the present application.

[0035] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0036] In the related art, in triangulation, the synchronous acquisition of multi-camera images is particularly important for camera calibration and spatial calculation. Image acquisition equipment (for example, cameras) contains a variety of different camera models and hardware configurations, and the core requirement is that the camera images collected by multiple cameras are basically synchronized when working (synchronization here means that the time difference between the image generation time is controlled within an acceptable range). However, due to the large number of camera types, different data synchronization strategies need to be adopted for different types of cameras in a multi-camera system, and the implementation process is relatively cumbersome. It can be seen from the above-mentioned related technologies that the image data synchronization method of multiple cameras in the prior art cannot be universal, and the scope of application is limited by the camera type.

[0037] In view of this, some embodiments of the present application provide a method for synchronizing image data, which can keep the image acquisition methods of each camera unchanged, merge each frame of image acquired by each camera into an orderly multi-camera image time series queue (that is, image queue), and select synchronized image groups that meet the synchronization requirements along the time axis characteristics. Since the image data synchronization method provided by the embodiments of the present application can keep the image acquisition methods of each camera unchanged, there are no restrictions on the camera model, hardware configuration, image output frame rate, etc. It is just a local optimal image synchronization strategy implemented by setting buffer storage and utilizing the time sequence of different cameras outputting images in combination with a priority algorithm. It has a wide range of applications and high practicality.

[0038] The following is combined with Figure 1 The overall composition structure of the system for synchronizing image data provided by some embodiments of the present application is exemplified.

[0039] like Figure 1 As shown, some embodiments of the present application provide a system for synchronizing image data, which may include: a first camera 110, a second camera 120 ... an Nth camera and an image synchronization terminal 200. The first camera 110, the second camera 120 ... an Nth camera (as a specific example of multiple cameras) may send their respective captured images to the image synchronization terminal 200. The image synchronization terminal 200 may mark the multiple frames of images captured by the N cameras and put them into an image queue. Afterwards, the image synchronization extraction controller in the image synchronization terminal 200 may traverse and extract the image queue to obtain a synchronized image group.

[0040] In some embodiments of the present application, each of the N cameras independently and continuously acquires images, and it starts the camera with an independent thread to perform acquisition work. Specifically, the camera is driven to continuously acquire and generate image data according to its own preset control parameters such as image resolution and frame rate until the camera is in a closed state and stops working. The image synchronization terminal 200 can also be called an image synchronization transfer station. The image synchronization transfer station refers to a unit that is independent of the camera working thread and is dedicated to temporary storage of multiple frames of images acquired by the camera and extraction of stored images. The images captured by each camera can be sent to the image synchronization transfer station in real time. Each camera can transmit the image to the image synchronization transfer station frame by frame through a callback function or communication method; it can be seen that the callback function connects the working thread of each camera and the image synchronization transfer station unit.

[0041] It should be understood that the value of N (N is a positive integer) can be set according to actual conditions and is not specifically limited in the embodiments of the present application.

[0042] The following is combined with Figure 2 The implementation process of image data synchronization performed by the image synchronization terminal 200 provided in some embodiments of the present application is exemplified.

[0043] Please see attached Figure 2 , Figure 2 A flowchart of a method for synchronizing image data is provided for some embodiments of the present application. The method for synchronizing image data may include:

[0044] S210, marking multiple frames of images captured by multiple cameras to obtain marked images.

[0045] For example, in some embodiments of the present application, after receiving multiple frames of images sent by N cameras, the image synchronization transfer station marks each frame of the image to obtain a marked image.

[0046] In some embodiments of the present application, S210 may include: marking a timestamp and a camera number for each frame of the multiple frames of images to obtain a marked image for each frame, wherein the camera number represents a source camera of each frame of the image.

[0047] For example, in some embodiments of the present application, the image synchronization transfer station marks the timestamp for each frame of image received in a unified manner, and records the ID number of the camera from which the frame of image comes (as a specific example of the camera number). The ID number can be used to clearly identify which camera each frame of image is captured by. The camera ID is passed along with the image by the callback function. This ID is the unique identification number assigned to the connected camera when the synchronizer of the image synchronization transfer station just starts working. The timestamp is based on the clock operation of the device where the synchronizer is located as a unified standard.

[0048] S220, putting each frame of the marked image in the marked image into a queue in chronological order to obtain an image queue.

[0049] For example, in some embodiments of the present application, the image synchronization transfer station puts each frame of the marked image into the same image queue in a first-come-first-served manner (i.e., time order) to obtain a time series image (as a specific example of an image queue). The images of all cameras (i.e., N cameras) share the same image queue.

[0050] In some embodiments of the present application, the method for synchronizing image data further includes: when the length of the image queue exceeds a preset length threshold, deleting frame images in the image queue that exceed the preset length threshold according to a first-in-first-out principle.

[0051] For example, in some embodiments of the present application, when the sequence length in the image queue is greater than a preset length threshold, frame images greater than the preset length threshold are cleared from the queue in a first-in-first-out manner, that is, frame images with earlier times are deleted and images that have arrived at the latest time are retained.

[0052] As an example, Figure 3 In the image queue schematic diagram shown, taking N=3 as an example, the image queue length threshold can be preset to 3 times the number of cameras. In this example, the image queue length threshold is 9. Figure 3 Each small square in the image queue represents an image output by the camera id (that is, the camera number, i.e. 1, 2, 3). Figure 3 The image on the left side is generated earlier than the image on the right side. S01 sends the newly acquired image to the image synchronization transfer station. The image synchronization transfer station marks the newly received image by S02 with the camera ID and timestamp, and pushes it to the last position on the right side of the image queue, and checks the queue length at the same time. When the queue length is greater than the preset length threshold, the image at the left end of the image queue that is greater than the preset length threshold (for example, Figure 3 The portion marked with a dotted line in the middle) is deleted from the image queue, thereby ensuring that the image queue always records the latest multi-frame images during the continuous working cycle of the camera, and the image queue will not overflow over time.

[0053] S230, traversing the image queue, and extracting an optimal synchronous image group from the image queue.

[0054] For example, in some embodiments of the present application, the image synchronization extraction controller can traverse the image queue and extract a group of images that can be synchronized from the image queue as a synchronization image group. The image synchronization extraction controller is another independently working thread unit.

[0055] In some embodiments of the present application, S230 may include:

[0056] S231, determining the number of cameras in a normal working state among the multiple cameras.

[0057] For example, in some embodiments of the present application, the working status of each camera in the camera queue to be synchronized is first determined, and the number n1 of cameras in a normal working state is confirmed.

[0058] S232, filtering out at least one image group to be synchronized from the image queue, wherein the number of image frames in each of the at least one image group to be synchronized is the same as the number of cameras, and each frame of the image in each of the image groups to be synchronized corresponds one-to-one to a camera in a normal working state.

[0059] For example, in some embodiments of the present application, according to the first-in-first-out principle, at least one group of images to be synchronized with a length of n1 is selected along the time axis of the time series image. The images in the image group to be synchronized need to come from different cameras and each camera ID corresponds to only one image.

[0060] In some embodiments of the present application, S232 may include: determining the first frame camera number and first frame timestamp of the first frame image in each image group to be synchronized; taking the first frame image as the starting point, searching the image closest to the first frame timestamp and matching other cameras in the image queue to obtain each image group to be synchronized; wherein the other cameras are cameras in the normal working state except the camera corresponding to the first frame camera number. The first frame image is determined by dynamically adjusting according to the nearest clustering principle within the group based on the camera number corresponding to the image to be synchronized in the process of selecting the image to be synchronized in each image group to be synchronized.

[0061] For example, in some embodiments of the present application, the process of acquiring a group of images to be synchronized includes: according to the first-in-first-out principle, starting from the image number 0 that first enters the image queue, searching for images that match the camera ID in normal working state, recording the starting number s1 and timestamp t1 of the most recent image of the first matched camera, at which time the image is the first frame image; continuing along the positive direction of the time axis, traversing the images corresponding to the remaining cameras (that is, other cameras) that are in normal working state and have not been matched at the next sequence position; when all cameras in normal working state have found corresponding images, it is considered that the detection of a group of images to be synchronized is completed. Afterwards, the s1 number is increased by 1, and with this as the starting point, the above steps are repeated to find all the image groups to be synchronized that can be grouped.

[0062] For example, Figure 4 Take the image sequence shown as an example to find the image group to be synchronized. Assume that the image synchronization transfer station is connected to 3 cameras that need to be synchronized, that is, n1=3, and their camera IDs are 1, 2, and 3 respectively. Figure 4 The image queue shown has 9 images, which come from the above 3 cameras. The image on the left of the image sequence arrives earlier than the image on the right. The image queues are numbered 0, 1, 2, 3, 4, 5, 6, 7, and 8 from left to right.

[0063] Specifically, according to the first-in-first-out principle, we start from the image with the sequence number Index0 and detect the image queue along the direction of the dotted arrow above. The image with the sequence number Index0 comes from camera 1, and s1 is recorded as sequence number 0 at this moment, and the timestamp t1 of the image is recorded; then we detect the image with the sequence number Index1 (i.e. the second square), and find that the image still comes from camera 1, so we update s1 to 1, and at the same time update the timestamp of the image with the sequence number Index1 at this moment (to ensure that the image is closest to the image found later in terms of time relationship), and the image is the first frame image at this moment. We continue to detect the images with the sequence numbers Index2 and Index3, and we can see that the two frames of images come from camera 2 and camera 3 respectively. The number of images found at this moment is 3, which corresponds to n1, so we get the first group of images to be synchronized, whose sequence numbers are Index1, Index2, and Index3.

[0064] because Figure 4The maximum sequence number of the image queue is Index8, and the maximum sequence number of the first group of images to be synchronized is Index3, so it is necessary to continue to detect whether there is a second group of images to be synchronized. The detection of the second group of images to be synchronized starts from position s1+1, that is, starting from the image with sequence number Index2 and detecting to the right. Based on the same principle mentioned above, the second group of images to be synchronized can be obtained, and their sequence numbers are Index2, Index3, and Index4. Continuing the next round of detection from the sequence number Index3 position, the sequence numbers of the third group of images to be synchronized can be obtained as Index6, Index7, and Index8. In this round of detection, starting from Index3, the first frame is first determined to be the Index3 image of camera No. 3, and then encounters Index4 of camera No. 1, and then encounters Index5 of the same camera No. 3. According to the principle of synchronous nearest clustering, the first frame is changed to Index4 of camera No. 1 at this moment. As the detection continues, the synchronous image group extraction is completed until the Index8 image of camera No. 2 is encountered. At this time, the first frame position is adjusted to the Index6 camera No. 3 image according to the nearest clustering principle to ensure that the selected three frames of images are closest to each other on the time axis. In the process of selecting the synchronous image group, it can be seen that the images with sequence numbers Index3, Index5 and Index6 are all from camera 3. According to the principle that the three frames are generated most recently, only the image with sequence number Index6 is recorded. Similarly, camera 1 only records the image with sequence number Index7, so that the images of the three cameras will be closer in time. Since the largest sequence number in the third group of images to be synchronized is Index8, which is equal to the maximum sequence number of the image queue, the extraction of the image group to be synchronized is completed at this time.

[0065] It is understandable that the extraction of the image group to be synchronized is in the time sequence of the image queue. After one group is extracted, the next group is searched, and this cycle is repeated until all the images that can be synchronized in the image queue are extracted. The images in the extracted image group to be synchronized are closest to each other in terms of time relationship.

[0066] S233: Determine the optimal synchronization image group based on the time difference of each image group to be synchronized and a preset synchronization time threshold.

[0067] For example, in some embodiments of the present application, based on the time difference of each image group to be synchronized in at least one group of images to be synchronized with a length of n1, a synchronization image group is selected, wherein the time interval between images in the synchronization image group is the shortest and meets the preset synchronization time threshold.

[0068] In some embodiments of the present application, S233 may include: obtaining the time difference of each image group to be synchronized, wherein the time difference is the difference between the first frame timestamp and the last frame timestamp of each image group to be synchronized; obtaining the image group to be synchronized corresponding to the minimum value of the time difference of each image group to be synchronized, and when confirming that the image group to be synchronized corresponding to the minimum value meets the preset synchronization time threshold, taking the image group to be synchronized corresponding to the minimum value as the optimal synchronization image group.

[0069] For example, in some embodiments of the present application, Figure 4 For example, the first group of images to be synchronized includes images with sequence numbers Index1, Index2, and Index3, where sequence number Index1 is the first frame image and sequence number Index3 is the last frame image. The difference in timestamps between the two frames of images (i.e., the time difference) is calculated to obtain time0; based on the same method, the time difference time1 between the two frames of images corresponding to sequence numbers Index2 and Index4 in the second group of images to be synchronized, and the time difference time2 between the two frames of images corresponding to sequence numbers Index6 and Index8 in the third group of images to be synchronized are calculated; time0, time1, and time2 are compared, and the image group to be synchronized with the minimum value among the three and meeting the preset synchronization time threshold is selected as the optimal synchronization image group. For example, if time0 is the smallest and meets the preset synchronization time threshold, the images with sequence numbers Index1, Index2, and Index3 in the first group of images to be synchronized will be output externally. The preset synchronization time threshold can be set according to actual conditions, and the embodiments of the present application are not specifically limited here.

[0070] In some embodiments of the present application, after executing S233, the method for synchronizing image data may further include: deleting expired and invalid images in the image queue according to a preset synchronization update strategy to obtain the latest image queue.

[0071] For example, in some embodiments of the present application, after the optimal synchronous image group is selected in the above method, the corresponding part needs to be removed from the image queue according to the preset synchronous update strategy. The preset synchronous update strategy can be set according to the actual application scenario. The preset synchronous update strategy can include frame rate priority, content update priority, or a moderate update strategy between the two.

[0072] Specifically, the preset synchronization update strategy is threshold setting, and the threshold setting can be to select a time point where a timestamp in the synchronization image group is located as a time threshold, and delete images in the image sequence that are earlier than the time threshold (as a specific example of expired and invalid images).

[0073] Frame rate priority is to delete only the first image and previous images in the synchronous image group to ensure that the synchronous image group data is provided to the outside at the fastest speed. At least one image in this group of synchronous images is newly updated. Taking the synchronous image group selected above as an example, the images with sequence numbers Index0 and Index1 in the image queue are deleted (as a specific example of expired and invalid images).

[0074] Content update priority is to delete the last and all previous images in the synchronized image group to ensure that all images in the synchronized image group selected from the next group are newly updated; taking the synchronized image group selected above as an example, the images with serial numbers Index0, Index1, Index2, and Index3 are deleted (as a specific example of expired and invalid images). Among them, the feature of frame number priority is that when the frame rates of different cameras are very different, images with slow frame rates may be used repeatedly with images with fast frame rates. Content update priority is to synchronize the image content based on the camera that updates the slowest image. The moderate update strategy ensures that a certain proportion of camera images are updated, such as 50% of the camera images.

[0075] The following is combined with Figure 5 The specific process of image data synchronization provided by some embodiments of the present application is exemplified.

[0076] Please see attached Figure 5 , Figure 5 A flow chart of a method for synchronizing image data is provided for some embodiments of the present application.

[0077] The above process is explained below as an example.

[0078] S510, multiple cameras send the captured multiple frames of images to an image synchronization transfer station.

[0079] For example, multiple cameras are independently started and continuously generate image data (ie, multiple frames of images) according to their own set parameters.

[0080] S520, the image synchronization transfer station marks the multiple frames of images and puts them into a queue to obtain an image queue sorted in chronological order.

[0081] For example, each camera is connected to an image synchronization transfer station through an image callback function or a communication method. The image synchronization transfer station performs a marking process on multiple frames of images before synchronization and imports them into a time series image queue in chronological order.

[0082] S530: The image synchronization extractor extracts the image queue to obtain the optimal synchronization image group.

[0083] For example, the image synchronization extractor continuously completes the extraction of synchronization image groups from the time series image queue, and outputs the extracted preferred synchronization image groups to the outside.

[0084] It should be noted that S510~S530 can be operated in a loop according to a preset period. Among them, S510 represents the cyclic operation of a camera continuously outputting images to an image synchronization transfer station; S520~S530 represent the cyclic operation of an image synchronization extractor continuously selecting a synchronous image group and outputting the synchronous image group to the outside. These two independent loop operations can be implemented through thread programming, and the preset cycle of the loop can be set as needed, which is not specifically limited in the embodiments of the present application. It should be understood that the specific implementation process of S510~S530 can refer to the method embodiments provided above. In order to avoid repetition, the detailed description is appropriately omitted here.

[0085] The following is combined with Figure 6 The specific process of synchronous image group selection provided by some embodiments of the present application is exemplified.

[0086] Please see attached Figure 6 , Figure 6 A flow chart of a method for synchronous image group selection provided for some embodiments of the present application.

[0087] The above process is explained below as an example.

[0088] S610: Detect the state of each camera among a plurality of cameras requiring image synchronization.

[0089] For example, the number of images in the synchronous image group that needs to be extracted is determined by detecting the state of the camera.

[0090] S620, determine whether all cameras are in the off state, if not, execute S630, otherwise end.

[0091] For example, if all the synchronous cameras to be extracted are in a closed state (indicating that the number of acquired images is 0), the synchronous image group extraction cycle operation is terminated. If there are cameras that are working normally, the following operations are continued.

[0092] S630: Select a synchronous image group from the image queue.

[0093] S640, determine whether there is a synchronous image group, if so, execute S650, otherwise return to S610.

[0094] For example, if the synchronous image group cannot be selected from the current image queue, the process continues to wait for a certain period of time (eg, wait for the next extraction cycle), and returns to S610 at the next moment to continue detecting.

[0095] S650, clearing out expired and invalid images in the image queue.

[0096] For example, if a synchronization image group has been selected from the current image queue, expired and invalid images (that is, expired and invalid images) in the image queue are cleared.

[0097] S660, output the synchronous image group, and return to S610 in the next cycle.

[0098] For example, the extracted synchronous image group is output externally, and the process returns to S610 to continue the next synchronous image group extraction operation.

[0099] It should be noted that the specific implementation process of S610 to S660 can refer to the method embodiment provided above, and in order to avoid repetition, the detailed description is appropriately omitted here.

[0100] Through some of the above-mentioned embodiments of the present application, it can be seen that the embodiments of the present application do not change the existing mechanism of acquiring images by different types of cameras. By setting up image transfer buffer storage, the images generated by different cameras are sorted in sequence according to the time of entering the image synchronization transfer station, and finally the minimum difference synchronization operation of the image content is achieved through local image queue synchronization optimization, and the applicability is wide.

[0101] Please refer to Figure 7 , Figure 7 The block diagram of the image data synchronization device provided by some embodiments of the present application is shown. It should be understood that the image data synchronization device corresponds to the above method embodiment and can execute each step involved in the above method embodiment. The specific functions of the image data synchronization device can be found in the above description. To avoid repetition, the detailed description is appropriately omitted here.

[0102] Figure 7 The image data synchronization device includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the image data synchronization device, and the image data synchronization device includes: an image marking module 710, used to mark multiple frames of images captured by multiple cameras to obtain marked images; an image processing module 720, used to put each marked image in the marked image into a queue in chronological order to obtain an image queue; an image synchronization module 730, used to traverse the image queue and extract the optimal synchronized image group from the image queue.

[0103] In some embodiments of the present application, the image marking module 710 is used to mark each frame of the multiple frames with a timestamp and a camera number to obtain a marked image for each frame, wherein the camera number represents the source camera of each frame of the image.

[0104] In some embodiments of the present application, the image synchronization module 730 is used to determine the number of cameras in the multiple cameras that are in normal working condition; filter out at least one image group to be synchronized from the image queue, wherein the number of frames of images in each image group to be synchronized in the at least one image group to be synchronized is the same as the number of cameras, and each frame of the image in each image group to be synchronized corresponds one-to-one to a camera in normal working condition; determine the optimal synchronized image group based on the time difference of each image group to be synchronized and a preset synchronization time threshold.

[0105] In some embodiments of the present application, the image synchronization module 730 is used to determine the first frame camera number and the first frame timestamp of the first frame image in each image group to be synchronized; taking the first frame image as the starting point, searching the image queue for the image that is closest to the first frame timestamp and matches other cameras to obtain each image group to be synchronized; wherein the other cameras are cameras in normal working state except the camera corresponding to the first frame camera number.

[0106] In some embodiments of the present application, the image synchronization module 730 is used to obtain the time difference of each image group to be synchronized, wherein the time difference is the difference between the first frame timestamp and the last frame timestamp of each image group to be synchronized; obtain the image group to be synchronized corresponding to the minimum value of the time difference of each image group to be synchronized, and when it is confirmed that the image group to be synchronized corresponding to the minimum value meets the preset synchronization time threshold, the image group to be synchronized corresponding to the minimum value is used as the optimal synchronization image group.

[0107] In some embodiments of the present application, the image synchronization module 730 is used to, after extracting the optimal synchronization image group from the image queue, delete expired and invalid images in the image queue according to a preset synchronization update strategy to obtain the latest image queue.

[0108] In some embodiments of the present application, the image processing module 720 is used to delete the frame images in the image queue that exceed the preset length threshold according to the first-in-first-out principle when the length of the image queue exceeds the preset length threshold.

[0109] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0110] Some embodiments of the present application further provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the operations of the method corresponding to any of the above methods provided in the above embodiments.

[0111] Some embodiments of the present application further provide a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operations corresponding to any of the above methods provided in the above embodiments.

[0112] like Figure 8 As shown, some embodiments of the present application provide an electronic device 800, which includes: a memory 810, a processor 820, and a computer program stored in the memory 810 and executable on the processor 820, wherein the processor 820 can implement a method as described in any of the above embodiments when reading the program from the memory 810 through a bus 630 and executing the program.

[0113] Processor 820 can process digital signals and can include various computing structures, such as complex instruction set computer structure, reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, processor 820 can be a microprocessor.

[0114] The memory 810 may be used to store instructions executed by the processor 820 or data related to the execution of instructions. These instructions and / or data may include codes for implementing some or all functions of one or more modules described in the embodiments of the present application. The processor 820 of the disclosed embodiment may be used to execute instructions in the memory 810 to implement the method shown above. The memory 810 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory, or other memory known to those skilled in the art.

[0115] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0116] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0117] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for synchronizing image data, characterized in that: include: Marking multiple frames of images captured by multiple cameras to obtain marked images; Put each frame of the marked image in the marked image into a queue in chronological order to obtain an image queue; The image queue is traversed, and an optimal synchronous image group is extracted from the image queue.

2. The method according to claim 1, characterized in that The step of marking multiple frames of images captured by multiple cameras to obtain marked images includes: Each frame of the multiple images is marked with a timestamp and a camera number to obtain a marked image for each frame, wherein the camera number represents the source camera of each frame of the image.

3. The method according to claim 1 or 2, characterized in that The step of extracting the optimal synchronous image group from the image queue comprises: Determining the number of cameras in normal working condition among the plurality of cameras; Filtering out at least one to-be-synchronized image group from the image queue, wherein the number of frames of images in each to-be-synchronized image group in the at least one to-be-synchronized image group is the same as the number of cameras, and each frame of the image in each to-be-synchronized image group corresponds one-to-one to a camera in a normal working state; The optimal synchronization image group is determined based on the time difference of each to-be-synchronized image group and a preset synchronization time threshold.

4. The method according to claim 3, characterized in that The step of selecting at least one image group to be synchronized from the image queue comprises: Determine the first frame camera number and first frame timestamp of the first frame image in each image group to be synchronized; Taking the first frame image as the starting point, searching the image queue for the image closest to the first frame timestamp and matching other cameras to obtain each image group to be synchronized; wherein the other cameras are cameras in normal working state except the camera corresponding to the first frame camera number.

5. The method according to claim 4, characterized in that The determining the optimal synchronized image group based on the time difference of each to-be-synchronized image group and a preset synchronization time threshold comprises: Obtaining a time difference of each image group to be synchronized, wherein the time difference is a difference between a first frame timestamp and a last frame timestamp of a last frame image in each image group to be synchronized; The image group to be synchronized corresponding to the minimum value of the time difference of each image group to be synchronized is obtained, and when it is confirmed that the image group to be synchronized corresponding to the minimum value meets the preset synchronization time threshold, the image group to be synchronized corresponding to the minimum value is used as the optimal synchronization image group.

6. The method according to any one of claims 1-2, 4-5, characterized in that: The method further comprises: When the length of the image queue exceeds a preset length threshold, frame images in the image queue that exceed the preset length threshold are deleted according to a first-in-first-out principle.

7. The method according to any one of claims 1-2, 4-5, characterized in that: After extracting the optimal synchronous image group from the image queue, the method further includes: According to the preset synchronous update strategy, expired and invalid images in the image queue are deleted to obtain the latest image queue.

8. The method according to claim 4 or 5, characterized in that The first frame image is determined by dynamic adjustment according to the nearest clustering principle within the group based on the camera numbers corresponding to the group images in the process of selecting the group images in each image group to be synchronized.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program executes the method according to any one of claims 1 to 7 when executed by a processor.

10. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the computer program executes the method according to any one of claims 1 to 7 when being run by the processor.

Images