Image synchronization method and related device

Abstract

Provided in embodiments of the present application are an image synchronization method and a related device. The method comprises: respectively sending trigger signals to a plurality of camera apparatuses, and controlling the plurality of camera apparatuses to perform photographing; acquiring images captured by the plurality of camera apparatuses, and performing initial matching on the acquired images to determine a first group of time-synchronized matched images; on the basis of the first group of time-synchronized matched images, performing process matching on images captured by the plurality of camera apparatuses after the initial matching, so as to determine a second group of time-synchronized matched images; and on the basis of the first group of matched images and the second group of matched images, performing time synchronization on images captured by the plurality of camera apparatuses after the process matching. In the embodiments of the present application, errors caused by time synchronization of image data can be avoided, thereby effectively improving the accuracy of the time synchronization of the image data.

Description

Image synchronization methods and related equipment

[0001] This application claims priority to Chinese Patent Application No. 202410918891.2, filed on July 9, 2024, entitled “Image Synchronization Method and Related Device”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application belongs to the field of image processing, and in particular relates to an image synchronization method and related equipment. Background Technology

[0003] In applications such as stereoscopic photography, virtual reality, video surveillance, and 3D scanning, it is necessary to use electronic devices such as smartphones and personal computers as control terminals to control multiple camera devices for simultaneous shooting and to synchronize the image data captured by multiple camera devices in time to ensure the accuracy and relevance of subsequent image analysis and processing. Related technologies typically employ frame number synchronization mechanisms or timestamp synchronization mechanisms for image data time synchronization. However, time synchronization based on frame number synchronization mechanisms or timestamp synchronization mechanisms is prone to errors, resulting in low accuracy and poor synchronization effects. Summary of the Invention

[0004] In view of this, embodiments of this application provide an image synchronization method and related equipment to solve the problem that image data synchronization based on frame number synchronization mechanism or timestamp synchronization mechanism is prone to errors, resulting in poor image data synchronization effect.

[0005] In a first aspect, embodiments of this application provide an image synchronization method, the method comprising: sending trigger signals to multiple camera devices respectively to control the multiple camera devices to capture images; acquiring images captured by the multiple camera devices; performing initial matching on the acquired images to determine a first set of matching images for time synchronization; performing process matching on the images captured by the multiple camera devices after the initial matching based on the first set of matching images for time synchronization to determine a second set of matching images for time synchronization; and performing time synchronization on the images captured by the multiple camera devices after the process matching based on the first set of matching images and the second set of matching images.

[0006] In one possible implementation, trigger signals are sent to multiple camera devices to control them to take pictures, including: simultaneously sending trigger signals to multiple camera devices based on a preset period, with each camera device taking a picture in response to each trigger signal.

[0007] In one possible implementation, initial matching of the acquired images is performed to determine the first set of matching images for time synchronization, including: comparing the frame numbers of the images captured by each camera device; if the frame numbers of the images captured by each camera device are the same, the multiple images with the same frame numbers are determined as the first set of matching images for time synchronization.

[0008] In one possible implementation, initial matching of the acquired images is performed to determine the first set of matching images for time synchronization, including: determining multiple images captured by multiple camera devices in response to the same trigger signal as the first set of matching images for time synchronization.

[0009] In one possible implementation, based on the first set of time-synchronized matching images, process matching is performed on images captured by multiple camera devices after the initial matching to determine the second set of time-synchronized matching images, including: if the frame numbers of the images captured by each camera device in response to a trigger signal are the same, the multiple images captured by the multiple camera devices in response to the trigger signal are determined as the second set of time-synchronized matching images.

[0010] In one possible implementation, based on the first set of time-synchronized matching images, process matching is performed on images captured by multiple camera devices after the initial matching to determine the second set of time-synchronized matching images. This includes: obtaining the timestamp of each image in the first set of matching images, and obtaining the timestamp of the image captured by each camera device in response to a trigger signal; calculating the interval between the timestamp of the image captured by each camera device in response to the trigger signal and the timestamp of the corresponding image captured by the camera device in the first set of matching images; if the difference between the intervals corresponding to the multiple images captured by the multiple camera devices in response to the trigger signal is within a preset difference range, the multiple images captured by the multiple camera devices in response to the trigger signal are determined as the second set of time-synchronized matching images.

[0011] In one possible implementation, the images captured by multiple camera devices in the process matching are synchronized in time based on the first set of matching images and the second set of matching images. This includes: calculating the runtime of each camera device based on the timestamps of the first set of matching images and the second set of matching images; calculating the ratio between the clock crystal oscillator frequencies of the multiple camera devices based on the runtime of the multiple camera devices; and calibrating the runtime of the multiple camera devices based on the ratio between the clock crystal oscillator frequencies of the multiple camera devices.

[0012] In one possible implementation, the method further includes filtering images captured by multiple camera devices.

[0013] In one possible implementation, filtering is performed on images captured by multiple camera devices, including: calculating the time difference between two adjacent images captured by each camera device; if the time difference between the two adjacent images is not a multiple of the preset period for sending the trigger signal, deleting the image with the later timestamp from the two adjacent images.

[0014] Secondly, embodiments of this application provide an image synchronization device, comprising: a control module configured to send trigger signals to multiple camera devices respectively, controlling the multiple camera devices to capture images; a determination module configured to acquire images captured by the multiple camera devices, perform initial matching on the acquired images, and determine a first set of matching images for time synchronization; the determination module is further configured to perform process matching on images captured by the multiple camera devices after the initial matching based on the first set of matching images for time synchronization, and determine a second set of matching images for time synchronization; and a synchronization module configured to perform time synchronization on images captured by the multiple camera devices after process matching based on the first set of matching images and the second set of matching images.

[0015] Thirdly, embodiments of this application provide an electronic device, which includes a processor and a memory. The memory is used to store instructions, and the processor is used to call the instructions in the memory, causing the electronic device to execute the above-described image synchronization method.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the aforementioned image synchronization method.

[0017] The image synchronization method and related equipment provided in this application can calibrate the image time error during long-term operation of the camera device, repair the problem of discontinuous frame numbers caused by the loss of trigger signals, avoid errors in the time synchronization of image data, and effectively improve the accuracy of image data time synchronization. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 is a schematic diagram of the application environment of the image synchronization method provided in an embodiment of this application.

[0020] Figure 2 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0021] Figure 3 is a flowchart of an image synchronization method provided in an embodiment of this application.

[0022] Figure 4 is a flowchart of an image synchronization method provided in another embodiment of this application.

[0023] Figure 5 is a schematic diagram of the structure of an image synchronization device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] It should be noted that in this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects, not to describe a specific order or sequence.

[0026] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner. Unless otherwise specified, the following embodiments and features described herein can be combined with each other.

[0027] In applications such as stereoscopic photography, virtual reality, and video surveillance, it is necessary to use electronic devices such as smartphones and personal computers as control terminals to control multiple camera devices to simultaneously capture images of the target scene, and to synchronize the image data captured by multiple camera devices in time to ensure the accuracy and relevance of subsequent image analysis and processing.

[0028] Related technologies typically employ frame number synchronization or timestamp synchronization mechanisms for image data time synchronization. Frame number synchronization determines matching by comparing the frame numbers of different image data; if they don't match, the image data with the smaller frame number is deleted until a matching image data is found. However, during operation, the camera may lose a trigger signal, failing to capture an image and thus unable to generate a corresponding frame number. When the next trigger signal is received, the camera captures an image and generates a corresponding frame number, but the lost trigger signal cannot be identified based on the frame number, leading to errors in image data time synchronization. Furthermore, during operation, the camera may capture multiple images in response to a single trigger signal, corresponding to multiple consecutively incremented frame numbers, which also causes time synchronization errors. Timestamp synchronization mechanisms use the timestamp of one of the multiple cameras as the standard timestamp, determining whether the timestamps of images captured by other cameras are within a preset error range. If the timestamps are outside this range, the image data with the earlier timestamp is deleted, and the image with the later timestamp is used as the new standard timestamp, continuing the comparison until a matching image data is found. However, different camera devices use different crystal oscillator frequencies for timing. After running for a period of time, the timing of different camera devices will deviate. For example, after camera device A runs for one minute, the timing duration is 60000ms, while after camera device B runs for one minute, the timing duration is 60013ms. The difference in timing duration between the two camera devices exceeds the preset error range, causing errors in the time synchronization of image data. In addition, due to power supply or calculation errors, the timestamps of two adjacent images captured by the camera device may fluctuate, causing the interval between two adjacent images to exceed the preset value, resulting in errors in the time synchronization of image data. Thus, image data time synchronization based on frame number synchronization mechanisms or timestamp synchronization mechanisms is prone to errors, resulting in low accuracy and poor synchronization effect.

[0029] Referring to Figure 1, this is a schematic diagram of the application environment of an image synchronization method provided in an embodiment of this application. The electronic device 10 is connected to multiple camera devices 20 via a wired or wireless network, or electrically via a USB interface. The electronic device 10 can be a personal computer, smartphone, server, etc., and can deploy a computer program product (e.g., software code, computer-readable instructions, etc.) programmed according to the image synchronization method provided in this embodiment of the application, thereby providing image synchronization services. The camera devices 20 can be cameras, webcams, electronic devices equipped with webcams, etc.

[0030] Referring to Figure 2, this is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The image synchronization method provided in this embodiment is applied to an electronic device 10, which includes, but is not limited to, a processor 110 and a memory 120 connected via a communication bus 130. Figure 2 is merely an example of an electronic device and does not constitute a limitation thereof. In other embodiments, the electronic device may include more components than those shown in the figure.

[0031] The memory 120 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM). The RAM can be directly read and written by the processor 110 and can be used to store executable programs (e.g., machine instructions) of the operating system or other running programs, as well as user and application data. The RAM may include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc.

[0032] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 110. Non-volatile memory can include disk storage devices and flash memory.

[0033] The memory 120 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 110. The one or more computer programs include multiple instructions that, when executed by the processor 110, enable an image synchronization method to be executed on the electronic device 10.

[0034] In other embodiments, the electronic device 10 also includes an external memory interface for connecting to an external memory to expand the storage capacity of the electronic device 10.

[0035] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0036] The processor 110 provides computing and control capabilities; for example, the processor 110 is used to execute computer programs stored in the memory 120 to implement the image synchronization method described above.

[0037] The communication bus 130 is used to provide a channel for communication between the memory 120 and the processor 110 in the electronic device 10.

[0038] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 10. In other embodiments of this application, the electronic device 10 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0039] Referring to Figure 3, a flowchart of an image synchronization method provided in an embodiment of this application is shown. The image synchronization method in this embodiment can be applied to the electronic device 10 shown in Figures 1 and 2. The image synchronization method includes the following steps:

[0040] S101 sends trigger signals to multiple camera devices to control them to take pictures.

[0041] In one embodiment of this application, a trigger signal is used to control a camera device to perform a shooting operation. For example, in a stereoscopic photography application scenario, an electronic device sends a trigger signal to multiple camera devices, controlling the multiple camera devices to shoot at a target object from different shooting angles, thereby obtaining multiple images of the target object. The electronic device simultaneously sends trigger signals to multiple camera devices based on a preset period, and each camera device responds to the trigger signal to shoot one image. For example, the preset period is 0.5 seconds, 1 second, 2 seconds, or other times.

[0042] In one embodiment of this application, after the electronic device is connected to multiple camera devices, the multiple camera devices are initialized through driver software. For example, initialization includes turning on the camera devices and setting the shooting parameters of the camera devices. After the initialization of the multiple camera devices is completed, the data stream between the electronic device and the multiple camera devices is started, and data transmission can be performed between the electronic device and the multiple camera devices, for example, the transmission of image data.

[0043] S102, acquire images captured by multiple camera devices, perform initial matching on the acquired images, and determine the first set of matching images for time synchronization.

[0044] In one embodiment of this application, after the camera device captures an image of the target object, it automatically transmits the captured image to the electronic device. In another embodiment of this application, the electronic device may also send an image acquisition command to the camera device, and the camera device, in response to the image acquisition command, transmits the captured image to the electronic device.

[0045] In one embodiment of this application, after each camera device captures an image in response to a trigger signal, it transmits the captured image to an electronic device. Each image carries frame number information. The frame numbers of the images captured by each camera device are compared. If the frame numbers of the images captured by each camera device are the same, the multiple images with the same frame number are determined as the first set of matching images for time synchronization. Each time a camera device receives a trigger signal, it captures an image in response to the trigger signal and numbers the images according to the capturing order, thereby obtaining the frame number of the image. For example, the frame number of the first captured image is 1, the frame number of the second image is 2, and so on. The frame numbers of multiple images captured by multiple camera devices are compared to determine the multiple images with the same frame number, and these multiple images with the same frame number are determined as the first set of matching images for time synchronization. For example, the electronic device is connected to three camera devices and controls the three camera devices to capture three images. If the frame number of all three images is 1, then these three images are determined as the first set of matching images for time synchronization.

[0046] In another embodiment of this application, multiple images captured by multiple camera devices in response to the same trigger signal can be determined as a first set of time-synchronized matching images. Each camera device captures one image in response to the trigger signal, and the multiple images captured by multiple camera devices in response to the same trigger signal are determined as a first set of time-synchronized matching images.

[0047] S103, based on the first set of time-synchronized matching images, perform process matching on the images captured by multiple camera devices after the initial matching, and determine the second set of time-synchronized matching images.

[0048] In one embodiment of this application, after determining the first set of time-synchronized matching images, the electronic device continues to send trigger signals to multiple camera devices based on a preset period to control the multiple camera devices to capture images. Based on the first set of time-synchronized matching images, process matching is performed on the images captured by the multiple camera devices to determine the second set of time-synchronized matching images.

[0049] In one embodiment of this application, a frame number synchronization mechanism is used to perform process matching on images captured by multiple camera devices to determine a second set of time-synchronized matching images. It is determined whether the frame numbers of the images captured by each camera device in response to a trigger signal are the same. If the frame numbers of the images captured by each camera device in response to the trigger signal are the same, the multiple images captured by the multiple camera devices in response to the trigger signal are determined as the second set of time-synchronized matching images. For example, if the frame number of the first set of images is 1, and the frame number of the images captured by each camera device in response to the trigger signal is 2, then the frame numbers of the images captured by each camera device in response to the trigger signal are the same, and the multiple images captured by the multiple camera devices in response to the trigger signal with frame number 2 are determined as the second set of time-synchronized matching images.

[0050] In another embodiment of this application, a timestamp synchronization mechanism is used to perform process matching on images captured by multiple camera devices to determine a second set of time-synchronized matching images. The timestamp of each image in the first set of matching images is obtained, as well as the timestamp of the image captured by each camera device in response to a trigger signal. The interval between the timestamp of the image captured by each camera device in response to the trigger signal and the timestamp of the corresponding image captured by the camera device in the first set of matching images is calculated. It is determined whether the difference between the intervals corresponding to the multiple images captured by the multiple camera devices in response to the trigger signal is within a preset difference range. If the difference between the intervals corresponding to the multiple images captured by the multiple camera devices in response to the trigger signal is within the preset difference range, the multiple images captured by the multiple camera devices in response to the trigger signal are determined as the second set of time-synchronized matching images. For example, the preset difference range is [-0.1s, 0.1s].

[0051] For example, an electronic device is connected to two camera devices, namely camera device A and camera device B. In the first set of matching images, the timestamp of image a1 captured by camera device A in response to trigger signal s1 is 16:30:25.1, the timestamp of image b1 captured by camera device B in response to trigger signal s1 is 16:30:25.1, the timestamp of image a2 captured by camera device A in response to trigger signal s2 is 16:30:26.5, and the timestamp of image b2 captured by camera device B in response to trigger signal s2 is 16:30:26.6. The timestamp format is hours:minutes:seconds. Therefore, image a2 and image a... The time interval between the timestamps of image b1 and image b2 is 1.4s, the time interval between the timestamps of image b2 and image b1 is 1.5s, and the difference between the time interval between the timestamps of image a2 and image a1 and the time interval between the timestamps of image b2 and image b1 is -0.1s. Within the preset difference range, image a2 captured by camera device A matches image b2 captured by camera device B, and image a2 captured by camera device A and image b2 captured by camera device B are determined to be the second set of matched images with time synchronization.

[0052] S104, based on the first set of matching images and the second set of matching images, synchronize the images captured by multiple camera devices after the process matching.

[0053] In one embodiment of this application, after determining the second set of matching images for time synchronization, the electronic device continues to send trigger signals to multiple camera devices based on a preset period to control the multiple camera devices to capture images. The runtime of each camera device is calculated based on the timestamps of the first and second sets of matching images. The ratio between the clock oscillator frequencies of the multiple camera devices is calculated based on their runtimes. The runtimes of the multiple camera devices are then calibrated based on this ratio, thereby synchronizing the images captured by the multiple camera devices in time.

[0054] In one embodiment of this application, the runtime of the camera device is obtained by calculating the difference between the timestamp of an image captured by a camera device in the second set of matched images and the timestamp of the same image captured by the same camera device in the first set of matched images. For example, an electronic device is connected to two camera devices, namely camera device A and camera device B. In the first set of matched images, the timestamp of image a1 captured by camera device A is 16:30:25.1, and the timestamp of image b1 captured by camera device B is 16:30:25.1. In the second set of matched images, the timestamp of image a2 captured by camera device A is 16:30:26.5, and the timestamp of image b2 captured by camera device B is 16:30:26.7. Then, the runtime of camera device A is 1.4s, and the runtime of camera device B is 1.6s.

[0055] In one embodiment of this application, the crystal oscillator frequency of the camera device is the ratio between the timed running duration and the actual duration of the camera device. Since the crystal oscillator frequencies of different camera devices are inconsistent, their timed running durations differ within the same actual duration. Therefore, when the actual duration is the same, the ratio between the crystal oscillator frequencies of multiple camera devices is the ratio between their running durations. For example, if the running duration of camera device A is 1.4s and the running duration of camera device B is 1.6s, then the ratio between the crystal oscillator frequency of camera device A and the crystal oscillator frequency of camera device B is R = 1.4 / 1.6 = 7 / 8.

[0056] In one embodiment of this application, the runtime of one of the multiple camera devices is used as the standard runtime. The runtimes of the other camera devices are multiplied by the crystal oscillator frequency to obtain the calibrated runtime. For example, after process matching, in camera devices A and B, the runtime of camera device A is used as the standard runtime. If the runtime of both camera device A and camera device B is 3 seconds, and the ratio R between the crystal oscillator frequency of camera device A and the crystal oscillator frequency of camera device B is 7 / 8, then the calibrated runtime of camera device B is 3.4 seconds. Based on the calibrated runtime, the time-synchronized image among the multiple images captured by the multiple camera devices can be determined, thereby achieving time synchronization of image data. For example, in the second set of matched images, the timestamp of image a2 taken by camera device A is 16:30:26.5, and the timestamp of image b2 taken by camera device B is 16:30:26.7. After the process matching, the timestamp of image an taken by camera device A is 16:30:29.5. The runtime between images a2 and an taken by camera device A is 3 seconds. The timestamp of image bn taken by camera device B, which is time-synchronized with image an, is 16:30:30.1. Thus, the runtime between images b2 and bn taken by camera device B is 3.4 seconds, corresponding to the calibrated runtime.

[0057] Referring to Figure 4, a flowchart of an image synchronization method provided in another embodiment of this application is shown. The image synchronization method in this embodiment can be applied to the electronic device 10 shown in Figures 1 and 2. The image synchronization method includes the following steps:

[0058] S201 sends trigger signals to multiple camera devices to control them to take pictures.

[0059] S202, acquire images captured by multiple camera devices, perform initial matching on the acquired images, and determine the first set of images that are synchronized in time among the multiple images.

[0060] S203 filters images captured by multiple camera devices.

[0061] In one embodiment of this application, the time difference between two adjacent images captured by each camera device is calculated, and it is determined whether the time difference between two adjacent images is a multiple of the preset period for sending the trigger signal. If the time difference between two adjacent images is not a multiple of the preset period, the image with the later timestamp among the two adjacent images is deleted; if the time difference between two adjacent images is a multiple of the preset period, the two adjacent images are retained.

[0062] S204, based on the first set of time-synchronized images, perform process matching on images captured by multiple camera devices to determine the second set of time-synchronized images.

[0063] S205, based on the first set of images and the second set of images, synchronizes the images captured by multiple camera devices in time.

[0064] The specific implementation methods of steps S201, S202, S204, and S205 are the same as those of steps S101-S104, and will not be repeated here.

[0065] Referring to Figure 5, a schematic diagram of the structure of an image synchronization device provided in an embodiment of this application is shown. In one embodiment of this application, the image synchronization device 200 may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the image synchronization device 200 may be stored in the memory of the electronic device 10 and executed by at least one processor to perform the image synchronization function (see Figures 3 and 4 for details).

[0066] In one embodiment of this application, the image synchronization device 200 can be divided into multiple functional modules according to the functions it performs. The functional modules of the image synchronization device 200 may include: a control module 201, a determination module 202, a synchronization module 203, and a filtering module 204. In this embodiment, a module refers to a series of computer program segments that can be executed by at least one processor and perform a fixed function, and are stored in memory.

[0067] The control module 201 is configured to send trigger signals to multiple camera devices respectively, and control the multiple camera devices to take pictures.

[0068] The determination module 202 is configured to acquire images captured by multiple camera devices, perform initial matching on the acquired images, and determine the first set of matching images that are synchronized in time.

[0069] The determining module 202 is also configured to perform process matching on images captured by multiple camera devices after the initial matching based on the first set of time-synchronized matching images.

[0070] The synchronization module 203 is configured to synchronize the images captured by multiple camera devices after the process matching based on the first set of matching images and the second set of matching images.

[0071] The filtering module 204 is configured to filter images captured by multiple camera devices.

[0072] The image synchronization method and apparatus provided in this application can calibrate the image time error during long-term operation of the camera device, repair the problem of discontinuous frame numbers caused by the loss of trigger signals, avoid errors in the time synchronization of image data, and effectively improve the accuracy of image data time synchronization.

[0073] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, and the method implemented when the program instructions are executed can refer to the methods in the above embodiments of this application.

[0074] The computer-readable storage medium can be the internal memory of the electronic device described in the above embodiments, such as the hard disk or memory of the electronic device. Alternatively, the computer-readable storage medium can be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device.

[0075] In one embodiment of this application, the computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function, etc.; and the data storage area may store data created based on the use of the electronic device, etc.

[0076] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0077] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0078] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0079] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0080] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application. Industrial applicability

[0081] The image synchronization method and related equipment disclosed herein can calibrate the image time error during long-term operation of the camera device, repair the frame number discontinuity problem caused by the loss of trigger signal, avoid errors in the time synchronization of image data, effectively improve the accuracy of image data time synchronization, and have strong industrial applicability.

Claims

1. An image synchronization method, wherein, The method includes: Trigger signals are sent to multiple camera devices respectively to control the multiple camera devices to take pictures; Acquire images captured by the multiple camera devices, perform initial matching on the acquired images, and determine the first set of matching images that are synchronized in time; Based on the first set of time-synchronized matching images, process matching is performed on the images captured by the multiple camera devices after the initial matching to determine the second set of time-synchronized matching images; Based on the first set of matched images and the second set of matched images, the images captured by the multiple camera devices after the process matching are synchronized in time.

2. The image synchronization method as described in claim 1, wherein, The step of sending trigger signals to multiple camera devices to control the multiple camera devices to take pictures includes: Trigger signals are simultaneously sent to the multiple camera devices at a preset period, and each camera device captures an image in response to each trigger signal.

3. The image synchronization method as described in claim 1, wherein, The initial matching of the acquired images to determine the first set of matching images for time synchronization includes: Compare the frame numbers of the images captured by each camera device. If the frame numbers of the images captured by each camera device are the same, the multiple images with the same frame number are identified as the first set of matching images that are time-synchronized.

4. The image synchronization method as described in claim 1, wherein, The initial matching of the acquired images to determine the first set of matching images for time synchronization includes: Multiple images captured by the multiple camera devices in response to the same trigger signal are determined as the first set of matched images that are time-synchronized.

5. The image synchronization method as described in claim 1, wherein, The step of performing process matching on images captured by the multiple camera devices after the initial matching, based on the first set of time-synchronized matching images, to determine the second set of time-synchronized matching images includes: If the frame numbers of the images captured by each camera device in response to a trigger signal are the same, the multiple images captured by the multiple camera devices in response to the trigger signal are determined as the second set of matching images that are time-synchronized.

6. The image synchronization method as described in claim 1, wherein, The step of performing process matching on images captured by the multiple camera devices after the initial matching, based on the first set of time-synchronized matching images, to determine the second set of time-synchronized matching images includes: Obtain the timestamp of each image in the first set of matched images, and obtain the timestamp of the image captured by each camera device in response to a trigger signal; Calculate the time interval between the timestamp of the image captured by each camera device in response to the trigger signal and the timestamp of the image captured by the corresponding camera device in the first set of matched images; If the time difference between the intervals of the multiple images captured by the multiple camera devices in response to the trigger signal is within a preset difference range, the multiple images captured by the multiple camera devices in response to the trigger signal are determined as the second set of matched images that are time-synchronized.

7. The image synchronization method as described in claim 1, wherein, The step of synchronizing the images captured by the multiple camera devices after the matching process, based on the first set of matching images and the second set of matching images, includes: The runtime of each camera device is calculated based on the timestamps of the first set of matched images and the second set of matched images. The ratio between the clock crystal oscillator frequencies of the multiple camera devices is calculated based on the runtime of the multiple camera devices. The runtime of the multiple camera devices is calibrated based on the ratio between the clock crystal oscillator frequencies of the multiple camera devices.

8. The image synchronization method as described in claim 1, wherein, The method further includes: The images captured by the multiple camera devices are filtered.

9. The image synchronization method as described in claim 8, wherein, The filtering process for the images captured by the plurality of camera devices includes: Calculate the time difference between two adjacent images captured by each camera device; If the time difference between two adjacent images is not a multiple of the preset period for sending the trigger signal, delete the image with the later timestamp among the two adjacent images.

10. An image synchronization device, wherein, The image synchronization device includes: The control module is configured to send trigger signals to multiple camera devices respectively, and control the multiple camera devices to take pictures; The determination module is configured to acquire images captured by the plurality of camera devices, perform initial matching on the acquired images, and determine the first set of matching images that are synchronized in time. The determining module is further configured to perform process matching on the images captured by the plurality of camera devices after the initial matching, based on the first set of time-synchronized matching images; and determine the second set of time-synchronized matching images. The synchronization module is configured to synchronize the images captured by the multiple camera devices after the process matching based on the first set of matching images and the second set of matching images.

11. An electronic device, wherein, The electronic device includes a processor and a memory, the memory being used to store instructions, and the processor being used to invoke the instructions in the memory, causing the electronic device to execute the image synchronization method according to any one of claims 1 to 9.

12. A computer-readable storage medium, wherein, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the image synchronization method as described in any one of claims 1 to 9.

Images