Image synchronization method and related device

Abstract

Embodiments of the present application provide an image synchronization method and related equipment, the method comprises: sending a trigger signal to a plurality of camera devices respectively, controlling the plurality of camera devices to shoot; obtaining images shot by the plurality of camera devices, performing initial matching on the obtained images, and determining a first group of time-synchronized matched images; performing process matching on the images shot by the plurality of camera devices after the initial matching according to the first group of time-synchronized matched images, and determining a second group of time-synchronized matched images; and performing time synchronization on the images shot by the plurality of camera devices after the process matching according to the first group of matched images and the second group of matched images. Embodiments of the present application can avoid errors in time synchronization of image data, and effectively improve the accuracy of time synchronization of image data.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image processing, and particularly relates to an image synchronization method and related equipment. BACKGROUND

[0002] In application scenarios such as stereophotography, virtual reality, video monitoring, and three-dimensional scanning, it is necessary to control multiple camera devices to perform synchronous shooting through electronic devices such as smartphones and personal computers as control terminals, and to perform time synchronization on image data obtained by the multiple camera devices, so as to ensure the accuracy and relevance of subsequent image analysis and processing. The related art usually adopts a frame number synchronization mechanism or a timestamp synchronization mechanism to perform time synchronization on image data. However, time synchronization on image data based on the frame number synchronization mechanism or the timestamp synchronization mechanism is prone to errors, resulting in low accuracy and poor synchronization effect of time synchronization on image data. SUMMARY

[0003] Therefore, the embodiments of the present application provide an image synchronization method and related equipment to solve the problem that image data synchronization based on a frame number synchronization mechanism or a timestamp synchronization mechanism is prone to errors, resulting in poor synchronization effect of image data.

[0004] In a first aspect, the embodiments of the present application provide an image synchronization method, which includes: sending trigger signals to multiple camera devices respectively, and controlling the multiple camera devices to perform shooting; obtaining images obtained by the multiple camera devices, performing initial matching on the obtained images, and determining a first group of matched images for time synchronization; performing process matching on images obtained by the multiple camera devices after initial matching according to the first group of matched images for time synchronization, and determining a second group of matched images for time synchronization; and performing time synchronization on images obtained by the multiple camera devices after process matching according to the first group of matched images and the second group of matched images.

[0005] In a possible implementation, the sending of the trigger signals to the multiple camera devices respectively and the control of the multiple camera devices to perform shooting include: sending trigger signals to the multiple camera devices simultaneously and respectively based on a preset period, and each camera device shooting an image in response to each trigger signal.

[0006] In a possible implementation, the initial matching on the obtained images and the determination of the first group of matched images for time synchronization include: comparing frame numbers of images obtained by each camera device, and if the frame numbers of the images obtained by each camera device are the same, determining multiple images with the same frame numbers as the first group of matched images for time synchronization.

[0007] In a possible implementation, the initial matching of the acquired images to determine a first set of time-synchronized matched images comprises: determining the images captured by the plurality of cameras in response to the same trigger signal as the first set of time-synchronized matched images.

[0008] In a possible implementation, the process matching of the images captured by the plurality of cameras after the initial matching according to the first set of time-synchronized matched images to determine a second set of time-synchronized matched images comprises: determining the images captured by the plurality of cameras in response to the same trigger signal as the second set of time-synchronized matched images if the frame numbers of the images captured by each camera in response to the trigger signal are the same.

[0009] In a possible implementation, the process matching of the images captured by the plurality of cameras after the initial matching according to the first set of time-synchronized matched images to determine a second set of time-synchronized matched images comprises: acquiring the time stamps of the images in the first set of time-synchronized matched images, and acquiring the time stamps of the images captured by each camera in response to the trigger signal; calculating the interval time between the time stamp of the image captured by each camera in response to the trigger signal and the time stamp of the image captured by the corresponding camera in the first set of time-synchronized matched images; and determining the images captured by the plurality of cameras in response to the trigger signal as the second set of time-synchronized matched images if the difference between the interval times corresponding to the images captured by the plurality of cameras in response to the trigger signal is within a preset difference range.

[0010] In a possible implementation, the time synchronization of the process matching of the images captured by the plurality of cameras according to the first set of time-synchronized matched images and the second set of time-synchronized matched images comprises: calculating the running time of each camera according to the time stamps of the first set of time-synchronized matched images and the second set of time-synchronized matched images, calculating the ratio between the clock crystal frequencies of the plurality of cameras according to the running times of the plurality of cameras, and calibrating the running times of the plurality of cameras according to the ratio between the clock crystal frequencies of the plurality of cameras.

[0011] In a possible implementation, the method further comprises: performing filtering processing on the images captured by the plurality of cameras.

[0012] In a possible implementation, the filtering processing on the images captured by the plurality of cameras comprises: calculating the time difference between two adjacent images captured by each camera; and deleting the image with the later time stamp of the two adjacent images if the time difference between the two adjacent images is not a multiple of the preset period of sending the trigger signal.

[0013] In a second aspect, an embodiment of the present application provides an image synchronization device, which comprises: a control module, configured to send a trigger signal to a plurality of camera devices respectively, and control the plurality of camera devices to capture images; a determination module, configured to acquire images captured by the plurality of camera devices, perform initial matching on the acquired images, and determine a first set of time-synchronized matched images; the determination module is further configured to perform process matching on images captured by the plurality of camera devices after the initial matching according to the first set of time-synchronized matched images, and determine a second set of time-synchronized matched images; and a synchronization module, configured to perform time synchronization on images captured by the plurality of camera devices after the process matching according to the first set of matched images and the second set of matched images.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, which comprises a processor and a memory, the memory is configured to store instructions, and the processor is configured to invoke the instructions in the memory, so that the electronic device performs the image synchronization method described above.

[0015] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which comprises computer instructions, when the computer instructions run on an electronic device, the electronic device performs the image synchronization method described above.

[0016] The image synchronization method and related device provided by the embodiments of the present application can calibrate the image time error in the long-time running process of the camera device, repair the frame number discontinuity problem caused by the loss of the trigger signal, avoid the error in the time synchronization of the image data, and effectively improve the accuracy of the time synchronization of the image data. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0018] Figure 1 The application environment schematic diagram of the image synchronization method provided by an embodiment of the present application.

[0019] Figure 2 The hardware structure schematic diagram of the electronic device provided by an embodiment of the present application.

[0020] Figure 3 The flowchart of the image synchronization method provided by an embodiment of the present application.

[0021] Figure 4 A flow chart of an image synchronization method provided for another embodiment of the present application.

[0022] Figure 5 A structural schematic diagram of an image synchronization device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be described in detail below with reference to the drawings and specific embodiments.

[0024] It should be noted that “at least one” in the present application means one or more, and “multiple” means two or more than two. “And / or” describes the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The terms “first”, “second”, “third”, “fourth” and the like (if any) in the specification and claims of the present application and the drawings are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0025] In the embodiments of the present application, the words “exemplary” or “for example” are used to mean serving as an example, instance, or illustration. Any embodiment or design presented as “exemplary” or “for example” in the embodiments of the present application should not be interpreted as being more preferred or advantageous than other embodiments or design solutions. Rather, the use of “exemplary” or “for example” is intended to present relevant concepts in a specific manner. The embodiments described below and the features in the embodiments can be combined with each other without conflict, if possible.

[0026] In the application scenarios of stereophotography, virtual reality, video monitoring, etc., it is necessary to control multiple camera devices to synchronously capture target scenes by electronic devices such as smart phones and personal computers as control terminals, and to perform time synchronization on image data captured by the multiple camera devices, so as to ensure the accuracy and relevance of subsequent image analysis and processing.

[0027] The related art usually adopts a frame number synchronization mechanism or a timestamp synchronization mechanism for time synchronization of image data. The frame number synchronization mechanism determines whether image data matches by comparing whether the frame numbers of different image data are the same. If the image data does not match, the image data with a smaller frame number is deleted until the matching image data is determined. However, the camera device may lose a trigger signal during operation and thus cannot capture an image. Therefore, the camera device cannot generate a corresponding frame number. When the next trigger signal is received, the camera device captures an image and generates a corresponding frame number. However, the lost trigger signal cannot be determined according to the frame number, which causes an error in time synchronization of image data. In addition, the camera device may capture multiple images in response to a trigger signal during operation and continuously increase multiple frame numbers corresponding to the images. This also causes an error in time synchronization of image data. The timestamp synchronization mechanism takes the timestamp of one camera device in multiple camera devices as a standard timestamp, determines whether the timestamps of images captured by other camera devices are within a preset error range, deletes image data with an earlier timestamp if the image timestamp is not within the preset error range, takes image data with a later timestamp as a new standard timestamp, and continues to compare until matching image data is determined. However, the crystal oscillator frequencies used for timing by different camera devices are different. After running for a period of time, the timing times of different camera devices deviate, for example, the timing time of camera device A is 60000 ms after running for one minute, and the timing time of camera device B is 60013 ms after running for one minute. The difference between the timing times of the two camera devices exceeds the preset error range, which causes an error in time synchronization of image data. In addition, due to power supply or calculation errors and other reasons, the timestamps of two adjacent images captured by the camera device may fluctuate, causing the interval time between the two adjacent images to exceed a preset value, which causes an error in time synchronization of image data. Therefore, time synchronization of image data based on the frame number synchronization mechanism or the timestamp synchronization mechanism is prone to errors, which causes low accuracy and poor synchronization effect of time synchronization of image data.

[0028] Referring to Figure 1 As shown in FIG. 1, an application environment of an image synchronization method provided by an embodiment of the present application is shown. An electronic device 10 and multiple camera devices 20 are connected in communication through a wired network or a wireless network, or are electrically connected through a USB interface. The electronic device 10 can be a personal computer, a smart phone, a server, or the like. The electronic device 10 can deploy a computer program product (for example, software code, computer readable instructions, or the like) programmed to implement the image synchronization method provided by the embodiment of the present application, thereby providing an image synchronization service. The camera device 20 can be a camera, a camera head, an electronic device configured with a camera head, or the like.

[0029] Referring to Figure 2As shown, it is a hardware structure schematic diagram of an electronic device provided in an embodiment of the present application. The image synchronization method provided in the embodiment of the present application is applied to an electronic device 10, which includes, but is not limited to, a processor 110 and a memory 120 connected through a communication bus 130. Figure 2 It is only an example of the electronic device, and does not constitute a corresponding limitation. In other embodiments, the electronic device can include more components than the diagram.

[0030] The memory 120 can include one or more random access memories (RAMs) and one or more non-volatile memories (NVMs). The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of an operating system or other programs running in the background, and can also be used to store data of users and applications, etc. The random access memory can include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc.

[0031] The non-volatile memory can also store executable programs and store data of users and applications, etc., which can be loaded in advance into the random access memory for direct reading and writing by the processor 110. The non-volatile memory can include a magnetic disk storage device, a flash memory.

[0032] The memory 120 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 110. The one or more computer programs include a plurality of instructions, which when executed by the processor 110, can implement the image synchronization method executed on the electronic device 10.

[0033] In other embodiments, the electronic device 10 further includes an external memory interface for connecting an external memory to realize the expansion of the storage capacity of the electronic device 10.

[0034] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0035] The processor 110 provides computing and control capabilities, for example, the processor 110 is used to execute the computer program stored in the memory 120 to realize the image synchronization method described above.

[0036] The communication bus 130 is used to provide a communication channel between the memory 120 and the processor 110 in the electronic device 10.

[0037] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 10. In other embodiments of the present application, the electronic device 10 can include more or fewer components than the illustration, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0038] Referring to Figure 3 The flow chart of the image synchronization method provided by an embodiment of the present application is shown. The image synchronization method in the embodiment can be applied to the electronic device 10 as shown in Figure 1 and Figure 2 The image synchronization method includes the following steps:

[0039] S101, respectively send a trigger signal to a plurality of camera devices, and control the plurality of camera devices to shoot.

[0040] In an embodiment of the present application, the trigger signal is used to control the camera device to perform the shooting operation. For example, in the application scenario of stereoscopic photography, the electronic device sends a trigger signal to a plurality of camera devices, controls the plurality of camera devices to shoot a target object based on different shooting angles, and obtains a plurality of images of the target object. The electronic device sends a trigger signal to a plurality of camera devices based on a preset period, and each camera device shoots an image in response to the trigger signal. For example, the preset period is 0.5 seconds, 1 second, 2 seconds or other time.

[0041] In an embodiment of the present application, after the electronic device is connected with the plurality of cameras, the plurality of cameras are initialized by driving software, for example, the initialization includes turning on the cameras and setting the shooting parameters of the cameras. After the initialization of the plurality of cameras is completed, the data flow between the electronic device and the plurality of cameras is started, and the data transmission, for example, the image data transmission, can be performed between the electronic device and the plurality of cameras.

[0042] S102, acquiring images shot by the plurality of cameras, performing initial matching on the acquired images, and determining a first group of matching images in time synchronization.

[0043] In an embodiment of the present application, after the camera shoots the image of the target object, the shot image is automatically transmitted to the electronic device. In another embodiment of the present application, the electronic device can also send an image acquisition instruction to the camera, and the camera transmits the shot image to the electronic device in response to the image acquisition instruction.

[0044] In an embodiment of the present application, after the camera shoots an image in response to each trigger signal, the shot image is transmitted to the electronic device, each image carries frame number information, the frame numbers of the images shot by each camera are compared, and if the frame numbers of the images shot by each camera are the same, the plurality of images with the same frame number are determined as the first group of matching images in time synchronization. When the camera receives a trigger signal, the camera shoots an image in response to the trigger signal, and numbers the images according to the shooting order, thereby obtaining the frame number of the image. For example, the frame number of the first shot image is 1, the frame number of the second shot image is 2, and so on. The frame numbers of the plurality of images shot by the plurality of cameras are compared, the plurality of images with the same frame number are determined, and the plurality of images with the same frame number are determined as the first group of matching images in time synchronization. For example, the electronic device is connected with three cameras, the three cameras are controlled to shoot, three images shot by the three cameras are acquired, and if the frame numbers of the three images are all 1, the three images are determined as the first group of matching images in time synchronization.

[0045] In another embodiment of the present application, the plurality of images shot by the plurality of cameras in response to the same trigger signal can also be determined as the first group of matching images in time synchronization. Each camera shoots an image in response to the trigger signal, and the plurality of images shot by the plurality of cameras in response to the same trigger signal are determined as the first group of matching images in time synchronization.

[0046] S103, performing process matching on the images shot by the plurality of cameras after the initial matching according to the first group of matching images in time synchronization, and determining a second group of matching images in time synchronization.

[0047] In an embodiment of the present application, after determining the first set of time-synchronized matching images, the electronic device continues to send a trigger signal to the plurality of cameras based on a preset period, and controls the plurality of cameras to capture images. Then, the images captured by the plurality of cameras are matched based on the first set of time-synchronized matching images, and a second set of time-synchronized matching images is determined.

[0048] In an embodiment of the present application, the frame number synchronization mechanism is used to match the images captured by the plurality of cameras, and a second set of time-synchronized matching images is determined. It is determined whether the frame numbers of the images captured by each camera in response to a trigger signal are the same. If the frame numbers of the images captured by each camera in response to the trigger signal are the same, the images captured by the plurality of cameras in response to the trigger signal are determined as the second set of time-synchronized matching images. For example, if the frame number of the first set of images is 1, and the frame numbers of the images captured by each camera in response to the trigger signal are all 2, the frame numbers of the images captured by each camera in response to the trigger signal are the same, and the images with the frame number 2 captured by the plurality of cameras in response to the trigger signal are determined as the second set of time-synchronized matching images.

[0049] In another embodiment of the present application, the timestamp synchronization mechanism is used to match the images captured by the plurality of cameras, and a second set of time-synchronized matching images is determined. The timestamps of the images in the first set of matching images are obtained, and the timestamps of the images captured by each camera in response to a trigger signal are obtained. The interval time between the timestamp of the image captured by each camera in response to the trigger signal and the timestamp of the image captured by the corresponding camera in the first set of matching images is calculated. It is determined whether the difference between the interval times corresponding to the images captured by the plurality of cameras in response to the trigger signal is within a preset difference range. If the difference between the interval times corresponding to the images captured by the plurality of cameras in response to the trigger signal is within the preset difference range, the images captured by the plurality of cameras in response to the trigger signal are determined as the second set of time-synchronized matching images. For example, the preset difference range is [-0.1s, 0.1s].

[0050] For example, the electronic device is connected with two cameras A and B. In the first set of matched images, the time stamp of image a1 taken by camera A in response to trigger signal s1 is 16:30:25.1, the time stamp of image b1 taken by camera B in response to trigger signal s1 is 16:30:25.1, the time stamp of image a2 taken by camera A in response to trigger signal s2 is 16:30:26.5, and the time stamp of image b2 taken by camera B in response to trigger signal s2 is 16:30:26.6. The format of the time stamp is hour:minute:second. The interval time between the time stamp of image a2 and image a1 is 1.4 s, the interval time between the time stamp of image b2 and image b1 is 1.5 s, and the difference between the interval time between the time stamp of image a2 and image a1 and the interval time between the time stamp of image b2 and image b1 is -0.1 s. Since the difference is within the preset difference range, image a2 taken by camera A is matched with image b2 taken by camera B, and it is determined that image a2 taken by camera A and image b2 taken by camera B are the second set of matched images that are time-synchronized.

[0051] In S104, the images taken by the plurality of cameras after the process of matching are time-synchronized according to the first set of matched images and the second set of matched images.

[0052] In an embodiment of the present application, after the second set of matched images that are time-synchronized is determined, the electronic device continues to send trigger signals to the plurality of cameras based on the preset period to control the plurality of cameras to take images. The running time of each camera is calculated according to the time stamp of the first set of matched images and the second set of matched images. The ratio between the clock crystal frequencies of the plurality of cameras is calculated according to the running time of the plurality of cameras. The running time of the plurality of cameras is calibrated according to the ratio between the clock crystal frequencies of the plurality of cameras, so that the images taken by the plurality of cameras are time-synchronized.

[0053] In an embodiment of the present application, the difference between the time stamp of an image taken by a camera in the second set of matched images and the time stamp of an image taken by the camera in the first set of matched images is calculated to obtain the running time of the camera. For example, the electronic device is connected with two cameras A and B. In the first set of matched images, the time stamp of image a1 taken by camera A is 16:30:25.1, and the time stamp of image b1 taken by camera B is 16:30:25.1. In the second set of matched images, the time stamp of image a2 taken by camera A is 16:30:26.5, and the time stamp of image b2 taken by camera B is 16:30:26.7. The running time of camera A is 1.4 s, and the running time of camera B is 1.6 s.

[0054] In an embodiment of the present application, the ratio between the crystal frequency of the camera and the running time of the camera is the ratio between the running time of the camera and the actual time. Since the crystal frequencies of different cameras are inconsistent, the running time of different cameras is different in the same actual time. Therefore, the ratio between the crystal frequencies of the multiple cameras is the ratio between the running time of the multiple cameras. For example, if the running time of camera A is 1.4s and the running time of camera B is 1.6s, the ratio between the crystal frequency of camera A and the crystal frequency of camera B is R = 1.4 / 1.6 = 7 / 8.

[0055] In an embodiment of the present application, the running time of one of the multiple cameras is taken as the standard running time, and the running time of the other cameras is multiplied by the ratio of the crystal frequency to obtain the calibrated running time. For example, after process matching, the running time of camera A is taken as the standard running time in cameras A and B. If the running time of camera A and camera B is 3s, and the ratio R between the crystal frequency of camera A and the crystal frequency of camera B is 7 / 8, the calibrated running time of camera B is 3.4s. Based on the calibrated running time, the time-synchronized images in the multiple images captured by the multiple cameras can be determined, so as to realize the time synchronization of the image data. For example, in the second set of matched images, the timestamp of image a2 captured by camera A is 16:30:26.5, and the timestamp of image b2 captured by camera B is 16:30:26.7. After process matching, the timestamp of image an captured by camera A is 16:30:29.5, and the running time between image a2 and an captured by camera A is 3s. The timestamp of image bn captured by camera B is 16:30:30.1, which is time-synchronized with image an. Thus, the running time between image b2 and bn captured by camera B is 3.4s, which corresponds to the calibrated running time.

[0056] Referring to Figure 4 FIG. 2 is a flowchart of an image synchronization method provided by another embodiment of the present application. The image synchronization method in the embodiment can be applied to the electronic device 10 as shown in Figure 1 and Figure 2 The image synchronization method includes the following steps:

[0057] S201, respectively sending trigger signals to the multiple cameras to control the multiple cameras to capture images.

[0058] S202, obtaining the images captured by the multiple cameras, and performing initial matching on the obtained images to determine a first set of time-synchronized images in the multiple images.

[0059] S203, performing filtering processing on the images captured by the plurality of cameras.

[0060] In an embodiment of the present application, a time difference between two adjacent images captured by each camera is calculated, and it is determined whether the time difference between the two adjacent images is a multiple of a preset period of the trigger signal. If the time difference between the two adjacent images is not a multiple of the preset period, the image with a later time stamp is deleted from the two adjacent images. If the time difference between the two adjacent images is a multiple of the preset period, the two adjacent images are retained.

[0061] S204, performing process matching on the images captured by the plurality of cameras according to the first set of time-synchronized images, to determine a second set of time-synchronized images.

[0062] S205, performing time synchronization on the images captured by the plurality of cameras according to the first set of images and the second set of images.

[0063] The specific embodiments of S201, S202, S204, and S205 are the same as those of S101-S104, and are not repeated here.

[0064] Referring to Figure 5 FIG. 2 shows a structural schematic diagram of an image synchronization device according to an embodiment of the present application. In an embodiment of the present application, the image synchronization device 200 can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the image synchronization device 200 can be stored in the memory of the electronic device 10 and executed by at least one processor to perform the image synchronization function (see detailed description of Figure 3 and 4 ).

[0065] In an embodiment of the present application, the image synchronization device 200 can be divided into a plurality of functional modules according to the functions performed thereby. The functional modules of the image synchronization device 200 can include a control module 201, a determination module 202, a synchronization module 203, and a filtering module 204. The modules in the embodiment of the present application refer to a series of computer program segments that can be executed by at least one processor and can complete a fixed function, which are stored in the memory.

[0066] The control module 201 is configured to send trigger signals to the plurality of cameras respectively, and control the plurality of cameras to capture images.

[0067] The determination module 202 is configured to obtain images captured by the plurality of cameras, perform initial matching on the obtained images, and determine a first set of time-synchronized matching images.

[0068] The determining module 202 is further configured to perform process matching on the images captured by the plurality of cameras after the initial matching according to the first set of time-synchronized matching images, to determine a second set of time-synchronized matching images.

[0069] The synchronizing module 203 is configured to perform time synchronization on the images captured by the plurality of cameras after the process matching according to the first set of matching images and the second set of matching images.

[0070] The filtering module 204 is configured to perform filtering processing on the images captured by the plurality of cameras.

[0071] The image synchronization method and device provided by the embodiments of the present application can calibrate the image time error in the long-time running process of the camera, repair the frame number discontinuity problem caused by the loss of the trigger signal, avoid the error in the time synchronization of the image data, and effectively improve the accuracy of the time synchronization of the image data.

[0072] The embodiments of the present application further provide a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program includes program instructions. The method implemented by the program instructions can refer to the method in the above embodiments of the present application.

[0073] The computer readable storage medium can be an internal storage of the electronic device, for example, a hard disk or a memory of the electronic device. The computer readable storage medium can also be an external storage of the electronic device, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0074] In an embodiment of the present application, the computer readable storage medium can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application required by a function, etc. The data storage area can store data created according to the use of the electronic device, etc.

[0075] In the above embodiments, the description of each embodiment has its own focus. The parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0076] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0077] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / terminal device and method can be implemented in other ways. For example, the division of the described apparatus / terminal device embodiments is merely a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0078] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0079] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An image synchronization method, characterized in that, The method includes: Trigger signals are sent to multiple camera devices respectively to control the multiple camera devices to take pictures; Acquire images captured by the multiple camera devices, perform initial matching on the acquired images, and determine the first set of matching images that are synchronized in time; Based on the first set of time-synchronized matching images, process matching is performed on the images captured by the multiple camera devices after the initial matching to determine the second set of time-synchronized matching images; Based on the first set of matched images and the second set of matched images, time synchronization is performed on the images captured by the multiple camera devices after the process matching, including: calculating the runtime of each camera device based on the timestamps of the first set of matched images and the second set of matched images; calculating the ratio between the clock crystal oscillator frequencies of the multiple camera devices based on the runtime of the multiple camera devices; and calibrating the runtime of the multiple camera devices based on the ratio between the clock crystal oscillator frequencies of the multiple camera devices.

2. The image synchronization method as described in claim 1, characterized in that, The step of sending trigger signals to multiple camera devices to control the multiple camera devices to take pictures includes: Trigger signals are simultaneously sent to the multiple camera devices at a preset period, and each camera device captures an image in response to each trigger signal.

3. The image synchronization method as described in claim 1, characterized in that, The initial matching of the acquired images to determine the first set of matching images for time synchronization includes: Compare the frame numbers of the images captured by each camera device. If the frame numbers of the images captured by each camera device are the same, the multiple images with the same frame number are identified as the first set of matching images that are time-synchronized.

4. The image synchronization method as described in claim 1, characterized in that, The initial matching of the acquired images to determine the first set of matching images for time synchronization includes: Multiple images captured by the multiple camera devices in response to the same trigger signal are determined as the first set of matched images that are time-synchronized.

5. The image synchronization method as described in claim 1, characterized in that, The step of performing process matching on images captured by the multiple camera devices after the initial matching, based on the first set of time-synchronized matching images, to determine the second set of time-synchronized matching images includes: If the frame numbers of the images captured by each camera device in response to a trigger signal are the same, the multiple images captured by the multiple camera devices in response to the trigger signal are determined as the second set of matching images that are time-synchronized.

6. The image synchronization method as described in claim 1, characterized in that, The step of performing process matching on images captured by the multiple camera devices after the initial matching, based on the first set of time-synchronized matching images, to determine the second set of time-synchronized matching images includes: Obtain the timestamp of each image in the first set of matched images, and obtain the timestamp of the image captured by each camera device in response to a trigger signal; Calculate the time interval between the timestamp of the image captured by each camera device in response to the trigger signal and the timestamp of the image captured by the corresponding camera device in the first set of matched images; If the time difference between the intervals of the multiple images captured by the multiple camera devices in response to the trigger signal is within a preset difference range, the multiple images captured by the multiple camera devices in response to the trigger signal are determined as the second set of matched images that are time-synchronized.

7. The image synchronization method as described in claim 1, characterized in that, The method further includes: The images captured by the multiple camera devices are filtered.

8. The image synchronization method as described in claim 7, characterized in that, The filtering process for the images captured by the plurality of camera devices includes: Calculate the time difference between two adjacent images captured by each camera device; If the time difference between two adjacent images is not a multiple of the preset period for sending the trigger signal, delete the image with the later timestamp among the two adjacent images.

9. An image synchronization device, characterized in that, The image synchronization device includes: The control module is used to send trigger signals to multiple camera devices respectively, and control the multiple camera devices to take pictures; The determination module is used to acquire images captured by the multiple camera devices, perform initial matching on the acquired images, and determine the first set of matching images that are synchronized in time. The determining module is further configured to perform process matching on the images captured by the plurality of camera devices after the initial matching, based on the first set of time-synchronized matching images; and determine the second set of time-synchronized matching images. The synchronization module is used to synchronize the images captured by the multiple camera devices after the process matching based on the first set of matching images and the second set of matching images. This includes: calculating the runtime of each camera device based on the timestamps of the first set of matching images and the second set of matching images; calculating the ratio between the clock crystal oscillator frequencies of the multiple camera devices based on the runtime of the multiple camera devices; and calibrating the runtime of the multiple camera devices based on the ratio between the clock crystal oscillator frequencies of the multiple camera devices.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory being used to store instructions, and the processor being used to invoke the instructions in the memory to cause the electronic device to execute the image synchronization method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the image synchronization method as described in any one of claims 1 to 8.

Images