Method, device and electronic equipment for synchronizing multi-channel images

Abstract

To provide a method for synchronizing multi-channel images, a device, a computer-readable storage medium, and an electronic apparatus.TECHNICAL MEANS: The method includes: determining, from at least two image transmission channels, target image data corresponding to a target image transmission channel at a preset timepoint; determining a mounting status of a frame identifier of the target image data; determining integrity information of the target image data based on the mounting status; generating the frame identifier of the target image data based on the integrity information; and, based on the frame identifier, generating a to-be-output image corresponding to each image transmission channel in the at least two image transmission channels, and outputting the to-be-output image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to the field of computer technology, and in particular to a method, apparatus, computer-readable storage medium and electronic device for synchronizing multi-channel images. [Background technology]

[0002] With the development of image processing technology, currently, in many fields, it is necessary to use multiple cameras to capture the same scene, and image data captured by different cameras is often transmitted and processed through different channels, but different channels cause different delays in the transmission and processing of image data. For example, in the field of intelligent driving, multiple images captured by multiple cameras need to be synchronized to accurately reflect the situation at a given time when the vehicle is traveling, and improper synchronization may cause serious accidents. Therefore, when image data captured by different cameras is transmitted through each channel, how to ensure that the image frames output from the different channels are synchronized in terms of capture time has become a problem to be solved. Summary of the Invention [Problem to be solved by the invention]

[0003] To solve the above technical problems, the embodiments of the present disclosure provide a method, an apparatus, a computer-readable storage medium, and an electronic device for synchronizing multi-channel images, which can solve the problem of difficulty in synchronizing the shooting time due to delays in the transmission and processing of image data in different channels. [Means for solving the problem]

[0004] The method for synchronizing multi-channel images according to the present disclosure includes the steps of: determining target image data corresponding to a preset time of a target image transmission channel from at least two image transmission channels; determining a mounting status of a frame identifier of the target image data; determining integrity information of the target image data based on the mounting status; generating a frame identifier of the target image data based on the integrity information; generating an output target image corresponding to each image transmission channel of the at least two image transmission channels based on the frame identifier, and outputting the output target image.

[0005] The apparatus for synchronizing multi-channel images according to the present disclosure includes a first determination module for determining target image data corresponding to a preset time of a target image transmission channel from at least two image transmission channels, a second determination module for determining a mounting status of a frame identifier of the target image data, a third determination module for determining integrity information of the target image data based on the mounting status, a generation module for generating a frame identifier of the target image data based on the integrity information, and an output module for generating an output target image corresponding to each image transmission channel of the at least two image transmission channels based on the frame identifier, and outputting the output target image.

[0006] A computer-readable storage medium according to the present disclosure stores a computer program that is executed by a processor to implement the method for synchronizing multi-channel images described above.

[0007] The electronic device according to the present disclosure comprises a processor and a memory for storing instructions executable by the processor, the processor being used to implement the above-mentioned method for synchronizing multi-channel images by reading and executing the executable instructions from the memory.

[0008] A computer program product according to the present disclosure implements a method for synchronizing multi-channel images according to the present disclosure when instructions in the computer program product are executed by a processor. [Effects of the Invention]

[0009] The method, apparatus, computer-readable storage medium, and electronic device for synchronizing multi-channel images disclosed herein include detecting frame numbers and detecting data integrity for image data transmitted through at least two image transmission channels to generate frame identifiers for the image data, processing the image data in each image transmission channel to generate output images, and outputting images corresponding to the frame identifiers from each image transmission channel, thereby achieving automatic detection and restoration for image data transmitted through multiple image transmission channels and improving the efficiency and quality of multi-channel image collection. Furthermore, detecting and adjusting the frame identifiers of the image data during the process of transmitting the image data within multiple channels can ensure that the frame identifiers are consistent throughout the entire process of transmitting images captured by multiple cameras through each image transmission channel, helping users extract images captured at the same time from the images output from each image transmission channel, thereby ensuring synchronization of image data when transmitted through multiple image transmission channels.

[0010] The technical solutions of the present disclosure are further described in detail below with reference to figures and examples. [Brief explanation of the drawings]

[0011] The above and other objects, features, and advantages of the present disclosure will become more apparent from the detailed description of the embodiments of the present disclosure with reference to the drawings. The drawings are used to further understand the embodiments of the present disclosure, are constituted as part of the specification, and are used to interpret the present disclosure together with the embodiments of the present disclosure, but are not intended to limit the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps. [Figure 1] FIG. 1 is a system diagram to which the present disclosure applies. [Figure 2] 1 is a flowchart of a method for synchronizing multi-channel images according to an exemplary embodiment of the present disclosure. [Figure 3] 10 is a flowchart of a method for synchronizing multi-channel images according to another exemplary embodiment of the present disclosure. [Figure 4] 10 is a flowchart of a method for synchronizing multi-channel images according to another exemplary embodiment of the present disclosure. [Figure 5] 10 is a flowchart of a method for synchronizing multi-channel images according to another exemplary embodiment of the present disclosure. [Figure 6] 10 is a flowchart of a method for synchronizing multi-channel images according to another exemplary embodiment of the present disclosure. [Figure 7] 10 is a flowchart of a method for synchronizing multi-channel images according to another exemplary embodiment of the present disclosure. [Figure 8] 1 is a structural schematic diagram of an apparatus for synchronizing multi-channel images according to an exemplary embodiment of the present disclosure; [Figure 9] FIG. 10 is a structural schematic diagram of an apparatus for synchronizing multi-channel images according to another exemplary embodiment of the present disclosure. [Figure 10] 1 is a structural diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the drawings. Obviously, the described embodiments are only some of the embodiments of the present disclosure, but not all of the embodiments, and the present disclosure is not limited to the described exemplary embodiments.

[0013] The relative arrangement of components and steps, numerical expressions and values ​​described in these examples do not limit the scope of the present disclosure unless specifically stated otherwise.

[0014] [Application Summary] Currently, the most commonly used method for achieving synchronization of images captured by multiple cameras in terms of shooting time is to synchronously control the shooting operations of the cameras and synchronously output the data captured by the cameras. However, during the data transmission process, factors such as different transmission channels, different processing methods of the image processing modules, and different program execution environments make it impossible for the data from each channel to be processed simultaneously in each module, which further causes the images output from each channel to be asynchronous in terms of time.

[0015] Furthermore, current technology does not allow automatic abnormality analysis when data is transmitted between different image processing modules. The abnormality analysis requires technician parameters, resulting in low data processing efficiency and unable to meet the needs of scenarios such as autonomous driving.

[0016] An embodiment of the present disclosure aims to solve the above problem and provides a method for synchronizing multi-channel images, which can detect frame identifiers and detect data integrity during the process of image data being transmitted over multiple channels, and ultimately output images with the same frame identifiers from multiple channels, thereby achieving synchronized data output after data transmission of multi-channel image data.

[0017] [Example System] FIG. 1 shows an exemplary system architecture 100 in which the method for synchronizing multi-channel images or the apparatus for synchronizing multi-channel images according to embodiments of the present invention can be applied.

[0018] As shown in FIG. 1, the system architecture 100 can include an electronic device 101 and at least two cameras 102 .

[0019] The at least two cameras 102 may be connected to the electronic device by a variety of connection methods, such as, for example, wires, wireless communication links, or fiber optic cables.

[0020] The at least two cameras 102 are used to capture a target scene, which can be various scenes. For example, if the at least two cameras 102 are installed on a vehicle, the target scene can be a scene such as a road where the vehicle is located, a parking lot, etc., and the at least two cameras 102 can capture the environment around the vehicle.

[0021] The electronic device can be provided with at least two image transmission channels, each corresponding to one camera, and each image transmission channel is provided with an image processing module, and image data captured by the camera is transmitted through the corresponding image transmission channel, processed by the image processing module, and then output from the corresponding image transmission channel.

[0022] The electronic device 101 can be, but is not limited to, various electronic devices such as dedicated devices such as chips dedicated to image processing and circuit boards dedicated to image processing, mobile general-purpose terminals such as in-vehicle terminals, mobile phones, notebook computers, and PADs (tablet computers), and fixed general-purpose terminals such as digital televisions and desktop computers.

[0023] The method for synchronizing multi-channel images according to the embodiments of the present disclosure is performed by the electronic device 101 , and accordingly, the device for synchronizing multi-channel images can be provided in the electronic device 101 .

[0024] It should be noted that the number of electronic devices and cameras in FIG. 1 is only approximate, and there can be any number of electronic devices and cameras according to actual needs.

[0025] [Exemplary Method] 2 is a flowchart of a method for synchronizing multi-channel images according to an exemplary embodiment of the present invention. This embodiment can be applied to an electronic device (electronic device 101 shown in FIG. 1), and as shown in FIG. 2, the method includes the following steps 201 to 205.

[0026] In step 201, from at least two image transmission channels, target image data corresponding to a preset time of a target image transmission channel is determined.

[0027] Here, the target image transmission channel can be one of at least two image transmission channels. The preset time can be a specified fixed time or a shooting time of a camera corresponding to the target image transmission channel. The target image data can be data captured by a camera corresponding to the target image transmission channel. Typically, the target image data can be data that has not undergone image processing, for example, raw data collected by a camera sensor.

[0028] In step 202, the mount status of the frame identifier of the target image data is determined.

[0029] Here, the frame identifier can be a mark for distinguishing image data captured at different times, and the frame identifier is typically associated with the capture time of the image data. For example, the frame identifier can include, but is not limited to, at least one of an image frame number, a timestamp indicating the capture time of the image, and the like. Typically, a frame identifier for the image data is generated after capturing one frame of image data. This process is called frame mounting, and the mount status can include two types: successful mounting and failed mounting. Missing or incomplete image data due to software or hardware failure can cause the frame identifier to fail to mount.

[0030] In step 203, the integrity information of the target image data is determined based on the mounting state.

[0031] Here, the integrity information represents the degree of integrity of the target image data. For example, the integrity information may be the number of rows that fail a data check (e.g., a CRC (Cyclic Redundancy Check) check) performed on each row of the image data, or the percentage of the number of rows that fail the check to the total number of rows. If the mounting state fails, it is necessary to obtain the integrity information and then determine whether the frame identifier of the target image data can be repaired.

[0032] In step 204, a frame identifier for the target image data is generated based on the integrity information.

[0033] Specifically, if the completeness information indicates that the target image data is complete (for example, the ratio of the number of successfully inspected rows to the total number of rows is greater than a predetermined ratio), a frame identifier for the target image data can be generated. For example, a frame identifier generation method can obtain the shooting time of the recorded target image data and generate a frame identifier based on the timestamp of the shooting time.

[0034] In step 205, based on the frame identifier, an output target image corresponding to each of the at least two image transmission channels is generated, and the output target image is output.

[0035] Specifically, in the process of transmitting target image data on each image transmission channel, a preset image processing module (e.g., an ISP (Image Signal Processing) model) is used to process the target image data, and an output target image can be obtained after processing. Since the shooting times of multiple output target images corresponding to the same frame identifier are the same or similar, the electronic device can simultaneously output output target images with the same frame identifier generated on each image transmission channel.

[0036] Optionally, when processed images corresponding to the frame identifiers are acquired from each image transmission channel, or when the number of acquired processed images corresponding to the frame identifiers is equal to or greater than a predetermined number, these processed images are determined as images to be output, and these images to be output can be output. If the number of acquired processed images corresponding to the frame identifiers is less than the predetermined number, the processed images corresponding to the frame identifiers are discarded, and step 201 is executed again.

[0037] The method according to the above embodiment of the present disclosure detects frame numbers and detects data integrity for image data transmitted through at least two image transmission channels, generates frame identifiers for the image data, processes the image data in each image transmission channel to generate images to be output, and outputs images corresponding to the frame identifiers via each image transmission channel, thereby achieving automatic detection and restoration for image data transmitted through multiple image transmission channels and improving the efficiency and quality of multi-channel image collection. Moreover, by detecting and adjusting the frame identifiers of the image data during the process of transmitting the image data within multiple channels, the consistency of frame identifiers can be maintained throughout the entire process of transmitting images captured by multiple cameras via each image transmission channel, which helps users extract images captured at the same time from the images output from each image transmission channel, thereby ensuring synchronization of image data when transmitted via multiple image transmission channels.

[0038] In some alternative embodiments, as shown in FIG. 3, step 204 includes the following steps 2041 to 2042.

[0039] In step 2041, the arrangement order of the image data in the image data sequence to which the target image data belongs is determined based on the integrity information.

[0040] Specifically, the plurality of image data transmitted through each image transmission channel can be determined as one image data sequence. When no data abnormality occurs, the frame identifiers of the image data in the image data sequence have an arrangement order. When a data abnormality occurs, that is, when the integrity information indicates that the target image data is incomplete, the frame identifiers of the target image data need to be generated according to the frame identifier order.

[0041] For example, if the integrity information indicates that the target image data is complete, the frame identifier order of the image data in the current image data sequence can be determined directly as the arrangement order of the image data. If the integrity information indicates that the target image data is incomplete, the target image data can be repaired. After the repair is completed, the repair process takes a certain amount of time, which may cause the frame identifier order to be out of order. Therefore, the image data sequence needs to be re-arranged based on the timestamp of each image data in the current image data sequence.

[0042] In step 2042, a frame identifier for the target image data is determined based on the sequence.

[0043] For example, the frame identifier includes a frame number and a timestamp, and if the frame number of the previous frame of the current frame (i.e., the target image data) is 1 and the timestamp is t1, and the timestamp of the current frame is t2, then the frame number of the current frame is set to 2. If the frame number of the previous frame is 3 and the timestamp is t3, and the timestamp of the current frame is t5, and a frame with the frame number 4 and the timestamp t4 is not acquired due to some fault, then the frame number of the current frame is set to 5.

[0044] In this embodiment, the frame identifier of the target image data is generated based on the arrangement order of the image data in the image data sequence, so that the frame identifier of the target image data can be accurately generated, and the frame identifier of the target image data can accurately represent the shooting time of the target image data, which helps to improve the accuracy of synchronous transmission and output of image data transmitted over multiple image transmission channels.

[0045] In some alternative embodiments, as shown in FIG. 4, step 2041 includes the following steps 20411 to 20412.

[0046] In step 20411, the recoverable state of the target image data is determined based on the integrity information.

[0047] Specifically, since the completeness information can represent the degree of completeness of the data amount of the target image data, if the completeness information represents that the degree of completeness has reached a set threshold, the repairable state of the target image data can be determined as repairable, and if the completeness information represents that the degree of completeness has not reached a set threshold, the repairable state of the target image data can be determined as unrepairable.

[0048] In step 20412, the arrangement order of the image data in the image data sequence to which the target image data belongs is determined based on the restorable state.

[0049] Specifically, if the repairable state indicates that repair is possible but does not require repair (for example, if the completeness of the target image data is greater than 95%), the arrangement order of the frame identifiers of the image data in the image data sequence is determined as the arrangement order of the image data; if the repairable state indicates that repair is possible and requires repair (for example, if the completeness of the target image data is less than 95% and greater than 70%), the target image data is repaired, and after the repair is completed, the arrangement order of the image data sequence including the repaired target image data is determined based on the timestamp of each image data.

[0050] This embodiment realizes purposefully determining the position of the target image data in the image data sequence based on the completeness of the image, thereby helping to improve the accuracy of generating the frame identifier of the target image data.

[0051] In some alternative embodiments, step 20412 may be performed as follows.

[0052] First, if the restoration state indicates a restorable state, the target image data is restored.

[0053] There are various methods for restoring the target image data, such as restoring the target image data using a convolutional neural network, or determining the value where the abnormality occurred from the target image data using a CRC check, and then interpolating using values ​​around the value where the abnormality occurred to obtain the restored target image data.

[0054] Next, the arrangement order of the image data in the image data sequence to which the restored target image data belongs is adjusted.

[0055] Since it usually takes a certain amount of time to restore the target image data, and receiving new image data within this time period may cause the image data order in the image data sequence to which the target image data belongs to to be disrupted, after the target image data has been restored, the image data in the image data sequence can be re-aligned, for example, using a timestamp.

[0056] In this embodiment, if an abnormality occurs in the target image data and can be repaired, the target image data is repaired and the arrangement order of the image data is adjusted, thereby realizing automatic detection and repair of the image data, which is useful for improving the continuity of images transmitted through the same image transmission channel.

[0057] In some alternative embodiments, the method further comprises the steps of:

[0058] If the restoration status indicates that the target image data cannot be restored, the target image data is discarded, and the image data corresponding to the preset time transmitted through at least two image transmission channels is discarded.

[0059] Specifically, image data that matches the timestamp of the target image data's shooting time (or the difference in timestamp is smaller than a preset time difference) can be discarded from other channels of the at least two image transmission channels.

[0060] Typically, if the target image data in the target image transmission channel is unrecoverable, a discard notification is sent to a thread corresponding to another channel, and after the thread corresponding to the other channel receives the discard notification, the corresponding image data can be discarded at a preset time.

[0061] In this embodiment, if the image data in a certain image transmission channel is corrupted and cannot be repaired at a certain time, the image data in the other corresponding image transmission channels can be discarded all at once, thereby ultimately avoiding the problem of inconsistencies in the shooting times of the images output from each image transmission channel.

[0062] In some alternative embodiments, as shown in FIG. 5, step 205 includes the following steps 2051 to 2054.

[0063] In step 2051, the target image data is processed using a preset image processing model to obtain a target processed image.

[0064] Here, the image processing model may be a model that performs various types of processing on target image data. For example, the image processing model may be an ISP model, which includes multiple image processing modules, such as a white balance module, a color difference module, and a color correction module.

[0065] In step 2052, the image to be targeted for output is determined based on the image after the target processing.

[0066] Specifically, the target processed image can be directly determined as the target output image, or the frame identifier of the target processed image can be determined, and if the frame identifier does not match the arrangement order of the processed images in the image transmission channel, the processed images can be re-arranged and the image to be output at the current time can be determined based on the order.

[0067] In step 2053, images to be output that correspond to the frame identifiers of the images to be targeted for output that have been processed by the image processing model are obtained from channels other than the target image transmission channel among the at least two image transmission channels.

[0068] Specifically, since the shooting times of each image corresponding to the same frame identifier are the same or similar (for example, the difference in the timestamps of the shooting times between images is smaller than a predetermined time difference), each image corresponding to the same frame identifier is determined to be the image to be output at the current time.

[0069] In step 2054, the image of the target output object and the images of each of the acquired output objects are output.

[0070] In this step, the images currently output, which are images taken at the same time by a plurality of cameras, can reflect the situation of the target scene at the same time but in different shooting directions.

[0071] In this embodiment, the target image data and image data of other image transmission channels are processed using an image processing model, and then images corresponding to the same frame identifier are output, thereby realizing that the images of each channel output at the same time are maintained so that their shooting times match, thereby solving the problem of asynchrony in the shooting times of output images due to differences in the transmission paths of the image data of each channel and delays in processing the image data of each channel in the image processing model.

[0072] In some alternative embodiments, as shown in FIG. 6, step 2052 includes the following steps 20521 to 20522.

[0073] In step 20521, the target processed image and the history processed image are re-aligned in response to the occurrence of a disorder in the order of the frame identifiers of the target processed image and the frame identifiers of the history processed image in the target image transmission channel.

[0074] Specifically, due to differences in the transmission paths of image data channels and differences in the processing times of image processing models, the processed images generated in one image transmission channel are not generated in the order of frame identifiers. For example, one module of the image processing model temporarily stores the processed data in memory, while other modules need to wait to extract the data from memory and continue processing. This causes the frame identifiers of the processed images in the channel to become discontinuous, and at this time the processed images need to be re-ordered.

[0075] In step 20522, from the aligned target processed image and history processed images, the processed image corresponding to the frame identifier corresponding to the current output round is determined as the image to be targeted for output.

[0076] Specifically, each image transmission channel outputs images sequentially according to the output round, and the order of the rounds corresponds to the order of the frame identifiers. Therefore, in order to ensure that the frame identifiers of the images output by each image transmission channel are consistent and continuous, each image transmission channel must output images sequentially according to the order of the frame identifiers. For example, if the frame identifier of the image output in the previous round is 3, then the image with frame identifier 4 should be output in the current round. However, after processing by the image processing model, the image with frame identifier 6 is acquired first, and then the image with frame identifier 5 is acquired. That is, the order of the frame identifiers is disrupted. At this time, the image with frame identifier 5 needs to be re-arranged so that it is arranged before the image with frame identifier 6. After that, when the image with frame identifier 4 is obtained, the re-arrangement is performed. If the frame identifier corresponding to the current round is 4, then the image with frame identifier 4 is selected as the target image to be output.

[0077] In this embodiment, the processed images generated at each time are re-aligned in the target image transmission channel, thereby maintaining the arrangement of the processed images in the channel according to their capture time, which helps to avoid the order of images output from a single channel being out of order and efficiently achieves consistency of frame identifiers for images output from multiple channels.

[0078] In some alternative embodiments, as shown in FIG. 7, step 2051 includes the following steps 20511 to 20514.

[0079] In step 20511, the target image data is processed using the image processing model to obtain an image after initial processing.

[0080] Step 20512 determines whether the quality of the image after initial processing is acceptable.

[0081] Here, the method for determining whether the quality is acceptable can include multiple methods. For example, a dead point detection is performed on the processed image to determine the number of dead points, and if the number of dead points is equal to or greater than a first preset number threshold, the quality is determined to be unacceptable. For example, a CRC check is performed on the image, and if the number of abnormal lines in the check result is equal to or greater than a first preset number threshold, the quality is determined to be unacceptable.

[0082] If unsuccessful, step 20513 determines whether the image after initial processing is repairable.

[0083] For example, if the number of dead points is greater than or equal to a second preset number threshold, the image is determined to be unrecoverable.Further, for example, if the number of abnormal rows is greater than or equal to a second preset row number threshold, the image is determined to be unrecoverable.

[0084] Optionally, the initial processed image, if acceptable, can be directly established as the target processed image.

[0085] In step 20514, if restoration is possible, the initial processed image is restored to obtain the target processed image.

[0086] Here, the method for restoring the image after initial processing can include multiple methods, such as restoring the image after initial processing using a dead point correction module of the ISP image processing model, or restoring the image after initial processing using a convolutional neural network.

[0087] In this embodiment, quality judgment is performed on the images obtained by processing using the image processing model, and images of unacceptable quality are repaired. This enables automatic detection and repair of the processed images generated during the process of transmitting images over multiple channels, thereby improving the continuity of the processed images generated on the same image transmission channel.

[0088] In some alternative embodiments, the method further includes the following step after step 5013 above:

[0089] If the initially processed image is not recoverable, the initially processed image is discarded, and image data corresponding to the frame identifiers of the initially processed image in the at least two image transmission channels is discarded.

[0090] Specifically, if the image after initial processing in the target image transmission channel cannot be restored, a discard notification is usually sent to the thread corresponding to the other channel, and after the thread corresponding to the other channel receives the discard notification, the image data corresponding to the frame identifier of the image after initial processing can be discarded.

[0091] It should be noted that the image data corresponding to the frame identifier of the image after initial processing here is not limited to the image data received from the camera and the processed image obtained by processing the image processing model, but if a signal indicating that the image after initial processing is unrecoverable is triggered at either the image transmission or processing stage, any type of image data corresponding to the above frame identifier can be discarded.

[0092] In this embodiment, if the processed image generated in a certain image transmission channel at a certain time is corrupted and cannot be repaired, the image data of the corresponding other image transmission channels can be discarded all at once, thereby ultimately avoiding the problem of inconsistent shooting times of the images output from each image transmission channel.

[0093] [Example Device] 8 is a structural schematic diagram of an apparatus for synchronizing multi-channel images according to an exemplary embodiment of the present disclosure. This embodiment can be applied to an electronic device, and as shown in FIG. 8, the apparatus for synchronizing multi-channel images includes: a first determining module 801 for determining target image data corresponding to a preset time of a target image transmission channel from at least two image transmission channels; a second determining module 802 for determining a mounting status of a frame identifier of the target image data; a third determining module 803 for determining integrity information of the target image data based on the mounting status; a generating module 804 for generating a frame identifier of the target image data based on the integrity information; and an output module 805 for generating output target images corresponding to each image transmission channel of the at least two image transmission channels based on the frame identifier, and outputting the output target images.

[0094] In this embodiment, the first determining module 801 determines, from at least two image transmission channels, target image data corresponding to a preset time of a target image transmission channel.

[0095] Here, the target image transmission channel can be one of at least two image transmission channels. The preset time can be a specified fixed time or the shooting time of a camera corresponding to the target image transmission channel. The target image data can be data captured by a camera corresponding to the target image transmission channel. Typically, the target image data can be data that has not undergone image processing, for example, raw data collected by a camera sensor.

[0096] In this embodiment, the second determination module 802 determines the mounting status of the frame identifier of the target image data.

[0097] Here, the frame identifier can be a mark for distinguishing image data captured at different times, and the frame identifier is typically associated with the capture time of the image data. For example, the frame identifier can include, but is not limited to, at least one of an image frame number, a timestamp indicating the capture time of the image, and the like. Typically, a frame identifier for the image data is generated after capturing one frame of image data. This process is called frame mounting, and the mount status can include two types: successful mounting and failed mounting. Missing or incomplete image data due to software or hardware failure can cause the frame identifier to fail to mount.

[0098] In this embodiment, the third determining module 803 determines the integrity information of the target image data based on the mounting state.

[0099] Here, the integrity information represents the degree of integrity of the target image data. For example, the integrity information may be the number of rows that fail a data check (e.g., a CRC (Cyclic Redundancy Check) check) performed on each row of the image data, or the percentage of the number of rows that fail the check to the total number of rows. If the mounting state fails, it is necessary to obtain the integrity information and then determine whether the frame identifier of the target image data can be repaired.

[0100] In this embodiment, the generation module 804 generates a frame identifier for the target image data based on the integrity information.

[0101] Specifically, if the completeness information indicates that the target image data is complete (for example, the ratio of the number of successfully inspected rows to the total number of rows is greater than a predetermined ratio), a frame identifier for the target image data can be generated. For example, a frame identifier generation method can obtain the shooting time of the recorded target image data and generate a frame identifier based on the timestamp of the shooting time.

[0102] In this embodiment, the output module 805 generates an output target image corresponding to each of the at least two image transmission channels based on the frame identifier, and outputs the output target image.

[0103] Specifically, in the process of transmitting target image data on each image transmission channel, a preset image processing module (e.g., an ISP (Image Signal Processing) model) is used to process the target image data, and an output target image can be obtained after processing. Since the shooting times of multiple output target images corresponding to the same frame identifier are the same or similar, the electronic device can simultaneously output output target images with the same frame identifier generated on each image transmission channel.

[0104] Please refer to FIG. 9, which is a structural schematic diagram of an apparatus for synchronizing multi-channel images according to another exemplary embodiment of the present disclosure.

[0105] In some optional embodiments, the generation module 804 includes a first determination unit 8041 for determining the arrangement order of image data in the image data sequence to which the target image data belongs based on the integrity information, and a second determination unit 8042 for determining a frame identifier of the target image data based on the arrangement order.

[0106] In some optional embodiments, the first determination unit 8041 includes a first determination subunit 80411 for determining a repairable state of the target image data based on the integrity information, and a second determination subunit 80412 for determining the arrangement order of image data in the image data sequence to which the target image data belongs based on the repairable state.

[0107] In some optional embodiments, the second determination subunit 80412 is further used to repair the target image data when the repair state represents a repairable state and adjust the arrangement order of image data in the image data sequence to which the repaired target image data belongs.

[0108] In some optional embodiments, the second determination subunit 80412 is further used to discard the target image data when the repair status indicates that the target image data cannot be repaired, and to discard image data corresponding to a predetermined time transmitted on at least two image transmission channels.

[0109] In some alternative embodiments, the output module 805 includes a processing unit 8051 for processing the target image data using a preset image processing model to obtain a target processed image, a third determination unit 8052 for determining a target output target image based on the target processed image, an acquisition unit 8053 for acquiring output target images corresponding to frame identifiers of the target output target images processed by the image processing model from channels other than the target image transmission channel among the at least two image transmission channels, and an output unit 8054 for outputting the target output target image and each acquired output target image.

[0110] In some optional embodiments, the third determination unit 8052 includes an alignment subunit 80521 for re-aligning the target processed image and the history processed image in response to a disruption occurring in the frame identifier of the target processed image and the frame identifier of the history processed image in the target image transmission channel, and a third determination subunit 80522 for determining, from the aligned target processed image and history processed image, the processed image corresponding to the frame identifier corresponding to the current output round as the image to be targeted for output.

[0111] In some alternative embodiments, the processing unit 8051 includes a processing subunit 80511 for processing the target image data using an image processing model to obtain an initial processed image, a fourth determination subunit 80512 for determining whether the quality of the initial processed image is acceptable, and if unacceptable, a fifth determination subunit 80513 for determining whether the initial processed image is repairable, and a repair subunit 80514 for repairing the initial processed image that is repairable to obtain a target processed image.

[0112] In some optional embodiments, the processing unit 8051 further includes a discarding subunit 80515 for discarding the initially processed image if the initially processed image is unrepairable, and discarding image data corresponding to the frame identifiers of the initially processed image in the at least two image transmission channels.

[0113] The multi-channel image synchronization device according to the above embodiment of the present disclosure detects frame numbers and detects data integrity for image data transmitted through at least two image transmission channels, generates frame identifiers for the image data, processes the image data for each image transmission channel to generate images to be output, and outputs images corresponding to the frame identifiers via each image transmission channel, thereby achieving automatic detection and restoration for image data transmitted through multiple image transmission channels and improving the efficiency and quality of multi-channel image collection. Moreover, by detecting and adjusting the frame identifiers of the image data during the process of transmitting the image data within multiple channels, the consistency of frame identifiers can be maintained throughout the entire process of transmitting images captured by multiple cameras via each image transmission channel, which helps users extract images captured at the same time from the images output from each image transmission channel, thereby ensuring synchronization of image data when transmitted through multiple image transmission channels.

[0114] [Example Electronic Devices] An electronic device according to an embodiment of the present disclosure will be described below with reference to Fig. 10. The electronic device can be the electronic device 101 shown in Fig. 1.

[0115] FIG. 10 illustrates a block diagram of an electronic device according to an embodiment of the present disclosure.

[0116] As shown in FIG. 10, the electronic device includes one or more processors 1001 and a memory 1002.

[0117] The processor 1001 may be a central processing unit (CPU) or other type of processing unit having data processing and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0118] The memory 1002 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored in the computer-readable storage medium, and the processor 1001 may execute the program instructions to implement the method for synchronizing multi-channel images of each embodiment of the present disclosure and / or other desired functions. Various types of content, such as image data, may also be stored in the computer-readable storage medium.

[0119] In one example, the electronic device may further include input devices 1003 and output devices 1004 connected to each other via a bus system and / or other type of connection mechanism (not shown).

[0120] For example, the input device 1003 may be a device such as a camera, a mouse, or a keyboard for inputting image data, various commands, etc. The input device 1003 may also be a communication network connector for receiving image data, various commands, etc. input from other devices connected to the electronic device.

[0121] The output device 1004 can output various information, including images output from each image transmission channel, to the outside, and can include, for example, a display, a speaker, a printer, a communication network, and a remote output device connected thereto.

[0122] For simplicity, Fig. 10 shows only some of the components in the electronic device that are relevant to the present disclosure, and omits components such as buses, input / output interfaces, etc. In addition, the electronic device may further include any other appropriate components depending on the specific application.

[0123] Exemplary Computer Program Products and Computer-Readable Storage Media In addition to the above methods and apparatuses, embodiments of the present disclosure may provide a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform steps in the method for synchronizing multi-channel images according to embodiments of the present disclosure described in the "Exemplary Method" section above.

[0124] The computer program product may have program code for carrying out operations of embodiments of the present disclosure written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and traditional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on the remote computing device or a server.

[0125] Moreover, the embodiments of the present disclosure may further provide a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps of the method for synchronizing multi-channel images according to the embodiments of the present disclosure described in the "Exemplary Method" section above.

[0126] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0127] Although the basic principles of the present disclosure have been described above with reference to specific embodiments, the benefits, advantages, effects, etc. mentioned in the present disclosure are merely illustrative and not limiting, and these benefits, advantages, effects, etc. are not necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details of the above disclosure are merely illustrative and easy-to-understand functions and are not limiting, and the above details do not necessarily limit the present disclosure to those realized by the above specific details.

[0128] Those skilled in the art can make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. determining target image data corresponding to a preset time of a target image transmission channel from at least two image transmission channels; determining a mounting state of a frame identifier of the target image data; determining integrity information of the target image data based on the mounting state; generating a frame identifier for the target image data based on the integrity information; generating an output target image corresponding to each of the at least two image transmission channels based on the frame identifier, and outputting the output target image; 10. A method for synchronizing multi-channel images, comprising:

2. generating a frame identifier for the target image data based on the integrity information, determining an arrangement order of image data in an image data sequence to which the target image data belongs based on the integrity information; determining a frame identifier for the target image data based on the arrangement order; The method of synchronizing multi-channel images according to claim 1 .

3. The step of determining an arrangement order of image data in an image data sequence to which the target image data belongs based on the integrity information includes: determining a recoverable state of the target image data based on the integrity information; determining an arrangement order of image data in an image data sequence to which the target image data belongs based on the restorable state; The method of synchronizing multi-channel images according to claim 2 .

4. The step of determining an arrangement order of image data in an image data sequence to which the target image data belongs based on the restorable state includes: if the restoration state indicates a restorable state, restoring the target image data; adjusting the arrangement order of image data in the image data sequence to which the restored target image data belongs; A method for synchronizing multi-channel images according to claim 3.

5. and further comprising a step of discarding the target image data when the restoration status indicates that the target image data cannot be restored, and discarding image data corresponding to the preset time transmitted through the at least two image transmission channels. Method for synchronizing multi-channel images according to claim 3 or 4.

6. The step of generating an output target image corresponding to each of the at least two image transmission channels based on the frame identifier and outputting the output target image includes: processing the target image data using a preset image processing model to obtain a target processed image; determining a target image to be output based on the target processed image; acquiring, from a channel other than the target image transmission channel among the at least two image transmission channels, images to be output that correspond to frame identifiers of the target output images processed by the image processing model; outputting the image of the target output object and the images of each output object acquired; The method of synchronizing multi-channel images according to claim 1 .

7. The step of determining an image to be a target output based on the target processed image includes: re-arranging the target processed image and the history processed image according to the occurrence of a disorder in the order of the frame identifiers of the target processed image and the frame identifiers of the history processed images in the target image transmission channel; determining, from the aligned target processed image and history processed image, a processed image corresponding to a frame identifier corresponding to a current output round as the target output image; A method for synchronizing multi-channel images according to claim 6.

8. The step of processing the target image data using a preset image processing model to obtain a target processed image includes: processing the target image data using the image processing model to obtain an initial processed image; determining whether the quality of the image after the initial processing is acceptable; If unsuccessful, determining whether the initially processed image is salvageable; and if so, restoring the initial processed image to obtain the target processed image. Method for synchronizing multi-channel images according to claim 6 or 7.

9. The method for synchronizing multi-channel images includes, after the step of determining whether the initially processed image is restorable, and further comprising the step of discarding the initially processed image if the initially processed image is unrestorable, and discarding image data corresponding to a frame identifier of the initially processed image in the at least two image transmission channels. A method for synchronizing multi-channel images according to claim 8.

10. a first determining module for determining target image data corresponding to a preset time of a target image transmission channel from at least two image transmission channels; a second determination module for determining a mounting state of a frame identifier of the target image data; a third determination module for determining integrity information of the target image data based on the mounting state; a generation module for generating a frame identifier for the target image data based on the integrity information; an output module for generating an output target image corresponding to each of the at least two image transmission channels based on the frame identifier, and outputting the output target image; 1. An apparatus for synchronizing multi-channel images, comprising:

11. Storing a computer program that is executed by a processor to implement the method for synchronizing multi-channel images according to claim 1; A computer-readable storage medium comprising:

12. a processor; a memory for storing instructions executable by the processor; The executable instructions are instructions for implementing the method for synchronizing multi-channel images of claim 1 , the processor reading and executing the executable instructions from the memory to implement the method for synchronizing the multi-channel images. An electronic device characterized by:

Images