Image processing method and device and electronic equipment

Abstract

The invention discloses an image processing method and device and electronic equipment, the image processing method and device are executed by the electronic equipment, the electronic equipment comprises a camera module, an application processor and an image processing chip, and the method comprises the steps that the application processor receives N first image frames output by the camera module; the image processing chip receives L second image frames output by the camera module; screening the L second image frames to obtain a third image frame; performing image processing based on the third image frame to obtain a first processed image; sending the first information to an application processor; the first information comprises identification information of the first processing image and the third image frame, and the N first image frames and the L second image frames are image frames output by two branches of the camera module in the same time period respectively; the application processor screens out a fourth image frame which is the same as the third image frame from the N first image frames based on the identification information; and performing image post-processing on the first processing image based on the fourth image frame to obtain a second processing image.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an image processing method, device and electronic equipment. Background Art

[0002] With the rapid development of electronic technology, electronic devices have been widely used, which can support the imaging function of electronic devices through camera modules and application processors (AP) to output images.

[0003] At present, in image processing application scenarios such as shooting business scenarios, electronic devices usually use AP to obtain one or several image frames output by the camera module, and perform image processing on the AP side based on the image frames to output images. However, since this image processing method directly performs image processing on the AP side, and the AP side hardware is of a general type and is usually not specifically optimized for a certain application scenario such as shooting business scenarios, this image processing method will have the problem of poor output image quality. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an image processing method, device and electronic device to solve the technical problem of relatively poor output image quality existing in the image processing method of directly performing image processing on the AP side, and to improve the output image quality of the electronic device.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, which is performed by an electronic device, wherein the electronic device includes a camera module, an application processor, and an image processing chip, and the method includes:

[0006] The application processor receives N first image frames output by the camera module;

[0007] The image processing chip receives L second image frames output by the camera module; selects a third image frame from the L second image frames; performs image processing based on the third image frame to obtain a first processed image; and sends first information to the application processor; wherein the first information includes identification information of the first processed image and the third image frame, the N first image frames and the L second image frames are image frames output by two branches of the camera module in the same time period, respectively, and N and L are both positive integers;

[0008] The application processor selects a fourth image frame that is identical to the third image frame from the N first image frames based on the identification information; and performs image post-processing on the first processed image based on the fourth image frame to obtain a second processed image.

[0009] In a second aspect, an embodiment of the present application provides an image processing device, which is applied to an electronic device, wherein the electronic device includes a camera module, an application processor, and an image processing chip, and the device includes:

[0010] A first receiving module, used for the application processor to receive N first image frames output by the camera module;

[0011] A second receiving module, used for the image processing chip to receive L second image frames output by the camera module; the N first image frames and the L second image frames are image frames output by two branches of the camera module in the same time period, and N and L are both positive integers;

[0012] A first screening module, used for the image processing chip to screen the L second image frames to obtain a third image frame;

[0013] An image processing module, used for the image processing chip to perform image processing based on the third image frame to obtain a first processed image;

[0014] A first sending module, configured for the image processing chip to send first information to the application processor; wherein the first information includes identification information of the first processed image and the third image frame;

[0015] a second screening module, configured for the application processor to screen out a fourth image frame identical to the third image frame from the N first image frames based on the identification information;

[0016] An image post-processing module is used for the application processor to perform image post-processing on the first processed image based on the fourth image frame to obtain a second processed image.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the image processing method described in the first aspect are implemented.

[0018] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the image processing method described in the first aspect are implemented.

[0019] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps of the image processing method as described in the first aspect.

[0020] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps of the image processing method as described in the first aspect.

[0021] In an embodiment of the present application, an application processor receives N first image frames output by a branch of a camera module; and an image processing chip receives L second image frames output by another branch of the camera module in the same time period; and a third image frame is obtained by screening out the L second image frames; image processing is performed based on the third image frame to obtain a first processed image; identification information of the first processed image and the third image frame is sent to the application processor; then the application processor screens out a fourth image frame identical to the third image frame from the N first image frames based on the identification information of the third image frame; and based on the fourth image frame, the first processed image is post-processed to obtain a second processed image. Since this embodiment performs image processing on the image frames output by the camera module through the image processing chip, and uses the AP to perform image post-processing on the first processed image based on the same image frame as the image processing chip, by performing image processing twice based on the same image frame, the image processing effect can be improved, thereby improving the output image quality of the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flowchart of an image processing method provided by some embodiments of the present application;

[0023] Figure 2 It is a schematic diagram of the process of receiving image frames by the AP and the ASIC chip provided in some embodiments of the present application;

[0024] Figure 3 is a schematic diagram of an interactive process of performing image processing by an electronic device provided in some embodiments of the present application;

[0025] Figure 4 is a schematic diagram of the structure of an image processing device provided by some embodiments of the present application;

[0026] Figure 5 is a schematic diagram of the structure of an electronic device provided by some embodiments of the present application;

[0027] Figure 6 A schematic diagram of the hardware structure of an electronic device provided for some embodiments of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0029] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0030] The image processing method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0031] It should be noted that the image processing method of the embodiment of the present application is executed by an electronic device, and the electronic device includes a camera module, an application processor and an image processing chip.

[0032] The camera module may include an image sensor (Sensor), which may be a complementary metal oxide semiconductor (CMOS) sensor. The camera module may perform image exposure through the CMOS sensor to generate an image frame. Afterwards, in some embodiments, the camera module may output the image frame to the AP and the image processing chip respectively.

[0033] Among them, image processing chip usually refers to an external chip or module used for graphics processing in mobile devices or computers. The image processing chip is usually used to provide additional graphics processing capabilities to improve the graphics performance of the device and handle graphics-related tasks. In mobile devices, image processing chips can be used to accelerate tasks such as games, video playback, and graphics rendering to provide a smoother user experience. The image processing chip usually works in conjunction with the central processing unit (CPU) to share the load of graphics processing, thereby improving the performance and efficiency of the overall system.

[0034] In some embodiments, the image processing chip may be an application-specific integrated circuit (ASIC) chip. In some embodiments, an external graphics processing unit (GPU) in a computer may also be considered as an image processing chip for processing complex graphics tasks, such as three-dimensional (3D) modeling, video editing, etc. In some high-performance computers or gaming devices, users may also choose to install additional graphics processors to improve graphics performance.

[0035] Among them, the application processor (AP) is a very large-scale integrated circuit that expands audio and video functions and dedicated interfaces based on the central processing unit. It plays the role of computing and calling other functional components in electronic equipment.

[0036] The AP may include an application processor system chip (AP SoC), which plays a role in computing and calling other functional components in the image processing of the electronic device. For example, the AP SoC may control the camera module to enable the camera module to output image frames, and the AP SoC may also control the image processing chip to enable the image processing chip to perform image processing based on the image frames output by the camera module to obtain a first processed image.

[0037] In some embodiments, the AP may further include an image post-processing module, which may perform image post-processing on the first processed image sent by the image processing chip based on the image frame output by the camera module to output the image of the electronic device, so that the electronic device can achieve image support capabilities through the camera module, AP and image processing chip. In some embodiments, such as in a shooting business scenario, the electronic device can achieve an image shooting function through the camera module, AP and image processing chip.

[0038] In some embodiments, the electronic device may include an AP, a camera module and an ASIC chip. The camera module may output image frames to the AP and the ASIC chip respectively through diversion, so that the AP and the ASIC chip can perform image processing based on the image frames respectively, that is, through two image processings, the purpose of improving the output image quality is achieved.

[0039] After receiving the image frame output by the camera module, the ASIC chip can perform image processing based on the image frame and send the obtained first processed image to the AP. Correspondingly, the AP can perform image post-processing on the first processed image based on the image frame output by the camera module to obtain a second processed image and output it accordingly to achieve image support capability.

[0040] Since the image processing is performed directly on the AP side for the image frames output by the camera module, this image processing method has a high demand on the computing power of the AP, and the AP plays the role of computing and calling other functional components in the electronic device. Therefore, in the related art, the image processing is performed directly on the AP side for the image frames output by the camera module, and there will also be problems with image processing efficiency. In this embodiment, the image processing is first performed based on the image frames output by the camera module by the ASIC chip, and the obtained first processed image is sent to the AP, and then the first processed image is post-processed by the AP based on the image frames output by the camera module. In this way, not only can the computing power of the AP for image processing be shared by the ASIC chip to improve the image processing efficiency, but also the output image quality of the electronic device can be improved by two image processing.

[0041] It should be noted that during the image processing process of the electronic device, in order to ensure the image quality output by the electronic device, the image frame processed by the image processing chip usually needs to be the same as the image frame used by the AP for image post-processing. This is explained in detail below.

[0042] Figure 1 is a flow chart of the image processing method provided in the embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0043] Step 101: an application processor receives N first image frames output by a camera module.

[0044] In some embodiments, the first image frame may be an image frame in a Mobile Industry Processor Interface (MIPI) data format, wherein the application processor may communicate with the camera module via the MIPI protocol to enable the application processor to receive the image frame output by the camera module, and the MIPI protocol is a set of standards developed by the MIPI Alliance for inter-chip communication in mobile devices.

[0045] In some embodiments, when the CMOS Sensor of the camera module is in operation, the CMOS Sensor continuously generates image frames and can send the image frames to the application processor frame by frame. The application processor can receive the image frames of the camera module through the MIPI protocol.

[0046] In some embodiments, the frame numbers of the N first image frames received by the application processor may be consecutive. For example, the N first image frames may be the 100th to 105th image frames of the camera module.

[0047] In some embodiments, when the application processor receives image frames from the camera module, the application processor may choose to discard some image frames due to high load or abnormal conditions of the application processor, that is, not receive the image frames. At this time, the N first image frames received by the application processor are discontinuous. For example, the camera module actually outputs the 100th to 105th image frames, and the N first image frames received by the application processor may be the 101st image frame, the 103rd image frame, and the 105th image frame.

[0048] In some embodiments, the N first image frames may be received when the camera module is started. When the camera module is started, it may send image frames to the application processor frame by frame. Accordingly, the application processor may receive one or more image frames output by the camera module for image processing. In some embodiments, the N first image frames may be received when the electronic device receives a shooting input. The shooting input may be a user input to a shooting control for controlling the electronic device to capture an image. That is, when the electronic device receives the shooting input, it may notify the AP through a signal. Accordingly, the AP may receive the image frames output by the camera module when the electronic device receives the shooting input, and obtain N first image frames.

[0049] It should be noted that the executor of this step may be the AP SoC in the AP, and the AP SoC may communicate with the camera module via the MIPI protocol to receive the N first image frames output by the camera module.

[0050] Step 102, the image processing chip receives L second image frames output by the camera module; filters out a third image frame from the L second image frames; performs image processing based on the third image frame to obtain a first processed image; and sends first information to an application processor; wherein the first information includes identification information of the first processed image and the third image frame, the N first image frames and the L second image frames are image frames output by two branches of the camera module in the same time period, respectively, and N and L are both positive integers.

[0051] In some embodiments, the second image frame may be an image frame in MIPI data format, wherein the image processing chip may communicate with the camera module via the MIPI protocol to enable the image processing chip to receive the image frame output by the camera module.

[0052] In some embodiments, when the CMOS Sensor of the camera module is in operation, the CMOS Sensor continuously generates image frames and can send the image frames to the image processing chip frame by frame. The image processing chip can receive the image frames of the camera module through the MIPI protocol.

[0053] In some embodiments, the frame numbers of the L second image frames received by the image processing chip may be consecutive. For example, the L second image frames may be the 99th to 106th image frames of the camera module.

[0054] In some embodiments, when the image processing chip receives image frames from the camera module, the image processing chip may choose to discard some image frames, that is, not receive the image frames, due to the high load or abnormality of the image processing chip. At this time, the L second image frames received by the image processing chip are discontinuous. For example, the camera module actually outputs the 99th to 106th image frames, and the N first image frames received by the image processing chip may be the 99th image frame, the 101st image frame, and the 103rd to 106th image frames.

[0055] In some embodiments, the L second image frames may be received when the camera module is started. When the camera module is started, it may send image frames to the image processing chip one by one. Accordingly, the image processing chip may receive one or more image frames output by the camera module for image processing. In some embodiments, the L second image frames may be received when the electronic device receives a shooting input. When the electronic device receives the shooting input, it may notify the image processing chip through a signal. Accordingly, the image processing chip may receive the image frames output by the camera module when the electronic device receives the shooting input, and obtain L second image frames.

[0056] In this embodiment, the output of the camera module can be split into two branches to output the image frames of the camera module to the application processor and the image processing chip respectively. This method of using the camera module as the source and dividing the output into two paths is called the same-source output method, and the image frames received by the application processor and the image processing chip can be the image frames output by the camera module through the same-source output method, that is, the N first image frames and the L second image frames are the image frames output by the two branches of the camera module in the same time period. Correspondingly, the image processing chip and the AP can successively perform image processing based on the image frames output by the same source of the camera module, that is, through two image processing, the purpose of improving the output image quality is achieved.

[0057] Ideally, the image frames received by the AP and the image processing chip should be the same, that is, ideally, the N first image frames and the L second image frames should be the same, and the N first image frames or the L second image frames are the image frames output by the camera module. However, the AP and the image processing chip may choose to discard some image frames due to their own conditions. For example, when they are highly loaded or local abnormalities are detected, they do not receive image frames. This may cause the image frames received by the AP and the image processing chip, as well as the number of image frames, to be different, that is, the N first image frames and the L second image frames are not completely the same, or N and L may be different.

[0058] For example, when an electronic device receives a shooting input, the image frames output by the camera module are the 99th to 106th image frames. When receiving the image frames, the AP chooses to discard some image frames, resulting in the N first image frames received being the 100th to 105th image frames, while the L second image frames received by the image processing chip are the 99th to 106th image frames.

[0059] When the image processing chip receives L second image frames, the image processing chip can randomly or under the control of the AP select a third image frame from the L second image frames, and the third image frame is used for image processing. The number of the third image frames is at least one.

[0060] In some embodiments, the AP may send control information to the image processing chip to inform the screening strategy. For example, the screening strategy may include the number of image frames to be screened and a timestamp list, etc. The timestamp of the third image frame screened by the image processing chip needs to be in the timestamp list.

[0061] In this step, the third image frame is obtained by screening from the L second image frames for image processing, which can reduce the amount of data for image processing and improve image processing efficiency.

[0062] When the third image frame is obtained by screening, the image processing chip can perform image processing based on the third image frame to obtain a first processed image. The image processing performed by the image processing chip may include image transformation, image enhancement, image denoising, etc., and the image processing is used to improve the graphics performance of the electronic device.

[0063] Afterwards, the image processing chip may report the identification information of the first processed image and the third image frame to the AP. The identification information of the third image frame may be a timestamp or a frame number of the third image frame, or the identification information of the third image frame may be the image itself of the third image frame. Accordingly, the AP SoC may receive the first information sent by the image processing chip, so that the AP SoC may perform image frame matching based on the first information, which is described in detail below.

[0064] Step 103 , the application processor selects a fourth image frame identical to the third image frame from the N first image frames based on the identification information; and performs image post-processing on the first processed image based on the fourth image frame to obtain a second processed image.

[0065] Among them, the third image frame and the fourth image frame are: the same image frames output by the camera module through the same source output method.

[0066] In some embodiments, the AP SoC in the AP can match the third image frame with the first image frame based on the identification information, with the aim of screening out a fourth image frame that is identical to the third image frame from the N first image frames, and the image frame processed by the image processing chip is the same as the image frame used by the AP for image post-processing, thereby improving the image quality output by the electronic device.

[0067] In some embodiments, the identification information may be the second timestamp information of the third image frame, and the second timestamp information may include the timestamps of each third image frame. In this embodiment, since the image frames received by the AP and the image processing chip are output by the same source of the camera module, if the timestamp of the image frame received by the AP is the same as or close to the timestamp of the image frame received by the image processing chip when the AP and the image processing chip are time synchronized, the two image frames are considered to be the same image frame. By matching the timestamp in the first timestamp information of N first image frames with the timestamp in the second timestamp information of the third image frame, a timestamp that is the same as or close to the timestamp of the third image frame can be matched from the first timestamp information, and the image frame corresponding to the timestamp is the fourth image frame that is the same as the third image frame.

[0068] In this embodiment, when the AP and the image processing chip respectively receive image frames output by the camera module, they can record the timestamp of the received image frames. The recorded timestamp of the image frame can be the timestamp of the starting moment of the image frame or the timestamp of the ending moment. Accordingly, the timestamp of each first image frame and the timestamp of each second image frame can be obtained.

[0069] In some embodiments, the identification information may be the frame number of the third image frame, and the application processor may match the third image frame with the first image frame by matching the frame numbers. For example, the image frame having the same frame number as the third image frame among the N first image frames is determined to be the same as the third image frame, thereby obtaining a fourth image frame.

[0070] In some embodiments, the identification information may also be the image of the third image frame itself, and the application processor may match the third image frame with the first image frame by image feature matching. That is, the application processor may obtain the image features of the third image frame, and obtain the image features of each image frame in the N first image frames, and determine the image frame that is the same as the third image frame from the image features of the N first image frames that are the same as the image features of the third image frame, and obtain the fourth image frame.

[0071] When the AP SoC screens and obtains the fourth image frame, the image post-processing module in the AP can perform image post-processing on the first processed image based on the fourth image frame to obtain a second processed image. The image post-processing can include secondary composition, noise reduction, sharpening and other processing.

[0072] In some embodiments, the obtained second processed image can be output as a captured image, which can improve the quality of the captured image output in the shooting business scenario of the electronic device.

[0073] In this embodiment, the N first image frames output by one branch of the camera module are received by the application processor; and the L second image frames output by another branch of the camera module in the same time period are received by the image processing chip; and the third image frame is screened out from the L second image frames; image processing is performed based on the third image frame to obtain a first processed image; identification information of the first processed image and the third image frame is sent to the application processor; then the application processor screens out a fourth image frame identical to the third image frame from the N first image frames based on the identification information of the third image frame; and based on the fourth image frame, the first processed image is post-processed to obtain a second processed image. Since this embodiment performs image processing on the image frames output by the camera module by the image processing chip, and uses the AP to post-process the first processed image based on the same image frame as the image processing chip, the image processing effect can be improved by performing image processing twice based on the same image frame, thereby improving the output image quality of the electronic device.

[0074] Optionally, the third image frame may be matched with the first image frame by a timestamp matching method, and step 103 specifically includes:

[0075] The application processor selects, based on the first timestamp information of the N first image frames and the second timestamp information of the third image frame, an image frame whose timestamp matches the timestamp in the second timestamp information from the N first image frames to obtain a fourth image frame;

[0076] The identification information includes second timestamp information, the first timestamp information is the timestamp when the application processor receives the N first image frames, and the second timestamp information is the timestamp when the image processing chip receives the third image frame.

[0077] In this embodiment, the first timestamp information may include the timestamp of each first image frame, which may be represented in the form of a timestamp list, and the second timestamp information may also include the timestamp of each third image frame, which may also be represented in the form of a timestamp list.

[0078] In some embodiments, the image frame whose timestamp matches the timestamp in the second timestamp information may refer to an image frame having the same timestamp as the third image frame, or an image frame corresponding to the second target timestamp, and the interval between the second target timestamp and the timestamp of the third image frame is very small.

[0079] In some embodiments, for example, when the image processing chip and the AP are not time synchronized, the timestamps of the image frames recorded by the image processing chip and the AP may be different. In this case, the AP may match the image frames based on the target difference recorded at the same time by the clock modules of the image processing chip and the AP.

[0080] For example, the fourth image frame whose timestamp matches the timestamp of the third image frame may be an image frame whose timestamp difference with the timestamp of the third image frame among the N first image frames is within a preset range, and the preset range is determined based on the target difference, and the preset range may be a range determined by a preset threshold of the target difference, such as a target value of 5 seconds, and a preset range may be 4.95 seconds to 5.05 seconds. For another example, the AP may also correct the timestamp of the third image frame based on the time difference to achieve the purpose of time synchronization between the image processing chip and the AP, and then perform timestamp matching.

[0081] In some embodiments, the AP can compare each timestamp in the second timestamp information with the timestamp in the first timestamp information one by one to determine the difference between the two, and determine the image frame corresponding to the timestamp among the N first image frames whose absolute value of the difference between the two is less than or equal to a preset threshold as the fourth image frame.

[0082] In some embodiments, the AP may also compare each timestamp in the first timestamp information with the timestamp in the second timestamp information one by one to determine the difference between the two, and determine the image frame corresponding to the timestamp among the N first image frames whose absolute value of the difference between the two is less than or equal to a preset threshold as the fourth image frame.

[0083] In some embodiments, double traversal can also be used to traverse image frames. The first layer of traversal is N first image frames, and the timestamp of each first image frame is compared with the timestamp of a third image frame reported by the image processing chip. The second layer of traversal is to compare the timestamp of each third image frame reported by the image processing chip with the timestamp of the first image frame. In this way, through double traversal, the accuracy of image frame matching can be improved.

[0084] In some embodiments, if the difference between the two timestamp comparisons is very small, such as less than or equal to a preset threshold, that is, if ABS(AP_img_ts_i, ASIC_ts_j)<=K, the match is successful. The AP may determine the first image frame as the fourth image frame, and the timestamps of the fourth image frame and the successfully matched third image frame may not participate in subsequent matching. If the match is not successful, the next timestamp is traversed for comparison. Among them, K is a preset threshold, such as 3 milliseconds (ms), AP_img_ts_i is the i-th timestamp in the first timestamp information, i is 0, 1, ..., N, ASIC_ts_j is the j-th timestamp in the second timestamp information, j is 0, 1, ..., P, P is the number of third image frames, and ABS(x, y) represents the absolute value of the difference between x and y.

[0085] In some embodiments, the image processing chip is time synchronized with the application processor, and if the timestamp matching result is that the timestamps of the two are the same or close, then the two image frames can be considered to be the same. Optionally, the application processor selects an image frame whose timestamp matches the timestamp in the second timestamp information from the N first image frames based on the first timestamp information of the N first image frames and the second timestamp information of the third image frame, to obtain a fourth image frame, including:

[0086] The application processor performs difference processing on a first target timestamp in the second timestamp information and each timestamp in the first timestamp information, where the first target timestamp is any timestamp in the second timestamp information;

[0087] The application processor determines the image frame corresponding to the second target timestamp among the N first image frames as the image frame whose timestamp matches the first target timestamp, to obtain a fourth image frame;

[0088] The second target timestamp is: a timestamp in the first timestamp information whose absolute value of the difference with the first target timestamp is less than or equal to a preset threshold.

[0089] Among them, the preset threshold can be set according to the actual situation. Under normal circumstances, the preset threshold is a value close to 0. In this way, when the absolute value of the difference between the two is less than or equal to the preset threshold, it can be characterized that the timestamps of the two are very close, and it can be determined that the corresponding image frames of the two are from the same image frames output by the camera module, which can improve the accuracy of timestamp matching.

[0090] Moreover, the same method of timestamp matching can be performed for any third image frame, so that a fourth image frame that matches the timestamp of each third image frame one by one can be matched from the N first image frames, thereby improving the accuracy of image frame matching and further improving the output image quality of the electronic device.

[0091] In some embodiments, image frame matching can be performed by matching the timestamps of the image frames when the image processing chip is synchronized with the application processor. Before image frame matching, it is necessary to ensure that the clock modules of the image processing chip and the application processor are synchronized, and it is necessary to record the timestamps of the received image frames when the image processing chip and the application processor are synchronized.

[0092] Optionally, before step 101, the method further includes:

[0093] The application processor controls the image processing chip to perform time synchronization with the application processor;

[0094] When the image processing chip is synchronized with the application processor, the application processor obtains first timestamp information of the N received first image frames, and the image processing chip obtains third timestamp information of the L received second image frames, where the third timestamp information includes the second timestamp information.

[0095] In some embodiments, the AP SoC can obtain the time difference between the clock module of the image processing chip and the clock module of the AP, and can send a time synchronization request to the image processing chip. The time synchronization request can include the time difference. Accordingly, the image processing chip can correct the time of the clock module of the image processing chip based on the time difference to synchronize it with the time of the clock module of the AP.

[0096] In some embodiments, the AP SoC may also control the image processing chip to synchronize with the AP SoC by sending a pulse signal and a time synchronization request. Optionally, the application processor controls the image processing chip to synchronize with the application processor, including:

[0097] The application processor sends a pulse signal and a time synchronization request to the image processing chip, where the time synchronization request includes the first time when the application processor sends the pulse signal;

[0098] When receiving the pulse signal, the image processing chip obtains the reception time of the pulse signal to obtain the second time; and performs time synchronization setting with the application processor based on the time difference between the first time and the second time.

[0099] The pulse signal may be a signal composed of a series of pulses, and the width and interval time of each pulse may be different. The time synchronization request may be used to control the image processing chip to perform time synchronization setting.

[0100] The pulse signal may be sent before, at the same time, or after the time synchronization request is sent. For example, the AP SoC may send a pulse signal to the image processing chip through the input output (IO) interface, and record the time of sending the pulse signal, i.e., the first time, through its own clock module, and record the time of receiving the pulse signal, i.e., the second time, as t2, through its own clock module when the image processing chip receives the pulse signal.

[0101] Afterwards, the AP SoC may send a time synchronization request to the image processing chip through the protocol bus. The time synchronization request may include the first time t1 at which the AP SoC sends the pulse signal.

[0102] After receiving the time synchronization request, the image processing chip can calculate the time difference diff=t1-t2, that is, the time difference between the clock module of the image processing chip and the clock module of the AP, and then obtain the current time t3 of the clock module of the image processing chip, and calculate t3'=t3+diff, and set the time t3' to the clock module of the image processing chip to make it the current time of the image processing chip, so that the time of the image processing chip and the AP are synchronized.

[0103] Correspondingly, the image processing chip can record the timestamp of the received second image frame in synchronization with the time of the AP, and send the second timestamp information of the third image frame selected from the L second image frames to the AP.

[0104] In this embodiment, the image processing chip and the AP SoC are controlled to perform time synchronization by sending a pulse signal and a time synchronization request. Since the pulse signal can be sent before, at the same time, or after the time synchronization request is sent, the control process of the AP time synchronization can be more flexible. By recording the reception time of the pulse signal through the respective clock modules of the image processing chip and the AP, the time synchronization between the image processing chip and the AP can be made more accurate.

[0105] Correspondingly, when the image processing chip and the AP are synchronized, the timestamp of the received image frame can be recorded according to their respective clock modules. The timestamp of the recorded image frame can be the timestamp of the start time of the image frame or the timestamp of the end time. The following takes the timestamp of the start time of the image frame as an example to illustrate the reception process of the image frame outputted by the same source of the camera module by the AP and the image processing chip.

[0106] In some embodiments, Figure 2 FIG. 1 is a schematic diagram of a process flow of receiving an image frame by an AP and an ASIC chip provided in some embodiments of the present application, such as Figure 2 As shown, both the AP and the ASIC chip can simultaneously receive the image frames output by the camera module and record the timestamp of the start time of receiving the image frames. The process is as follows:

[0107] Step 200: When the electronic device starts the camera module, the camera module can output image frames to the AP and the image processing chip frame by frame;

[0108] Step 201a: The AP receives a first image frame from a camera module;

[0109] Step 202a: The AP records the timestamp of the start time of receiving the first image frame each time it receives a frame. In this way, the AP receives N first image frames in total and obtains first timestamp information.

[0110] Step 201b: while the AP receives the image frames output by the camera module, the ASIC chip also receives the image frames output by the camera module, that is, the data source of the image streams received by the AP and the ASIC chip is the same, that is, both are the camera module;

[0111] Step 202b: The ASIC chip records the timestamp of the start time of receiving the second image frame each time it receives a frame. In this way, the ASIC chip receives L second image frames in total and records the timestamp of each of the L second image frames to obtain third timestamp information.

[0112] In this embodiment, the AP SoC can control the image processing chip to synchronize time with the AP. In the case of time synchronization, when the AP receives the first image frame, the clock module that records the timestamp when the image processing chip receives the second image frame is synchronized. In this way, the image processing chip can record the timestamp of the received second image frame based on the clock module synchronized with the AP time, so that the timestamps of the first image frame and the second image frame output from the same source are close, which can simplify the AP's matching process for timestamps, so that the AP can easily select the fourth image frame whose timestamp matches the timestamp of the third image frame from the N first image frames, and can improve the accuracy of image matching and the output image quality of the electronic device.

[0113] In some embodiments, the AP may control the image processing chip to filter out the third image frame from the L second image frames. Optionally, before filtering out the third image frame from the L second image frames, the method further includes:

[0114] The application processor sends first timestamp information to the image processing chip;

[0115] The image processing chip selects the L second image frames to obtain a third image frame, including:

[0116] The image processing chip selects the L second image frames based on the first timestamp information to obtain a third image frame; wherein each timestamp in the second timestamp information of the third image frame is in the timestamp list of the first timestamp information.

[0117] The timestamp list may include all timestamps in the first timestamp information.

[0118] The AP SoC can send the first timestamp information to the image processing chip to control the image processing chip to filter the third image frame from the L second image frames, so that the timestamp of the filtered third image frame is within the timestamp list of the first timestamp information. This can ensure that the AP matches the fourth image frame that is the same as the third image frame from the N first image frames, so that the image frame used by the image processing chip for image processing is exactly the same as the image frame used by the AP for image post-processing, thereby greatly improving the output image quality of the electronic device.

[0119] The specific process of the image processing method of this embodiment is described in detail below with a specific example. Figure 3 is a schematic diagram of an interactive process of performing image processing by an electronic device provided in some embodiments of the present application, such as Figure 3 As shown, the electronic device may include a camera module, an AP and an ASIC chip, and the interaction process is as follows:

[0120] Step 301, the AP SoC initiates time synchronization with the ASIC chip, so that the clock modules of the ASIC chip and the AP SoC are time synchronized;

[0121] Step 302, when the electronic device starts the camera module, the camera module outputs image frames one by one;

[0122] Step 303: When the ASIC chip is time-synchronized with the AP SoC, the AP SoC receives N first image frames from the image frames output by the camera module, and records the first timestamp information of the received N first image frames; in some embodiments, the N first image frames are image frames output by the camera module within a time period when the electronic device receives a shooting input;

[0123] Step 304: When the ASIC chip is synchronized with the AP SoC, the ASIC chip receives L second image frames from the image frames output by the camera module, and records third timestamp information of the received L second image frames.

[0124] Step 305, the ASIC chip may select a third image frame from the L second image frames, and generate a first processed image through image processing;

[0125] Step 306, the ASIC chip transmits the second timestamp information of the selected third image frame to the AP SoC;

[0126] Step 307, the ASIC chip transmits the first processed image to the AP SoC;

[0127] Step 308, the AP SoC selects a fourth image frame whose timestamp matches the timestamp in the second timestamp information from the N first image frames cached in the AP SoC, and transmits the fourth image frame to the image post-processing module in the AP;

[0128] Step 309: the image post-processing module in the AP performs image post-processing on the first processed image based on the fourth image frame, and outputs a second processed image; in some embodiments, the second processed image is an image captured by the electronic device.

[0129] The image processing method provided in the embodiment of the present application can be executed by an image processing device. In the embodiment of the present application, an image processing device executing the image processing method is taken as an example to illustrate the image processing device provided in the embodiment of the present application.

[0130] See also Figure 4 , Figure 4 : is a structural schematic diagram of an image processing device provided in an embodiment of the present application, which is applied to an electronic device, the electronic device includes a camera module, an application processor and an image processing chip, and the device includes:

[0131] A first receiving module 401 is used for the application processor to receive N first image frames output by the camera module;

[0132] The second receiving module 402 is used for the image processing chip to receive L second image frames output by the camera module; the N first image frames and the L second image frames are image frames output by two branches of the camera module in the same time period, and N and L are both positive integers;

[0133] A first screening module 403 is used for the image processing chip to screen the L second image frames to obtain a third image frame;

[0134] An image processing module 404 is used for the image processing chip to perform image processing based on the third image frame to obtain a first processed image;

[0135] A first sending module 405 is used for the image processing chip to send first information to the application processor; wherein the first information includes identification information of the first processed image and the third image frame;

[0136] A second screening module 406 is used for applying the processor to screen out a fourth image frame that is identical to the third image frame from the N first image frames based on the identification information;

[0137] The image post-processing module 407 is used for applying the processor to perform image post-processing on the first processed image based on the fourth image frame to obtain a second processed image.

[0138] Optionally, the second screening module 406 includes:

[0139] An image screening unit is used for applying a processor to screen out an image frame whose timestamp matches the timestamp in the second timestamp information from the N first image frames based on the first timestamp information of the N first image frames and the second timestamp information of the third image frame, so as to obtain a fourth image frame;

[0140] The identification information includes second timestamp information, the first timestamp information is the timestamp when the application processor receives the N first image frames, and the second timestamp information is the timestamp when the image processing chip receives the third image frame.

[0141] Optionally, the device further comprises:

[0142] A control module, used for the application processor to control the image processing chip to synchronize time with the application processor;

[0143] The acquisition module is used for, when the image processing chip and the application processor are time synchronized, the application processor acquires the first timestamp information of the N received first image frames, and the image processing chip acquires the third timestamp information of the L received second image frames, wherein the third timestamp information includes the second timestamp information.

[0144] Optionally, the control module is specifically configured to:

[0145] The application processor sends a pulse signal and a time synchronization request to the image processing chip, where the time synchronization request includes the first time when the application processor sends the pulse signal;

[0146] When receiving the pulse signal, the image processing chip obtains the reception time of the pulse signal to obtain the second time; and performs time synchronization setting with the application processor based on the time difference between the first time and the second time.

[0147] Optionally, the device further comprises:

[0148] A second sending module, configured for the application processor to send first timestamp information to the image processing chip;

[0149] The first screening module is specifically used for the image processing chip to screen L second image frames to obtain a third image frame based on the first timestamp information; wherein each timestamp in the second timestamp information of the third image frame is in the timestamp list of the first timestamp information.

[0150] Optionally, the image screening unit is specifically used to:

[0151] The application processor performs difference processing on a first target timestamp in the second timestamp information and each timestamp in the first timestamp information, where the first target timestamp is any timestamp in the second timestamp information;

[0152] The application processor determines the image frame corresponding to the second target timestamp among the N first image frames as the image frame whose timestamp matches the first target timestamp, to obtain a fourth image frame;

[0153] The second target timestamp is: a timestamp in the first timestamp information whose absolute value of the difference with the first target timestamp is less than or equal to a preset threshold.

[0154] In this embodiment, the N first image frames output by one branch of the camera module are received by the application processor; and the L second image frames output by another branch of the camera module in the same time period are received by the image processing chip; and the third image frame is screened out from the L second image frames; image processing is performed based on the third image frame to obtain a first processed image; identification information of the first processed image and the third image frame is sent to the application processor; then the application processor screens out a fourth image frame identical to the third image frame from the N first image frames based on the identification information of the third image frame; and based on the fourth image frame, the first processed image is post-processed to obtain a second processed image. Since this embodiment performs image processing on the image frames output by the camera module by the image processing chip, and uses the AP to post-process the first processed image based on the same image frame as the image processing chip, the image processing effect can be improved by performing image processing twice based on the same image frame, thereby improving the output image quality of the electronic device.

[0155] The image processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0156] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0157] The image processing device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0158] Alternatively, if Figure 5 As shown, an embodiment of the present application also provides an electronic device 500, including a processor 501 and a memory 502, wherein the memory 502 stores programs or instructions that can be executed on the processor 501, and when the program or instructions are executed by the processor 501, the various steps of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0159] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0160] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0161] The electronic device 600 includes, but is not limited to, components such as a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609, and a processor 610. The electronic device also includes a camera module, an application processor, and an image processing chip.

[0162] Those skilled in the art will appreciate that the electronic device 600 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 610 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0163] The processor 610 is configured to:

[0164] The application processor receives N first image frames output by the camera module;

[0165] The image processing chip receives L second image frames output by the camera module; obtains a third image frame by screening the L second image frames; performs image processing based on the third image frame to obtain a first processed image; and sends the first information to the application processor; wherein the first information includes identification information of the first processed image and the third image frame, the N first image frames and the L second image frames are image frames output by two branches of the camera module in the same time period, respectively, and N and L are both positive integers;

[0166] The application processor selects a fourth image frame identical to the third image frame from the N first image frames based on the identification information; and performs image post-processing on the first processed image based on the fourth image frame to obtain a second processed image.

[0167] In this embodiment, the N first image frames output by one branch of the camera module are received by the application processor; and the L second image frames output by another branch of the camera module in the same time period are received by the image processing chip; and the third image frame is screened out from the L second image frames; image processing is performed based on the third image frame to obtain a first processed image; identification information of the first processed image and the third image frame is sent to the application processor; then the application processor screens out a fourth image frame identical to the third image frame from the N first image frames based on the identification information of the third image frame; and based on the fourth image frame, the first processed image is post-processed to obtain a second processed image. Since this embodiment performs image processing on the image frames output by the camera module by the image processing chip, and uses the AP to post-process the first processed image based on the same image frame as the image processing chip, the image processing effect can be improved by performing image processing twice based on the same image frame, thereby improving the output image quality of the electronic device.

[0168] Optionally, the processor 610 is further configured to:

[0169] The application processor selects, based on the first timestamp information of the N first image frames and the second timestamp information of the third image frame, an image frame whose timestamp matches the timestamp in the second timestamp information from the N first image frames to obtain a fourth image frame;

[0170] The identification information includes second timestamp information, the first timestamp information is the timestamp when the application processor receives the N first image frames, and the second timestamp information is the timestamp when the image processing chip receives the third image frame.

[0171] Optionally, the processor 610 is further configured to:

[0172] The application processor controls the image processing chip to perform time synchronization with the application processor;

[0173] When the image processing chip is synchronized with the application processor, the application processor obtains first timestamp information of the N received first image frames, and the image processing chip obtains third timestamp information of the L received second image frames, where the third timestamp information includes the second timestamp information.

[0174] Optionally, the processor 610 is further configured to:

[0175] The application processor sends a pulse signal and a time synchronization request to the image processing chip, where the time synchronization request includes the first time when the application processor sends the pulse signal;

[0176] When receiving the pulse signal, the image processing chip obtains the reception time of the pulse signal to obtain the second time; and performs time synchronization setting with the application processor based on the time difference between the first time and the second time.

[0177] Optionally, the processor 610 is further configured to:

[0178] The application processor sends first timestamp information to the image processing chip;

[0179] The image processing chip selects the L second image frames based on the first timestamp information to obtain a third image frame; wherein each timestamp in the second timestamp information of the third image frame is in the timestamp list of the first timestamp information.

[0180] Optionally, the processor 610 is further configured to:

[0181] The application processor performs difference processing on a first target timestamp in the second timestamp information and each timestamp in the first timestamp information, where the first target timestamp is any timestamp in the second timestamp information;

[0182] The application processor determines the image frame corresponding to the second target timestamp among the N first image frames as the image frame whose timestamp matches the first target timestamp, to obtain a fourth image frame;

[0183] The second target timestamp is: a timestamp in the first timestamp information whose absolute value of the difference with the first target timestamp is less than or equal to a preset threshold.

[0184] It should be understood that in the embodiment of the present application, the input unit 604 may include a graphics processor (GPU) 6041 and a microphone 6042, and the graphics processor 6041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 606 may include a display panel 6061, and the display panel 6061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 607 includes a touch panel 6071 and at least one of other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include two parts: a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, a physical keyboard, function keys (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0185] The memory 609 can be used to store software programs and various data. The memory 609 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 609 may include a volatile memory or a non-volatile memory, or the memory 609 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 609 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0186] The processor 610 may include one or more processing units; in some embodiments, the processor 610 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 610.

[0187] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0188] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0189] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0190] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0191] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0192] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0193] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling an electronic device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0194] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image processing method, characterized in that: The method is performed by an electronic device, the electronic device comprising a camera module, an application processor and an image processing chip, and comprises: The application processor receives N first image frames output by the camera module; The image processing chip receives L second image frames output by the camera module; selects a third image frame from the L second image frames; performs image processing based on the third image frame to obtain a first processed image; and sends first information to the application processor; wherein the first information includes identification information of the first processed image and the third image frame, the N first image frames and the L second image frames are image frames output by two branches of the camera module in the same time period, respectively, and N and L are both positive integers; The application processor selects a fourth image frame that is identical to the third image frame from the N first image frames based on the identification information; and performs image post-processing on the first processed image based on the fourth image frame to obtain a second processed image.

2. The method according to claim 1, characterized in that The application processor selects, based on the identification information, a fourth image frame that is identical to the third image frame from the N first image frames, including: The application processor selects, based on the first timestamp information of the N first image frames and the second timestamp information of the third image frame, an image frame whose timestamp matches the timestamp in the second timestamp information from the N first image frames to obtain the fourth image frame; The identification information includes the second timestamp information, the first timestamp information is the timestamp when the application processor receives the N first image frames, and the second timestamp information is the timestamp when the image processing chip receives the third image frame.

3. The method according to claim 2, characterized in that Before the application processor receives the N first image frames output by the camera module, the method further includes: The application processor controls the image processing chip to perform time synchronization with the application processor; When the image processing chip is time synchronized with the application processor, the application processor obtains first timestamp information of the N received first image frames, and the image processing chip obtains third timestamp information of the L received second image frames, wherein the third timestamp information includes the second timestamp information.

4. The method according to claim 3, characterized in that The application processor controls the image processing chip to perform time synchronization with the application processor, including: The application processor sends a pulse signal and a time synchronization request to the image processing chip, wherein the time synchronization request includes a first time when the application processor sends the pulse signal; When receiving the pulse signal, the image processing chip obtains the reception time of the pulse signal to obtain a second time; and performs time synchronization setting with the application processor based on the time difference between the first time and the second time.

5. The method according to claim 2, characterized in that: Before the image processing chip selects the L second image frames to obtain the third image frame, the method further includes: The application processor sends the first timestamp information to the image processing chip; The image processing chip selects the L second image frames to obtain a third image frame, including: The image processing chip selects a third image frame from the L second image frames based on the first timestamp information; wherein each timestamp in the second timestamp information of the third image frame is in a timestamp list of the first timestamp information.

6. The method according to claim 2, characterized in that The application processor selects, based on the first timestamp information of the N first image frames and the second timestamp information of the third image frame, an image frame whose timestamp matches the timestamp in the second timestamp information from the N first image frames to obtain the fourth image frame, including: The application processor performs difference processing on a first target timestamp in the second timestamp information and each timestamp in the first timestamp information, where the first target timestamp is any timestamp in the second timestamp information; The application processor determines the image frame corresponding to the second target timestamp among the N first image frames as the image frame whose timestamp matches the first target timestamp, to obtain the fourth image frame; The second target timestamp is: a timestamp in the first timestamp information whose absolute value of difference with the first target timestamp is less than or equal to a preset threshold.

7. An image processing device, characterized in that: Applied to electronic equipment, the electronic equipment includes a camera module, an application processor and an image processing chip, and the device includes: A first receiving module, used for the application processor to receive N first image frames output by the camera module; A second receiving module, used for the image processing chip to receive L second image frames output by the camera module; the N first image frames and the L second image frames are image frames output by two branches of the camera module in the same time period, and N and L are both positive integers; A first screening module, used for the image processing chip to screen the L second image frames to obtain a third image frame; An image processing module, used for the image processing chip to perform image processing based on the third image frame to obtain a first processed image; A first sending module, configured for the image processing chip to send first information to the application processor; wherein the first information includes identification information of the first processed image and the third image frame; a second screening module, configured for the application processor to screen out a fourth image frame identical to the third image frame from the N first image frames based on the identification information; An image post-processing module is used for the application processor to perform image post-processing on the first processed image based on the fourth image frame to obtain a second processed image.

8. The device according to claim 7, characterized in that The second screening module comprises: an image screening unit, configured for the application processor to screen, from the N first image frames, an image frame whose timestamp matches the timestamp in the second timestamp information, based on the first timestamp information of the N first image frames and the second timestamp information of the third image frame, to obtain the fourth image frame; The identification information includes the second timestamp information, the first timestamp information is the timestamp when the application processor receives the N first image frames, and the second timestamp information is the timestamp when the image processing chip receives the third image frame.

9. The device according to claim 8, characterized in that The device also includes: A control module, used for the application processor to control the image processing chip to perform time synchronization with the application processor; An acquisition module is used for, when the image processing chip is time synchronized with the application processor, the application processor acquires first timestamp information of the N received first image frames, and the image processing chip acquires third timestamp information of the L received second image frames, wherein the third timestamp information includes the second timestamp information.

10. The device according to claim 9, characterized in that The control module is specifically used for: The application processor sends a pulse signal and a time synchronization request to the image processing chip, wherein the time synchronization request includes a first time when the application processor sends the pulse signal; When receiving the pulse signal, the image processing chip acquires the reception time of the pulse signal to obtain a second time; Time synchronization is performed with the application processor based on a time difference between the first time and the second time.

11. The device according to claim 8, characterized in that The device also includes: A second sending module, configured for the application processor to send the first timestamp information to the image processing chip; The first filtering module is specifically used for the image processing chip to filter the L second image frames to obtain a third image frame based on the first timestamp information; wherein each timestamp in the second timestamp information of the third image frame is in the timestamp list of the first timestamp information.

12. The device according to claim 8, characterized in that The image screening unit is specifically used for: The application processor performs difference processing on a first target timestamp in the second timestamp information and each timestamp in the first timestamp information, where the first target timestamp is any timestamp in the second timestamp information; The application processor determines the image frame corresponding to the second target timestamp among the N first image frames as the image frame whose timestamp matches the first target timestamp, to obtain the fourth image frame; The second target timestamp is: a timestamp in the first timestamp information whose absolute value of difference with the first target timestamp is less than or equal to a preset threshold.

13. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the image processing method according to any one of claims 1 to 6 are implemented.

14. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the image processing method according to any one of claims 1 to 6 are implemented.

Images