Image processing method and electronic equipment

Abstract

The invention provides an image processing method and electronic equipment, relates to the technical field of image application, and aims to enable details of images or videos shot by a plurality of cameras to be richer in a scene in which the electronic equipment starts the plurality of cameras for shooting, so that the shooting effect of the electronic equipment can be improved. According to the method, after the electronic equipment responds to a zooming operation input by a user, a target zooming magnification is determined, and a second camera is started; and when the target zoom magnification is greater than or equal to a first preset value and less than or equal to a second preset value, enabling the first camera and the second camera to collect image frames in an overlapping exposure mode, and then carrying out fusion processing on the image frames collected by the first camera and the second camera to generate a preview image.

Description

[0001] This application is a divisional application. The application number of the original application is 202210112240.5, and the original application date is January 29, 2022. The entire contents of the original application are incorporated into this application by reference.

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on October 9, 2021, with application number 202111176340.6 and invention name “An Image Processing Method and Electronic Device”, all contents of which are incorporated by reference in this application. Technical Field

[0003] The present application relates to the field of image application technology, and in particular to an image processing method and electronic equipment. Background Art

[0004] At present, more and more people use electronic devices (such as mobile phones) to take photos and videos to record every bit of life. In order to improve the shooting performance of electronic devices (for example, to make the focal length and shooting functions of electronic devices richer and the image quality better), electronic devices can use multiple cameras to shoot photos and videos. Among them, the functions of the multiple cameras are different. For example, the multiple cameras can include wide-angle cameras, telephoto cameras, and blur cameras, etc. In this way, the multiple cameras can be combined to shoot photos and videos to improve the shooting performance of electronic devices.

[0005] However, since the parameters of different cameras (such as focal length, field of view, etc.) are inconsistent when multiple cameras are combined to take photos and videos, when electronic devices use multiple cameras to take photos and videos, the dynamic range of the photos or videos finally generated will be inconsistent, resulting in poor photo or video effects. Summary of the invention

[0006] The present application provides an image processing method and an electronic device, which can make the details of images or videos captured by multiple cameras richer in a scenario where the electronic device starts multiple cameras to shoot, thereby helping to improve the shooting effect of the electronic device.

[0007] In order to achieve the above technical objectives, this application adopts the following technical solutions:

[0008] In a first aspect, an image processing method is provided, which is applied to an electronic device, the electronic device comprising a first camera and a second camera; the method comprising: the electronic device displays a first preview image in response to a user starting an operation of the first camera; the first preview image is generated by a first long exposure image frame; the first long exposure image frame is generated by the electronic device performing a fusion process on the first image frame and the second image frame; the electronic device determines a target zoom ratio and starts the second camera in response to a zoom operation input by the user, and displays a second preview image corresponding to the target zoom ratio; when the target zoom ratio is greater than or equal to a first preset value and less than or equal to a second preset value, the second preview image is generated by the second long exposure image frame The exposure image frame is generated by fusing the third exposure image frame and the third long exposure image frame; the second long exposure image frame is generated by the electronic device fusing the third image frame and the fourth image frame; the third long exposure image frame is generated by the electronic device fusing the fifth image frame and the sixth image frame; wherein, the exposure time of the first image frame, the third image frame and the fifth image frame is the first exposure time; the exposure time of the second image frame, the fourth image frame and the sixth image frame is the second exposure time; the first exposure time is different from the second exposure time; the first image frame, the second image frame, the third image frame and the fourth image frame are captured by the first camera; the fifth image frame and the sixth image frame are captured by the second camera.

[0009] Based on the first aspect, first, in response to the user's operation of starting the first camera, the electronic device displays a first preview image; then, when the user inputs a zoom operation, the electronic device determines a target zoom ratio and starts the second camera to display a second preview image corresponding to the target zoom ratio; since the second preview image is generated by fusing the second long exposure image frame and the third long exposure image frame; and the second long exposure image frame is generated by the electronic device fusing the third image frame and the fourth image frame; the third long exposure image frame is generated by the electronic device fusing the fifth image frame and the sixth image frame, and the exposure duration of the third image frame and the fifth image frame is the first exposure duration; the exposure duration of the fourth image frame and the sixth image frame is the second exposure duration; The first exposure time is different from the second exposure time; the third image frame and the fourth image frame are captured by the first camera; the fifth image frame and the sixth image frame are captured by the second camera; that is, since the first camera and the second camera adopt the same exposure method to capture the original image frames (i.e., the third image frame, the fourth image frame, the fifth image frame and the sixth image frame), the dynamic range of the second long exposure image frame and the third long exposure image frame generated by the electronic device after fusing the original image frames captured by the first camera and the second camera is consistent; on this basis, the electronic device generates a second preview image by fusing the second long exposure image frame and the third long exposure image frame, so that the details of the second preview image are richer and the effect is better.

[0010] In a possible implementation manner of the first aspect, the method further includes: when the target zoom ratio is greater than or equal to a third preset value and less than or equal to a fourth preset value, the second preview image is generated by a second long exposure image frame; wherein the fourth preset value is less than the first preset value.

[0011] It should be noted that, in this embodiment, when the target zoom ratio is greater than or equal to the third preset value and less than or equal to the fourth preset value, the second preview image is generated by the second long exposure image frame; and the second long exposure image frame is generated by the fusion processing of the third image frame and the fourth image frame captured by the first camera. In other words, in this embodiment, if the target zoom ratio is small, the second preview image is generated by the fusion processing of the original image frame captured by the first camera; on this basis, the second preview image can also be generated by the first long exposure image frame. Exemplarily, the third preset value can be, for example, 1×, and the fourth preset value can be, for example, 4.4×. When the target zoom ratio is 1×, the electronic device can generate a second preview image based on the first long exposure image frame.

[0012] In this design, when the target zoom ratio is greater than or equal to the third preset value and less than or equal to the fourth preset value, the second preview image is generated by the second long exposure image frame; because the fourth preset value is less than the first preset value, that is, when the target zoom ratio is small, the electronic device fuses the original image frames (that is, the third image frame and the fourth image frame) captured by the first camera to generate the second preview image, which is beneficial to reducing the power consumption of the device.

[0013] In a possible implementation manner of the first aspect, the method further includes: when the target zoom ratio is greater than or equal to a fifth preset value, generating the second preview image by a third long exposure image frame; wherein the fifth preset value is greater than the second preset value.

[0014] In this design, when the target zoom ratio is greater than or equal to the fifth preset value, the second preview image is generated by the third long exposure image frame; since the fifth preset value is greater than the second preset value, that is, when the target zoom ratio is large, the electronic device fuses the original image frames (that is, the fifth image frame and the sixth image frame) captured by the second camera to generate the second preview image, which is beneficial to reducing device power consumption.

[0015] In a possible implementation manner of the first aspect, the electronic device activates the second camera, including: when a target zoom magnification is greater than a first preset zoom magnification, the electronic device activates the second camera.

[0016] In this design, the electronic device will activate the second camera only when the target zoom ratio is greater than the first preset zoom ratio, which is beneficial to further reduce the power consumption of the device.

[0017] In a possible implementation manner of the first aspect, the second preview image is a video preview image; the method also includes: the electronic device displays a first interface; the first interface is a preview interface during shooting, and the first interface includes a recording control; the electronic device generates a video file in response to a user's operation of the recording control; wherein the video file is generated by the electronic device through a fusion processing of the second long exposure image frame and the third long exposure image frame.

[0018] It should be noted that, in this embodiment, the video file generated by the electronic device is also related to the target zoom ratio. For example, when the target zoom ratio is greater than or equal to the third preset value and less than or equal to the fourth preset value, the video file is generated by the electronic device fusing the second long exposure image frame. When the target zoom ratio is greater than or equal to the fifth preset value, the video file is generated by the electronic device fusing the third long exposure image frame.

[0019] In this design, since the first camera and the second camera of the electronic device adopt the same exposure method to capture the original image frames, the second preview image generated by the electronic device through the fusion processing of the original image frames captured by the first camera and the second camera has richer image details; further, when the second preview image is a video preview image, the electronic device responds to the user's operation of the recording control and starts recording the video, so that the video recorded by the electronic device can have richer details and better effect.

[0020] In a possible implementation manner of the first aspect, the second preview image is a preview image of the electronic device during video recording.

[0021] In this design, since the first camera and the second camera of the electronic device adopt the same exposure method to capture the original image frames, the second preview image generated by the electronic device through the fusion processing of the original image frames captured by the first camera and the second camera has richer image details; further, when the second preview image is a preview image of the electronic device during the video recording process, the preview image of the electronic device during the video recording process can be made richer in details and the effect is better.

[0022] In a possible implementation manner of the first aspect, the second preview image is a photo preview image; the method also includes: the electronic device displays a second interface; the second interface is a preview interface during shooting, and the second interface includes a photo control; the electronic device generates a photo file in response to the user's operation of the photo control; the photo file is generated by the electronic device through a fusion process of the second long exposure image frame and the third long exposure image frame.

[0023] It should be noted that, in this embodiment, the photo file generated by the electronic device is also related to the target zoom ratio. For example, when the target zoom ratio is greater than or equal to the third preset value and less than or equal to the fourth preset value, the photo file is generated by the electronic device fusing the second long exposure image frame. When the target zoom ratio is greater than or equal to the fifth preset value, the photo file is generated by the electronic device fusing the third long exposure image frame.

[0024] In this design, since the first camera and the second camera of the electronic device adopt the same exposure method to capture the original image frames, the second preview image generated by the electronic device through the fusion processing of the original image frames captured by the first camera and the second camera has richer image details; further, when the second preview image is a photo preview image, the electronic device generates a photo file in response to the user's operation of the photo control, which can make the image details of the generated photo file richer and the effect better.

[0025] In a possible implementation manner of the first aspect, before displaying a second preview image corresponding to a target zoom ratio, the method further includes: the electronic device performs image conversion processing on the second long exposure image frame and the third long exposure image frame; the image conversion processing includes: the electronic device converts the second long exposure image frame into a second long exposure image frame in a target format, and converts the third long exposure image frame into a third long exposure image frame in a target format; a bandwidth of the second long exposure image frame during transmission is higher than a bandwidth of the second long exposure image frame in the target format during transmission, and a bandwidth of the third long exposure image frame during transmission is higher than a bandwidth of the third long exposure image frame in the target format during transmission.

[0026] In this design, before displaying the second preview image corresponding to the target zoom ratio, the electronic device performs image conversion processing on the second long exposure image frame and the third long exposure image frame, that is, the electronic device converts the second long exposure image frame into a second long exposure image frame in the target format, and converts the third long exposure image frame into a third long exposure image frame in the target format; since the bandwidth of the second long exposure image frame during transmission is higher than the bandwidth of the second long exposure image frame in the target format during transmission, and the bandwidth of the third long exposure image frame during transmission is higher than the bandwidth of the third long exposure image frame in the target format during transmission, the bandwidth of the second long exposure image frame and the third long exposure image frame during transmission can be reduced, which is beneficial to reducing device power consumption. In a possible implementation manner of the first aspect, the electronic device includes N consecutive frames of the second long exposure image frames and M consecutive frames of the third long exposure image frames; N≥1, M≥1; the image conversion processing also includes: the electronic device, in the second long exposure image frames of the N consecutive frames, if local information in the second long exposure image frame at the nth moment does not meet a preset condition, then the electronic device repairs it according to the local information in the second long exposure image frames at the n-1th moment and the n+1th moment; n≥2; and / or, in the third long exposure image frames of the M consecutive frames, if local information in the third long exposure image frame at the mth moment does not meet the preset condition, then the electronic device repairs it according to the local information in the third long exposure image frames at the m-1th moment and the m+1th moment; m≥2; wherein the local information includes at least one of color, texture or shape.

[0027] In this design, when the local information of the second long exposure image frame at the current moment is insufficient, the electronic device can supplement it according to the second long exposure image frames at the previous moment and the next moment; when the local information of the third long exposure image frame at the current moment is insufficient, the electronic device can supplement it according to the third long exposure image frames at the previous moment and the next moment, further improving the image display effect.

[0028] In a possible implementation manner of the first aspect, the method further includes: the electronic device performs multi-camera smoothing algorithm processing on the second long exposure image frame in the target format and the third long exposure image frame in the target format; the multi-camera smoothing algorithm is used to reduce noise or distortion of the second long exposure image frame in the target format and the third long exposure image frame in the target format.

[0029] In this design, the electronic device performs multi-camera smoothing algorithm processing on the second long exposure image frame in the target format and the third long exposure image frame in the target format, which is beneficial to reducing noise or distortion of the second long exposure image frame in the target format and the third long exposure image frame in the target format.

[0030] In a possible implementation manner of the first aspect, the method also includes: the electronic device performs a first preset algorithm processing on the second long exposure image frame in the target format and the third long exposure image frame in the target format; wherein the first preset algorithm processing includes at least one of image simulation transformation processing, multi-frame high dynamic range image processing or gamma processing.

[0031] In this design, the electronic device performs a first preset algorithm processing on the second long exposure image frame of the target format and the third long exposure image frame of the target format. Since the first preset algorithm processing includes at least one of image simulation transformation processing, multi-frame high dynamic range image processing or gamma processing, the electronic device can further improve the dynamic range of the image through the first preset algorithm.

[0032] In a possible implementation manner of the first aspect, the method further includes: after the electronic device displays the second preview image, the electronic device caches the image frames captured by the first camera into a first photographing queue; and caches the image frames captured by the second camera into a second photographing queue.

[0033] In a possible implementation of the first aspect, the electronic device generates a photo file in response to a user's operation of a photo control, including: the electronic device selects a first image from a first photo queue and a second image from a second photo queue in response to the user's operation of the photo control; the first image is an image of the latest frame of all images in the first photo queue; the second image is an image of the latest frame of all images in the second photo queue; the electronic device performs a second preset algorithm processing on the first image and the second image to generate a photo file in a target image format; the second preset algorithm processing is used to retain details in the first image and the second image.

[0034] In this design method, when the electronic device is taking a photo, the electronic device selects a first image from the first photo queue and a second image from the second photo queue; since the first image is the latest frame of all images in the first photo queue; and the second image is the latest frame of all images in the second photo queue, when the electronic device processes the first image and the second image using a second preset algorithm to generate a photo file in a target image format, it is beneficial to reduce the delay in producing the photo file.

[0035] In a possible implementation manner of the first aspect, the method further includes: the electronic device processes the first image and the second image using a third preset algorithm; and the third preset algorithm is used to fuse the field of view angles of the first image and the second image.

[0036] In this design, the electronic device processes the first image and the second image using a third preset algorithm; since the third preset algorithm is used to blend the field of view of the first image and the second image, the image effect of the generated photo file can be further improved.

[0037] In a possible implementation of the first aspect, the electronic device performs a second preset algorithm processing on the first image and the second image, including: the electronic device performs a second preset algorithm processing on the first target image in the first photographing queue and the second target image in the second photographing queue; the timestamp of the first target image and the timestamp of the second target image are the same; or, the difference between the timestamp of the first target image and the timestamp of the second target image is less than a preset value.

[0038] In this design method, when the electronic device performs the second preset algorithm processing on the first target image in the first photographing queue and the second target image in the second photographing queue, since the timestamp of the first target image and the timestamp of the second target image are the same; or the difference between the timestamp of the first target image and the timestamp of the second target image is less than the preset value, the power consumption of the device can be further reduced.

[0039] In a second aspect, an electronic device is provided, which has the function of implementing the above-mentioned first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned function.

[0040] According to a third aspect, an electronic device is provided, which includes a display screen, a memory and one or more processors; the display screen, the memory and the processor are coupled; the memory is used to store computer program code, and the computer program code includes computer instructions; when the computer instructions are executed by the processor, the electronic device performs the following steps: the electronic device displays a first preview image in response to a user starting an operation of a first camera; the first preview image is generated by a first long exposure image frame; the first long exposure image frame is generated by the electronic device performing a fusion process on the first image frame and the second image frame; the electronic device determines a target zoom ratio and starts a second camera in response to a zoom operation input by the user, and displays a second preview image corresponding to the target zoom ratio; when the target zoom ratio is greater than or equal to the first When the first preview image is greater than or equal to the first preset value and is less than or equal to the second preset value, the second preview image is generated by fusing the second long exposure image frame and the third long exposure image frame; the second long exposure image frame is generated by the electronic device fusing the third image frame and the fourth image frame; the third long exposure image frame is generated by the electronic device fusing the fifth image frame and the sixth image frame; wherein, the exposure time of the first image frame, the third image frame and the fifth image frame is the first exposure time; the exposure time of the second image frame, the fourth image frame and the sixth image frame is the second exposure time; the first exposure time is different from the second exposure time; the first image frame, the second image frame, the third image frame and the fourth image frame are captured by the first camera; the fifth image frame and the sixth image frame are captured by the second camera.

[0041] In a possible design of the third aspect, when the computer instruction is executed by the processor, the electronic device further performs the following steps: when the target zoom ratio is greater than or equal to a third preset value and less than or equal to a fourth preset value, a second preview image is generated by a second long exposure image frame; wherein the fourth preset value is less than the first preset value.

[0042] In a possible design of the third aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: when the target zoom ratio is greater than or equal to a fifth preset value, the second preview image is generated by a third long exposure image frame; wherein the fifth preset value is greater than the second preset value.

[0043] In a possible design of the third aspect, when the computer instruction is executed by the processor, the electronic device specifically performs the following steps: when the target zoom ratio is greater than the first preset zoom ratio, the electronic device starts the second camera.

[0044] In a possible design of the third aspect, the second preview image is a video preview image; when the computer instruction is executed by the processor, the electronic device further performs the following steps: the electronic device displays a first interface; the first interface is a preview interface during shooting, and the first interface includes a recording control; the electronic device generates a video file in response to the user's operation of the recording control; wherein the video file is generated by the electronic device through a fusion process of the second long exposure image frame and the third long exposure image frame.

[0045] In a possible design of the third aspect, the second preview image is a preview image of the electronic device during video recording.

[0046] In a possible design of the third aspect, the second preview image is a photo preview image; when the computer instruction is executed by the processor, the electronic device further performs the following steps: the electronic device displays a second interface; the second interface is a preview interface during shooting, and the second interface includes a photo control; the electronic device generates a photo file in response to the user's operation of the photo control; the photo file is generated by the electronic device through a fusion process of the second long exposure image frame and the third long exposure image frame.

[0047] In a possible design of the third aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: the electronic device performs image conversion processing on the second long exposure image frame and the third long exposure image frame; the image conversion processing includes: the electronic device converts the second long exposure image frame into a second long exposure image frame in a target format, and converts the third long exposure image frame into a third long exposure image frame in a target format; the bandwidth of the second long exposure image frame during transmission is higher than the bandwidth of the second long exposure image frame in the target format during transmission, and the bandwidth of the third long exposure image frame during transmission is higher than the bandwidth of the third long exposure image frame in the target format during transmission.

[0048] In a possible design of the third aspect, the electronic device includes N consecutive frames of the second long exposure image frames and M consecutive frames of the third long exposure image frames; N≥1, M≥1; when the computer instructions are executed by the processor, the electronic device further performs the following steps: in the second long exposure image frames of the N consecutive frames, if the local information in the second long exposure image frame at the nth moment does not meet the preset condition, the electronic device repairs according to the local information in the second long exposure image frames at the n-1th moment and the n+1th moment; n≥2; and / or, in the third long exposure image frames of the M consecutive frames, if the local information in the third long exposure image frame at the mth moment does not meet the preset condition, the electronic device repairs according to the local information in the third long exposure image frames at the m-1th moment and the m+1th moment; m≥2; wherein the local information includes at least one of color, texture or shape.

[0049] In a possible design of the third aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: the electronic device performs a multi-camera smoothing algorithm on the second long exposure image frame in the target format and the third long exposure image frame in the target format; the multi-camera smoothing algorithm is used to reduce noise or distortion of the second long exposure image frame in the target format and the third long exposure image frame in the target format.

[0050] In a possible design of the third aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: the electronic device performs a first preset algorithm processing on the second long exposure image frame in the target format and the third long exposure image frame in the target format; wherein the first preset algorithm processing includes at least one of image simulation transformation processing, multi-frame high dynamic range image processing or gamma processing.

[0051] In a possible design of the third aspect, when the computer instructions are executed by the processor, the electronic device specifically performs the following steps: the electronic device selects a first image from a first photo queue and a second image from a second photo queue in response to the user's operation of a photo control; the first image is an image of the latest frame of all images in the first photo queue; the second image is an image of the latest frame of all images in the second photo queue; the electronic device performs a second preset algorithm processing on the first image and the second image to generate a photo file in a target image format; the second preset algorithm processing is used to retain details in the first image and the second image.

[0052] In a possible design of the third aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: the electronic device processes the first image and the second image using a third preset algorithm; the third preset algorithm is used to fuse the field of view angles of the first image and the second image.

[0053] In a possible design of the third aspect, when the computer instructions are executed by the processor, the electronic device specifically performs the following steps: the electronic device performs a second preset algorithm processing on the first target image in the first photographing queue and the second target image in the second photographing queue; the timestamp of the first target image is the same as the timestamp of the second target image; or, the difference between the timestamp of the first target image and the timestamp of the second target image is less than a preset value.

[0054] In a fourth aspect, a computer-readable storage medium is provided, wherein computer instructions are stored in the computer-readable storage medium. When the computer instructions are executed on a computer, the computer can execute the image processing method described in any one of the above-mentioned first aspects.

[0055] In a fifth aspect, a computer program product comprising instructions is provided, and when the instructions are executed on a computer, the computer can execute the image processing method described in any one of the first aspects.

[0056] Among them, the technical effects brought about by any design method in the second to fourth aspects can refer to the technical effects brought about by different design methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of an overlapping exposure method provided in an embodiment of the present application;

[0058] Figure 2a A schematic diagram of the composition of a pixel unit provided in an embodiment of the present application;

[0059] Figure 2b A schematic diagram of the composition of a red sub-pixel provided in an embodiment of the present application;

[0060] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0061] Figure 4 A schematic diagram of a software architecture of an electronic device provided in an embodiment of the present application;

[0062] Figure 5a A schematic diagram of a process of a mobile phone entering a video mode provided in an embodiment of the present application;

[0063] Figure 5b A schematic diagram of an image processing process provided in an embodiment of the present application Figure 1 ;

[0064] Figure 6 A schematic diagram of an exposure method of a main camera provided in an embodiment of the present application;

[0065] Figure 7a A schematic diagram of an interface for recording a video on a mobile phone provided in an embodiment of the present application;

[0066] Figure 7b A second schematic diagram of an image processing process provided in an embodiment of the present application;

[0067] Figure 8a A schematic diagram of a zoom ratio fusion of a main camera and a secondary camera provided in an embodiment of the present application;

[0068] Figure 8b A schematic diagram of an image processing process provided in an embodiment of the present application Figure 3 ;

[0069] Fig. 9A schematic diagram of an image processing process provided in an embodiment of the present application Figure 4 ;

[0070] Fig.10a A schematic diagram of a mobile phone camera interface provided in an embodiment of the present application;

[0071] Fig.10b A flowchart of an image processing method provided in an embodiment of the present application is shown in FIG5 ;

[0072] Fig.11 A schematic diagram of an image processing process provided in an embodiment of the present application Figure 6 ;

[0073] Fig.12 Schematic diagram 7 of an image processing process provided in an embodiment of the present application;

[0074] Fig.13 A flowchart of an image processing method provided in an embodiment of the present application;

[0075] Fig.14 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0077] In the related art, an electronic device may use one or more cameras to capture images. Taking the example of an electronic device using one camera to capture images, illustratively, the electronic device starts the main camera (or main camera) to capture raw image frames; then, the electronic device encodes the raw image frames captured by the main camera to generate a preview image. Taking the example of an electronic device using two cameras to capture images, illustratively, the electronic device simultaneously starts the main camera (or main camera) and the secondary camera (or secondary camera) to capture raw image frames; then, the electronic device fuses the raw image frames captured by the main camera and the radiation to generate a preview image.

[0078] However, in some embodiments, due to the different parameters of the main camera and the secondary camera, the main camera can capture multiple original image frames (or multi-frame images) in one exposure period, while the secondary camera can only capture one original image frame (or single-frame image) in one exposure period; in this way, due to the limited dynamic range of the single-frame image, the dynamic range of the original image frame captured by the main camera and the original image frame captured by the secondary camera are inconsistent, which in turn leads to the preview image obtained by merging the original image frame captured by the main camera and the original image frame captured by the secondary camera. The details are not rich enough and the effect is not good enough. Furthermore, since the preview image can be a photo preview image or a video preview image, on this basis, when the user clicks the photo control (or video control), the electronic device can encode the photo preview image to generate a photo file; correspondingly, the electronic device can also encode the video preview image to generate a video file. In this way, the details of the generated photo file (or video file) are not rich enough and the effect is not good.

[0079] Among them, the parameters of the main camera and the secondary camera are different, for example, the focal lengths of the main camera and the secondary camera (such as the main camera is a wide-angle camera and the secondary camera is a telephoto camera) are different, the field of view (FOV) is different, etc.

[0080] Based on this, an embodiment of the present application provides an image processing method, which can be applied to an electronic device with multiple cameras; the method can make the dynamic range of the original image frames collected by each camera in the multiple cameras consistent; then, the electronic device can perform the same dynamic image algorithm synthesis processing on the original image frames collected by each camera, so that the image details of the processed original image frames are richer, and then the preview image generated after the original image frames collected by each camera are fused and processed has richer details, which is beneficial to improving the shooting effect of the electronic device.

[0081] In some embodiments, the electronic device can process the original image frames collected by each camera using the same high dynamic range image (High-Dynamic Range, HDR for short) algorithm. Among them, HDR can provide more dynamic range and image details. It should be noted that the dynamic range refers to the ratio of the maximum value to the minimum value of the variable signal. In the image, the dynamic range is the ratio of the brightness value of the brightest object in the image to the brightness value of the darkest object. The larger the ratio of the brightness value of the brightest object in the image to the brightness value of the darkest object, the more image brightness is displayed in the image, and the brightness displayed by the image is closer to the brightness of the actual shooting environment. Since the dynamic range of the image is large, the image can also show more brightness states, making the image more detailed.

[0082] Exemplarily, the electronic device may include two cameras. For example, the electronic device includes a main camera (or main camera) and a secondary camera (or secondary camera). On this basis, in order to make the dynamic range of the original image frames captured by the main camera and the secondary camera consistent, in some embodiments, the main camera and the secondary camera can capture the original image in the same exposure mode. Then, the electronic device performs HDR algorithm synthesis processing on the original image frames captured by the main camera and the secondary camera, so that the details of the original image frames captured by the main camera and the secondary camera are richer.

[0083] It should be noted that the original image frames captured by the main camera and the auxiliary camera may be, for example, low dynamic range images (LDR).

[0084] In some embodiments, the exposure mode of the main camera and the auxiliary camera to capture the original image frame is an overlapping exposure mode. Exemplarily, the main camera and the auxiliary camera can capture the original image frame in a 2x (or 2-exp) overlapping exposure mode. Specifically, the 2-exp overlapping exposure mode is: in one exposure cycle, the electronic device can control the camera to expose respectively with two exposure durations, and capture image frames within each exposure duration. In some embodiments, the two exposure durations may not be equal, such as the two exposure durations include a long exposure duration and a short exposure duration; the electronic device can capture image frames in a staggered exposure mode of a long exposure duration and a short exposure duration. Based on the overlapping exposure mode, the electronic device can capture two columns of image frames. For example, the electronic device captures a long exposure image frame within a long exposure duration, and captures a short exposure image frame within a short exposure duration.

[0085] For example, Figure 1 FIG. 1 is a schematic diagram of a staggered exposure method provided in an embodiment of the present application. Figure 1 As shown, the long exposure time can be marked as L, and the short exposure time can be marked as S. For example, taking the duration of an exposure cycle of 33ms (milliseconds) as an example, if the camera of the electronic device collects image frames at a rate of 30fps (frames per second), the long exposure time can be as long as 30ms, and the short exposure time can be as short as 5ms. In this way, the camera can collect a column of image frames within 30ms and a column of image frames within 5ms, that is, the camera can collect two columns of image frames in one exposure cycle.

[0086] Take an electronic device including two cameras (such as a main camera and a secondary camera) as an example. For example, the main camera acquires a first image frame within a long exposure time (such as 30ms) and acquires a second image frame within a short exposure time (such as 5ms). The secondary camera acquires a third image frame within a long exposure time (such as 30ms) and acquires a fourth image frame within a short exposure time (such as 5ms). In some embodiments, the electronic device may perform HDR algorithm synthesis processing on the first image frame and the second image frame acquired by the main camera to generate a first long exposure image frame; at the same time, the electronic device performs HDR algorithm synthesis processing on the third image frame and the fourth image frame acquired by the secondary camera to generate a second long exposure image frame. Among them, the first long exposure image frame and the second long exposure image frame have an HDR effect, that is, the first long exposure image frame and the second long exposure image frame have a larger dynamic range and richer image details. Then, the electronic device transmits the first long exposure image frame and the second long exposure image frame to the display screen, so that the display screen displays a preview image according to the first long exposure image frame and the second long exposure image frame. It should be understood that the preview image is an image with an HDR effect.

[0087] It can be understood that, since both the main camera and the secondary camera capture image frames in an overlapping exposure manner, the dynamic range of the first image frame and the second image frame captured by the main camera is consistent with that of the third image frame and the fourth image frame captured by the secondary camera; therefore, the electronic device can use the same HDR algorithm to synthesize the first image frame and the second image frame, as well as the third image frame and the fourth image frame, so that the dynamic range of the synthesized first long exposure image frame and the second long exposure image frame is consistent, and has a larger dynamic range and more image details, that is, the image details of the first long exposure image frame and the second long exposure image frame are richer. Furthermore, when the electronic device transmits the first long exposure image frame and the second long exposure image frame to the display screen, the preview image generated by the display screen based on the first long exposure image frame and the second long exposure image frame is richer in details, thereby improving the shooting effect of the electronic device.

[0088] In addition, when the electronic device uses overlapping exposure to capture the original image frame, the overlapping exposure method allows the camera to capture two columns of image frames in one exposure cycle, and one exposure cycle includes two exposure times of different lengths; therefore, when the electronic device fuses the two columns of image frames captured into one column of image frames based on different exposure times, the image spots, or ghosts, in the image frames can be effectively reduced. Image spots refer to the phenomenon that the image frames captured by the electronic device are whitish or haloed due to different lighting during the shooting process.

[0089] In some other embodiments, the exposure method of the primary camera and the secondary camera to capture the original image frame is a color filter array (CFA) exposure method. The color filter array can also be called a quad bayer coding array (QCFA). For example, Figure 2a As shown in FIG. 1 , a four-pixel Bayer array is a pixel unit including a red sub-pixel (R), two green sub-pixels (G) and a blue sub-pixel (B). The pixel unit may also be referred to as an RGGB pixel unit. Each sub-pixel forms a 2×2 matrix. For example, referring to Figure 2a As shown, R1, R2, R3 and R4 included in the red sub-pixel R form a 2×2 matrix; G1, G2, G3 and G4 included in the green sub-pixel G form a 2×2 matrix and G5, G6, G7 and G8 form a 2×2 matrix; B1, B2, B3 and B4 included in the blue sub-pixel B form a 2×2 matrix.

[0090] Taking the red sub-pixel as an example, for example, Figure 2a ,like Figure 2b As shown, R1 and R4 use the same exposure time to output image frames; R2 and R3 use different exposure times to output image frames. For example, R1 and R4 use medium exposure time (M) to output image frames; R2 uses long exposure time (L) to output image frames; and R3 uses short exposure time (S) to output image frames. Correspondingly, for green sub-pixels, G1 and G4 use the same exposure time to output image frames; and G2 and G3 use different exposure times to output image frames. For example, G1 and G4 use medium exposure time (M) to output image frames; G2 uses long exposure time (L) to output image frames; and G3 uses short exposure time (S) to output image frames. Correspondingly, for blue sub-pixels, B1 and B4 use the same exposure time to output image frames; and B2 and B3 use different exposure times to output image frames. For example, B1 and B4 use a medium exposure time (M) to output image frames; B2 uses a long exposure time (L) to output image frames; and B3 uses a short exposure time (S) to output image frames.

[0091] On this basis, the electronic device can perform HDR algorithm synthesis processing on the image frames output by the long exposure time, the image frames output by the medium exposure time, and the image frames output by the short exposure time to synthesize an image frame with HDR effect, so that the dynamic range of the synthesized image frame is larger and the image details are richer. Furthermore, the display screen of the electronic device can display a preview image according to the synthesized image frame, so that the image details of the preview image are richer, thereby improving the shooting effect of the electronic device.

[0092] It should be noted that the above-mentioned overlapping exposure method and the exposure method of the four-pixel Bayer array are merely some examples of the embodiments of the present application and do not constitute a limitation of the present application. It should be understood that as long as the exposure method can make the dynamic range of the original image frames captured by the main camera and the auxiliary camera consistent, it belongs to the protection scope of the embodiments of the present application, and other exposure methods are not listed one by one here.

[0093] The image processing method provided for the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings of the specification.

[0094] Exemplarily, the electronic device in the embodiment of the present application may be an electronic device with a shooting function. For example, the electronic device may be a mobile sports camera (GoPro), a digital camera, a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, a vehicle-mounted device, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR)\virtual reality (VR) device, etc. The embodiment of the present application does not impose any special restrictions on the specific form of the electronic device.

[0095] like Figure 3 , which is a schematic diagram of the structure of an electronic device 100. The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0096] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0097] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0098] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0099] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0100] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0101] It is understandable that the interface connection relationship between the modules illustrated in this embodiment is only a schematic illustration and does not constitute a structural limitation of the electronic device. In other embodiments, the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0102] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of an electronic device. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0103] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0104] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0105] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0106] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Mini-LED, Micro-OLED, Micro-OLED, quantum dot light-emitting diodes (QLED), etc.

[0107] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0108] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0109] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0110] In an embodiment of the present application, the electronic device may include two or more cameras. Taking the electronic device including two cameras (such as a main camera and a secondary camera) as an example, the main camera and the secondary camera use the same exposure method to capture the original image frame, and then the electronic device performs HDR algorithm synthesis processing on the original image frames captured by the main camera and the secondary camera, so that the original image frames captured by the main camera and the secondary camera of the electronic device have a larger dynamic range and richer image details.

[0111] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals. For example, when an electronic device selects a frequency point, a digital signal processor is used to perform Fourier transform on the frequency point energy.

[0112] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. In this way, the electronic device can play or record videos in multiple coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0113] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and continuously self-learn. NPU can realize applications such as intelligent cognition of electronic devices, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0114] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0115] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110. The speaker 170A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. The microphone 170C, also known as the "microphone" or "microphone", is used to convert sound signals into electrical signals.

[0116] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0117] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, audio, video and other files are stored in the external memory card.

[0118] The internal memory 121 may be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 121. For example, in an embodiment of the present application, the processor 110 may execute the instructions stored in the internal memory 121, and the internal memory 121 may include a program storage area and a data storage area.

[0119] The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0120] The button 190 includes a power button, a volume button, etc. The button 190 can be a mechanical button. It can also be a touch button. The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect the SIM card. The SIM card can be inserted into the SIM card interface 195, or pulled out from the SIM card interface 195 to achieve contact and separation with the electronic device. The electronic device can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0121] In some embodiments, the software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture or a cloud architecture. The present application embodiment takes the layered architecture Android system as an example to exemplarily illustrate the software structure of the electronic device 100.

[0122] Figure 4 A software structure diagram of an electronic device provided in an embodiment of the present application.

[0123] It is understandable that the layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system may include an application layer (application, APP), a framework layer (framework, FWK), a hardware abstraction layer (hardware abstraction layer, HAL) and a kernel layer (kernel). Figure 4 As shown, the Android system may also include an Android runtime and a system library. For ease of understanding, in the embodiments of the present application, Figure 4 The software structure diagram shown also includes some Figure 3 The hardware structure of the electronic equipment in the system, such as camera, display screen, etc.

[0124] The above application layer may include a series of application packages. Figure 4 As shown, the application package may include APP1, APP2, and APP3, etc. In some embodiments, the application package may include some applications with shooting functions (such as camera applications, etc.). Among them, when the electronic device runs the camera application, the electronic device starts the camera and collects the original image frame through the camera. In an embodiment of the present application, the electronic device includes multiple cameras, for example, the electronic device includes two cameras, a main camera and a secondary camera.

[0125] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions. The application framework layer provides programming services to the application layer through the API interface. Figure 4 As shown, the application framework layer includes a camera service framework.

[0126] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0127] The core library consists of two parts: one part is the functional functions that Java voice needs to call, and the other part is the Android core library.

[0128] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0129] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0130] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0131] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0132] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis and layer processing, etc.

[0133] A 2D graphics engine is a drawing engine for 2D drawings.

[0134] The hardware abstraction layer is an interface layer between the kernel layer and the hardware, and can be used to abstract the hardware. Figure 4 As shown, the application abstraction layer includes a camera interface.

[0135] The kernel layer provides the underlying drivers for various hardware of the electronic device. Figure 4 As shown, the kernel layer includes a camera driver module.

[0136] Combination Figure 4 The software structure diagram shown in the figure illustrates the image processing method provided by the embodiment of the present application. In some embodiments, when the user starts the camera application, the electronic device needs to call the camera to collect raw image frames. Based on this, when the user starts the camera application, the camera application triggers the instruction to start the camera. Then, the camera application calls the API interface of the framework layer to send the instruction to start the camera to the camera service framework, and the camera service framework calls the camera interface in the hardware abstraction layer to send the instruction to start the camera to the camera driver module. The camera driver module can drive the camera to collect raw image frames.

[0137] On this basis, the camera transmits the collected raw image frames to the camera interface in the hardware abstraction layer. The camera interface raw image frames are processed by the HDR algorithm, and then the processed raw image frames are transmitted to the display screen, so that the display screen displays the preview image. It should be understood that the processed raw image frames can provide a larger dynamic range and more image details.

[0138] Exemplarily, an ISP may be provided in the camera interface. The camera performs HDR algorithm synthesis processing on the raw image frame through the ISP. In some embodiments, the ISP includes a preset RAW domain processing module, an ISP front-end module, and an ISP back-end module. For example, the camera interface may perform HDR algorithm synthesis processing on the raw image frame through a preset RAW domain processing module.

[0139] Among them, the preset RAW domain processing module refers to a module designed by a preset RAW image processing algorithm. The preset RAW image processing algorithm is a deep learning network for image quality enhancement in the RAW domain. In some embodiments, the preset RAW domain processing algorithm may be a software image processing algorithm. The preset RAW domain processing algorithm may be a software algorithm in a hardware abstraction layer algorithm library of an electronic device. In other embodiments, the preset RAW domain processing algorithm may be a hardware image processing algorithm. The preset RAW domain processing algorithm may be a hardware image processing algorithm implemented by calling the image processing algorithm capability of an ISP.

[0140] It should be noted that the preset RAW domain processing algorithm may also be referred to as a preset image processing algorithm. The reason why it is referred to as a preset RAW domain processing algorithm in the embodiment of the present application is that the input of the preset RAW domain processing algorithm is an image in the RAW domain. The output of the preset RAW domain processing algorithm may be an image in the RAW domain or an image in the RGB domain, and the embodiment of the present application does not limit this.

[0141] For example, taking the electronic device as a mobile phone as an example, in some embodiments, Figure 5a As shown in (1), in response to the user operating the icon 201 of the "Camera" application in the main screen interface of the mobile phone, the mobile phone displays the following Figure 5a Interface 202 shown in (2). Among them, interface 202 is a preview interface for mobile phone photography, which is used to display the preview image when the mobile phone takes pictures (i.e., the photo preview image). For example, when the mobile phone responds to the user's "camera" application icon 201, the mobile phone starts the camera (such as starting the main camera and the secondary camera), and then the mobile phone captures the original image frame through the main camera and the secondary camera, and displays it in the preview interface (i.e., interface 202). Figure 5aAs shown in (2), the interface 202 also includes a "portrait" mode, a "video" mode, a "movie" mode, and a "professional" mode. The "video" mode and the "movie" mode are both used to record video files; the "professional" mode is used to take photos. Taking the "movie" mode as an example, in other embodiments, such as Figure 5a As shown in (2), in response to the user selecting the "Movie" mode 203, the mobile phone displays the following Figure 5a The interface 204 shown in (3) is a preview interface before the mobile phone records video, and the preview interface is used to display the preview image (i.e., video preview image) when the mobile phone records video. For example, when the mobile phone responds to the user selecting the "movie" mode 203, the mobile phone captures the original image frame through the camera (such as starting the main camera and the auxiliary camera) and displays it in the preview interface (i.e., interface 204).

[0142] It should be noted that in the embodiment of the present application, regardless of whether the electronic device is in the state of taking pictures or recording videos, after the electronic device captures the original image frames through the main camera and the auxiliary camera, the original image frames captured by the main camera and the auxiliary camera will be synthesized by the same HDR algorithm and finally displayed in the preview interface. After the HDR algorithm synthesis process, the preview image displayed in the preview interface (such as the photo preview image and the video preview image) has a larger dynamic range and more image details.

[0143] Taking an electronic device including two cameras, the two cameras are a main camera and a secondary camera, and the main camera and the secondary camera use a 2-exp overlap exposure method to capture the original image frame as an example, the specific process of the image processing method provided in the embodiment of the present application is exemplified. Figure 5b As shown, the main camera is connected to the first preset RAW domain processing module, and the first preset RAW domain processing module is connected to the first ISP front-end module. The secondary camera is connected to the second preset RAW domain processing module, and the second preset RAW domain processing module is connected to the second ISP front-end module. Figure 5b As shown, the first ISP front-end module and the second ISP front-end module are respectively connected to the multi-camera smoothing algorithm (satisfiability, SAT) module; the SAT module is respectively connected to the first ISP back-end module and the second ISP back-end module.

[0144] Still Figure 5bAs shown, the main camera uses overlapping exposure to collect the first image frame L0 within the first exposure time, and collects the second image frame S0 within the second exposure time; the main camera inputs the first image frame L0 and the second image frame S0 into the first preset RAW domain processing module; the first preset RAW domain processing module is used to perform HDR algorithm synthesis processing on the first image frame L0 and the second image frame S0 to generate a first long exposure image frame L0'. Then, the first preset RAW domain processing module inputs the first long exposure image frame L0' into the first ISP front-end module, and the first ISP front-end module is used to perform "YUV domain" processing on the first long exposure image frame L0', and convert the first long exposure image frame L0' into the first long exposure image frame L0' in YUV format. Then, the first ISP front-end module transmits the first long exposure image frame L0' in YUV format to the SAT module, and the SAT module is used to smooth the first long exposure image frame L0', improve the image quality of the first long exposure image frame L0', and reduce interference (such as reducing image noise or distortion). Then, the SAT module transmits the processed first long exposure image frame L0 ′ to the first ISP back-end module, and the first ISP back-end module is used to perform image enhancement on the first long exposure image frame L0 ′.

[0145] Accordingly, still Figure 5b As shown, the secondary camera captures overlapping exposure in a manner that the third image frame L1 is captured within the first exposure time, and the fourth image frame S1 is captured within the second exposure time; the secondary camera inputs the third image frame L1 and the fourth image frame S1 into the second preset RAW domain processing module; the second preset RAW domain processing module is used to perform HDR algorithm synthesis processing on the third image frame L1 and the fourth image frame S1 to generate a second long exposure image frame L1'. Then, the second preset RAW domain processing module inputs the second long exposure image frame L1' into the second ISP front-end module, and the second ISP front-end module is used to perform "YUV domain" processing on the second long exposure image frame L1', and convert the second long exposure image frame L1' into a second long exposure image frame L1' in YUV format. Then, the second ISP front-end module transmits the second long exposure image frame L1' in YUV format to the SAT module, and the SAT module is used to perform smoothing processing on the second long exposure image frame L1', improve the image quality of the second long exposure image frame L1', and reduce interference. Then, the SAT module transmits the processed second long exposure image frame L1 ′ to the second ISP back-end module, and the second ISP back-end module is used to perform image enhancement on the second long exposure image frame L1 ′.

[0146] It should be noted that the YUV format is a type of image color encoding, in which Y represents luminance, and U and V represent chrominance. It should be understood that the first long exposure image frame L0 ′ and the second long exposure image frame L1 ′ have a larger dynamic range and richer image details.

[0147] Combined with the above embodiments, Figure 5b As shown, the first ISP backend transmits the first long exposure image frame L0' to the display screen, and at the same time, the second ISP backend transmits the second long exposure image frame L1' to the display screen, so that the display screen displays a preview image according to the first long exposure image frame L0' and the second long exposure image frame L1'. On this basis, when the user clicks the shooting control (such as the photo control and the video control; the video control is used to record a video file), the electronic device encodes the first long exposure image frame L0' and the second long exposure image frame L1' to generate a photo file or a video file.

[0148] It can be seen from the above embodiments that when the camera of the electronic device captures raw image frames in an overlapping exposure manner, a complete exposure cycle includes a long exposure time (for example, marked as Expo L) and a short exposure time (for example, marked as Expo S). Of course, the camera of the electronic device can capture raw image frames in multiple consecutive exposure cycles, and perform HDR algorithm synthesis processing on the raw image frames captured by the camera in each exposure cycle. Taking the camera of the electronic device as the main camera, capturing raw image frames in two consecutive exposure cycles as an example, for example, Figure 6 As shown, the main camera acquires a long exposure image frame VC0(n) in the long exposure time Expo L of the first exposure cycle, and acquires a short exposure image frame VC1(n) in the short exposure time Expo S of the first exposure cycle. Correspondingly, the main camera acquires a long exposure image frame VC0(n+1) in the long exposure time Expo L of the second exposure cycle, and acquires a short exposure image frame VC1(n+1) in the short exposure time Expo S of the second exposure cycle. Then, the main camera transmits the long exposure image frame VC0(n) and the short exposure image frame VC1(n) acquired in the first exposure cycle, and the long exposure image frame VC0(n+1) and the short exposure image frame VC1(n+1) acquired in the second exposure cycle to the first preset RAW domain processing module, and the first preset RAW domain processing module performs HDR algorithm synthesis processing on the long exposure image frame VC0(n) and the short exposure image frame VC1(n) to generate the first long exposure image frame, and performs HDR algorithm synthesis processing on the long exposure image frame VC0(n+1) and the short exposure image frame VC1(n+1) to generate the second long exposure image frame. In this way, the first preset RAW domain processing module includes a plurality of image frames consisting of the first long exposure image frame and the second long exposure image frame.

[0149] Furthermore, after the first long exposure image frame and the second long exposure image frame are processed by the first preset RAW domain processing module, they can also be processed by the first ISP front-end module, the SAT module and the first ISP back-end module. The specific processing method can be referred to Figure 5b And the corresponding embodiments are not described in detail here.

[0150] It should be noted that the camera of the electronic device can also capture original image frames within three consecutive exposure cycles; or capture original image frames within multiple (four or more) exposure cycles. The specific acquisition method can refer to the above embodiment and will not be listed here one by one.

[0151] In an embodiment of the present application, the electronic device includes a plurality of cameras; for example, the electronic device may include a main camera and a secondary camera. Exemplarily, the main camera may be a medium-focus camera (or a standard camera), and the secondary camera may be a telephoto camera. In some embodiments, before a user uses an electronic device to shoot, the display screen of the electronic device displays the image captured by the camera, that is, the display screen of the electronic device displays a preview image. On this basis, the user can enlarge the preview image according to specific needs; for example, the user can input a zoom operation to adjust the size of the preview image displayed on the display screen. Among them, the zoom operation is used for the display screen of the electronic device to display a preview image corresponding to the target zoom ratio. In some embodiments, taking the base zoom ratio of the main camera (i.e., the medium-focus camera) as 1× and the base zoom ratio of the secondary camera (i.e., the telephoto camera) as 5× as an example, for example, when the electronic device is shooting, the electronic device can start the main camera to capture the original image frame. Before the electronic device receives the user's zoom operation, the zoom ratio displayed by the electronic device is the base zoom ratio of the main camera (e.g., 1×). Then, the main camera of the electronic device captures the original image frame and displays the preview image corresponding to the base zoom ratio.

[0152] It should be noted that the above zoom ratio may be an optical zoom ratio or a digital zoom ratio. For example, the above zoom ratio may be 1×, 3×, 4×, 4.5×, 4.9× or 5×, etc. Among them, "1×" means that the zoom ratio is 1x; "3×" means that the zoom ratio is 3x; "4×" means that the zoom ratio is 4x. In addition, the ratio in the embodiment of the present application may also be referred to as a multiple. In other words, the above zoom ratio may also be referred to as a zoom multiple.

[0153] Take a mobile phone as an example. When a mobile phone is recording a video, the mobile phone displays Figure 7a The interface 301 shown in (1) of FIG. 301 includes a recording control 302. In response to the user's operation on the recording control 302, the mobile phone displays the following Figure 7aThe interface 303 shown in (2) is a viewfinder interface of the mobile phone during video recording. The interface 303 includes a pause button 304 and an end recording button 305. Exemplarily, the mobile phone pauses the video recording in response to the user's operation on the pause button 304; or, the mobile phone ends the video recording in response to the user's operation on the end recording button 305, and saves the recorded video in the mobile phone (such as the photo album application).

[0154] Still Figure 7a As shown in (2), the interface 303 also includes a zoom control 306 for adjusting the zoom ratio. For example, the zoom ratio displayed in the interface 303 is 4.5×. When the mobile phone responds to the user's operation of "+" in the zoom control 306, the mobile phone increases the current zoom ratio, for example, to 5.0×. When the mobile phone responds to the user's operation of "-" in the zoom control 306, the mobile phone reduces the current zoom ratio, for example, to 4.0×.

[0155] In some embodiments, when the electronic device receives a zoom operation from the user, the electronic device starts the corresponding camera to capture the original image frame according to the zoom operation. Exemplarily, in order to display a preview image corresponding to the target zoom ratio, the electronic device needs to start the secondary camera, and collect the original image frame through the joint work of the main camera and the secondary camera, and then the electronic device displays the preview image corresponding to the target zoom ratio according to the original image frame. Exemplarily, when the target zoom ratio range is [1.0×, 4.4×], the electronic device starts the main camera to collect the original image frame, and displays the preview image corresponding to the target zoom ratio according to the original image frame collected by the main camera. When the target zoom ratio range is [4.5×, 4.9×], the electronic device starts the main camera and the secondary camera to collect the original image frame at the same time, and displays the preview image corresponding to the target zoom ratio according to the original image frame collected by the main camera and the secondary camera. When the target zoom ratio is greater than 5.0×, the electronic device starts the secondary camera to collect the original image frame, and displays the preview image corresponding to the target zoom ratio according to the original image frame collected by the secondary camera.

[0156] It should be noted that when the target zoom ratio is greater than 5.0×, the main camera of the electronic device can also capture original image frames, but the preview image finally displayed by the electronic device is generated by the original image frames captured by the secondary camera.

[0157] Exemplarily, when a user starts any camera (e.g., the main camera) of the electronic device to shoot, the electronic device displays a preview image. Before the electronic device receives a zoom operation input by the user, the electronic device displays a preview image corresponding to the base zoom ratio (e.g., 1×) of the main camera. After the electronic device receives a zoom operation input by the user, if the zoom operation instructs the electronic device to display a preview image with a target zoom ratio of 4.5×, the electronic device starts the secondary camera. Then, the main camera and the secondary camera of the electronic device simultaneously capture original image frames, and display a preview image with a target zoom ratio of 4.5× based on the original image frames captured by the main camera and the secondary camera.

[0158] In some embodiments, when the electronic device does not receive a zoom operation input by the user, or when the electronic device receives a zoom operation by the user and the target zoom ratio indicated by the zoom operation is in the range of [1.0×, 4.4×], the electronic device uses the original image frame captured by the main camera to display a preview image corresponding to the target zoom ratio. For example, Figure 7b As shown, the main camera uses a 2-exp overlapping exposure method to capture raw image frames. For example, the main camera captures the first image frame L0 within a first exposure time (such as a long exposure time), and captures the second image frame S0 within a second exposure time (such as a short exposure time); then, the main camera inputs the first image frame L0 and the second image frame S0 into the first preset RAW domain processing module, and the first preset RAW domain processing module performs HDR algorithm synthesis processing on the first image frame L0 and the second image frame S0 to generate a first long exposure image frame L0'. The first preset RAW domain processing module transmits the first long exposure image frame L0' to the first ISP front-end module, and the first ISP front-end module performs "YUV domain" processing on the first long exposure image frame L0', and converts the first long exposure image frame L0' into a first long exposure image frame L0' in YUV format. Then, the first ISP front-end module transmits the first long exposure image frame L0' in YUV format to the first ISP back-end module, and the first ISP back-end module performs image enhancement on the first long exposure image frame L0' in YUV format. Finally, the first ISP back-end module outputs the first long exposure image frame L0 ′ to the display screen, so that the display screen displays a preview image according to the first long exposure image frame L0 ′.

[0159] The first ISP front-end module and the first ISP back-end module may be provided with a preset image algorithm to process the first long exposure image frame L0'. Figure 7bAs shown, the first ISP front-end module is preset with a graph transformation matching (GTM) algorithm module; the GTM module is used to perform "YUV domain" processing on the first long exposure image frame L0'. The GTM module is also used to supplement the local information in the image of the previous moment and the next moment in the continuous first long exposure image frame L0' if the local information in the first long exposure image frame L0' at a certain moment is insufficient. The first ISP back-end module is preset with an image simulation transformation (WRAP) algorithm module, a multi-frame HDR algorithm module and a gamma (GAMMA) algorithm module. Among them, the WRAP module is used to perform image enhancement on the first long exposure image frame L0'. The multi-frame HDR algorithm module is used to correct the dynamic range of the first long exposure image frame L0'; for example, after the first long exposure image frame L0' is processed by the multi-frame HDR algorithm module, a continuous multi-frame HDR image is generated. The GAMMA algorithm module can perform electronic image stabilization (EIS) processing on the first long exposure image frame L0', and dynamically compress the first long exposure image frame L0'.

[0160] In some other embodiments, the electronic device receives a zoom operation input by the user, and the zoom operation is used to instruct the electronic device to display a preview image with a target zoom ratio of 4.5×. On this basis, the electronic device responds to the zoom operation and starts the secondary camera to capture the original image frame, that is, the main camera and the secondary camera included in the electronic device capture the original image frame at the same time. For example, Figure 8a As shown, the main camera of the electronic device captures the original image frame at a target zoom ratio of 4.5×, that is, the zoom ratio of the original image frame output by the main camera is 4.5×. At the same time, the secondary camera of the electronic device captures the original image frame at a base zoom ratio of 5.0×, that is, the zoom ratio of the original image frame output by the secondary camera is 5.0×. On this basis, the electronic device performs algorithms such as fusion and equalization on the original image frame with a zoom ratio of 4.5× captured by the main camera and the original image frame with a zoom ratio of 5.0× captured by the secondary camera to generate a preview image with a zoom ratio of 4.5×.

[0161] For example, Figure 8bAs shown, the main camera uses a 2-exp overlapping exposure method to capture raw image frames. For example, the main camera captures the first image frame L0 within a first exposure time (such as a long exposure time), and captures the second image frame S0 within a second exposure time (such as a short exposure time); the main camera inputs the first image frame L0 and the second image frame S0 into the first preset RAW domain processing module; the first preset RAW domain processing module is used to perform HDR algorithm synthesis processing on the first image frame L0 and the second image frame S0 to generate a first long exposure image frame L0'. Then, the first preset RAW domain processing module inputs the first long exposure image frame L0' into the first ISP front-end module, and the first ISP front-end module is used to perform "YUV domain" processing on the first long exposure image frame L0', and convert the first long exposure image frame L0' into a first long exposure image frame L0' in YUV format. Then, the first ISP front-end module transmits the first long exposure image frame L0' in YUV format to the SAT module. At the same time, the secondary camera captures raw image frames in a 2-exp overlapping exposure method. For example, the secondary camera captures the third image frame L1 within the first exposure time (such as the long exposure time), and captures the fourth image frame S1 within the second exposure time (such as the short exposure time); the secondary camera inputs the third image frame L1 and the fourth image frame S1 into the second preset RAW domain processing module; the second preset RAW domain processing module is used to perform HDR algorithm synthesis processing on the third image frame L1 and the fourth image frame S1 to generate a second long exposure image frame L1'. Then, the second preset RAW domain processing module inputs the second long exposure image frame L1' into the second ISP front-end module, and the second ISP front-end module is used to perform "YUV domain" processing on the second long exposure image frame L1', and convert the second long exposure image frame L1' into a second long exposure image frame L1' in YUV format. Then, the second ISP front-end module transmits the second long exposure image frame L1' in YUV format to the SAT module.

[0162] Among them, the SAT module is used to fuse the first long exposure image frame L0' and the second long exposure image frame L1' and perform equalization algorithm processing, and the zoom ratio of the fused first long exposure image frame L0' and the second long exposure image frame L1' is 4.5×. On this basis, the SAT module transmits the first long exposure image frame L0' to the first ISP back-end module, and the first ISP back-end module is used to perform image enhancement on the first long exposure image frame L0'. At the same time, the SAT module transmits the second long exposure image frame L1' to the second ISP back-end module, and the second ISP back-end module is used to perform image enhancement on the second long exposure image frame L1'. Then, the electronic device displays a preview image with a zoom ratio of 4.5× according to the first long exposure image frame L0' and the second long exposure image frame L1'. When the user clicks the shooting control (such as a photo control or a video control), the electronic device encodes the first long exposure image frame L0' and the second long exposure image frame L1' so that the display screen of the electronic device displays a photo file (or video file) with a zoom ratio of 4.5×.

[0163] In addition, the first ISP front-end module and the first ISP back-end module may be provided with a preset image algorithm to process the first long exposure image frame L0'; the second ISP front-end module and the second ISP back-end module may be provided with a preset image algorithm to process the second long exposure image frame L1'. Figure 8bAs shown, the first ISP front-end module and the second ISP front-end module are preset with a GTM algorithm module; the GTM algorithm module is used to process the first long exposure image frame L0' and perform "YUV domain" processing on the second long exposure image frame L1'. The GTM algorithm module is also used to, in the continuous first long exposure image frame L0', if the local information in the first long exposure image frame L0' at a certain moment is insufficient, it can be supplemented according to the local information in the image at the previous moment and the next moment; and in the continuous second long exposure image frame L1', if the local information in the second long exposure image frame L1' at a certain moment is insufficient, it can be supplemented according to the local information in the image at the previous moment and the next moment. The first ISP back-end module and the second ISP back-end module are preset with a WRAP algorithm module, a multi-frame HDR algorithm module, a GAMMA algorithm module, etc. Among them, the WRAP module is used to perform image enhancement on the first long exposure image frame L0' and the second long exposure image frame L1'. The multi-frame HDR algorithm module is used to correct the dynamic range of the first long exposure image frame L0' and the second long exposure image frame L1'; for example, after the first long exposure image frame L0' is processed by the multi-frame HDR algorithm module, a continuous multi-frame HDR image is generated; after the second long exposure image frame L1' is processed by the multi-frame HDR algorithm module, a continuous multi-frame HDR image is generated. The GAMMA algorithm module can perform EIS processing on the first long exposure image frame L0' and the second long exposure image frame L1', and dynamically compress the first long exposure image frame L0' and the second long exposure image frame L1'.

[0164] It should be noted that, in the embodiment of the present application, the GAMMA algorithm module preset in the first ISP backend module can use the ordinary version of EIS to perform EIS processing on the first long exposure image frame L0'. The GAMMA algorithm module preset in the second ISP backend module can use the enhanced version of EIS to perform EIS processing on the second long exposure image frame L1'.

[0165] In some other embodiments, the electronic device receives a zoom operation input by a user, and the zoom operation is used to instruct the electronic device to display a preview image with a target zoom ratio of 5.0×. Based on this, the electronic device responds to the zoom operation and starts the secondary camera to capture the original image frame. For example, Fig. 9As shown, the secondary camera uses a 2-exp overlapping exposure method to capture raw image frames. For example, the secondary camera captures the third image frame L1 within the first exposure time (such as the long exposure time), and captures the fourth image frame S1 within the second exposure time (such as the short exposure time); then, the secondary camera inputs the third image frame L1 and the fourth image frame S1 into the second preset RAW domain processing module, and the second preset RAW domain processing module performs HDR algorithm synthesis processing on the third image frame L1 and the fourth image frame S1 to generate a second long exposure image frame L1'. The second preset RAW domain processing module transmits the first long exposure image frame L1' to the second ISP front-end module, and the second ISP front-end module performs "YUV domain" processing on the second long exposure image frame L1', and converts the second long exposure image frame L1' into a second long exposure image frame L1' in YUV format. Then, the second ISP front-end module transmits the second long exposure image frame L1' in YUV format to the second ISP back-end module, and the second ISP back-end module performs image enhancement on the second long exposure image frame L1' in YUV format. Finally, the second ISP back-end module outputs the second long exposure image frame L1 ′ to the display screen, so that the display screen displays a preview image with a zoom ratio of 5.0× according to the second long exposure image frame L1 ′.

[0166] The second ISP front-end module and the second ISP back-end module may be provided with a preset image algorithm to process the second long exposure image frame L1'. Fig. 9 As shown, the second ISP front-end module is preset with a GTM algorithm module; the GTM module is used to perform "YUV domain" processing on the second long exposure image frame L1'. The GTM module is also used to supplement the local information in the image at the previous moment and the next moment in the continuous second long exposure image frame L1' if the local information in the second long exposure image frame L1' at a certain moment is insufficient. The second ISP back-end module is preset with a WRAP algorithm module, a multi-frame HDR algorithm module and a GAMMA algorithm module, etc. Among them, the WRAP module is used to perform image enhancement on the second long exposure image frame L1'. The multi-frame HDR algorithm module is used to correct the dynamic range of the second long exposure image frame L1'; for example, after the second long exposure image frame L1' is processed by the multi-frame HDR algorithm module, a continuous multi-frame HDR image is generated. The GAMMA algorithm module can perform EIS processing on the second long exposure image frame L1', and dynamically compress the second long exposure image frame L1'.

[0167] It should be understood that in the embodiment of the present application, the electronic device can realize two functions of taking pictures and recording videos, that is, when the display screen of the electronic device displays a preview image, the preview image includes a photo preview image and a video preview image. In some embodiments, the difference between the electronic device realizing the photo function and realizing the video recording function is that the electronic device needs to be compatible with the zero shutter lag (ZSL) mechanism to realize the photo function. Among them, ZSL refers to the electronic device when realizing the photo function, after the display screen of the electronic device displays the photo preview image, the electronic device will keep the latest original image frames in the image cache queue. When the electronic device receives the user's photo operation (that is, the photo control is triggered), the electronic device will find some related original image frames from the image cache queue for encoding and display to generate a photo file. In other words, when the user triggers the photo, the electronic device can find the relevant image frame from the image cache queue at the first time, and immediately present it on the display screen of the electronic device, thereby realizing zero-delay fast photo.

[0168] In some embodiments, the electronic device is further provided with a zero shutter lag manager (ZSL Manager). The ZSL manager is used to manage the original image frames collected by the camera when taking pictures, and to configure, queue, select frames, and perform other operations on the original image frames.

[0169] It should be noted that when an electronic device receives a zoom operation from a user while taking a photo, the electronic device needs to display a preview image corresponding to the target zoom ratio on the one hand, and manage an image cache queue on the other hand, so that the electronic device can quickly display the photo file when receiving the user's photo operation.

[0170] Take a mobile phone as an example. When taking a photo, the phone displays Fig.10a The interface 401 shown in FIG. 401 includes a zoom control 402 for the user to adjust the zoom ratio. Exemplarily, the current zoom ratio displayed in the interface 401 is 1.0×, and then the mobile phone adjusts the current zoom ratio in response to the user's operation on the zoom control 402. For example, the mobile phone increases the current zoom ratio in response to the user's sliding up operation on the zoom control 402.

[0171] Still Fig.10a As shown, the interface 401 also includes a photo control 403. The mobile phone can generate a photo file in response to the user's operation on the photo control 403, and save the photo file to the mobile phone (such as an album application).

[0172] Taking a user holding an electronic device to take a photo as an example, in combination with the above embodiments, when the electronic device is taking a photo, the electronic device can start any camera (such as the main camera) to capture the original image frame. Before the electronic device receives the zoom operation input by the user, the electronic device displays a photo preview image corresponding to the base zoom ratio of the main camera (such as 1×). After the electronic device receives the zoom operation input by the user, if the zoom operation instructs the electronic device to display a photo preview image with a target zoom ratio greater than 4.5×, the electronic device starts the secondary camera. Then, the main camera and the secondary camera of the electronic device simultaneously capture the original image frames, and display the photo preview image with a target zoom ratio greater than 4.5× based on the original image frames captured by the main camera and the secondary camera.

[0173] Exemplarily, when the electronic device does not receive a zoom operation input by the user, or when the electronic device receives a zoom operation by the user, and the target zoom ratio indicated by the zoom operation is in the range of [1.0×, 4.4×], the electronic device uses the original image frame captured by the main camera to display a photo preview image corresponding to the target zoom ratio. When the target zoom ratio indicated by the zoom operation is in the range of [4.5×, 4.9×], the electronic device simultaneously starts the secondary camera to capture the original image frame, and displays the photo preview image corresponding to the target zoom ratio based on the original image frames captured by the main camera and the secondary camera. When the target zoom ratio indicated by the zoom operation is greater than 5.0×, the electronic device starts the secondary camera to capture the original image frame, and displays the photo preview image corresponding to the target zoom ratio based on the original image frame captured by the secondary camera.

[0174] In some embodiments, when the electronic device receives a zoom operation input by the user, and the zoom operation is used to instruct the electronic device to display a photo preview image with a target zoom ratio of 4.4×, such as Fig.10bAs shown, the main camera uses a 2-exp overlapping exposure method to capture raw image frames. For example, the main camera captures the first image frame L0 within a first exposure time (such as a long exposure time), and captures the second image frame S0 within a second exposure time (such as a short exposure time); then, the main camera inputs the first image frame L0 and the second image frame S0 into the first preset RAW domain processing module. On the one hand, the first preset RAW domain processing module performs HDR algorithm synthesis processing on the first image frame L0 and the second image frame S0 to generate a first long exposure image frame L0'. The first preset RAW domain processing module transmits the first long exposure image frame L0' to the first ISP front-end module, and the first ISP front-end module performs "YUV domain" processing on the first long exposure image frame L0', and converts the first long exposure image frame L0' into a first long exposure image frame L0' in YUV format. Then, the first ISP front-end module transmits the first long exposure image frame L0' in YUV format to the first ISP back-end module, and the first ISP back-end module performs image enhancement on the first long exposure image frame L0' in YUV format. Finally, the first ISP back-end module outputs the first long exposure image frame L0' to the display screen, so that the display screen displays the photo preview image according to the first long exposure image frame L0'. On the other hand, the first preset RAW domain processing module retains the first image frame L0 and the second image frame S0 in the image cache queue to generate a first photo queue. Exemplarily, the first preset RAW domain processing module can transfer the first image frame L0 and the second image frame S0 to a first double data rate synchronous dynamic random access memory (double data rate, DDR) to generate a photo queue. Then, when the electronic device receives a user's photo operation, the electronic device reads the first photo queue from the first DDR and processes the first photo queue to generate a photo file in a target image format. Among them, the target image format can be, for example, (joint photographic experts group, JPEG) or other formats, which is not limited in the embodiments of the present application.

[0175] It should be noted that, for the example description of the first ISP front-end module and the first ISP back-end module, reference can be made to the aforementioned embodiments, which will not be described in detail here.

[0176] Exemplarily, the electronic device includes a preset Bayer offline processing module and a preset RAW domain processing module. On this basis, the electronic device can perform Bayer processing and RAW domain processing on the photo queue. Then, the electronic device transmits the processed first photo queue to the first ISP back-end module; the first ISP back-end module is used to generate a photo file in a target image format (such as a JPEG format file) according to the first photo queue.

[0177] For ease of understanding, the Bayer domain and RAW domain are explained here. Bayer domain: Each lens on a digital camera has a light sensor to measure the brightness of light, but to obtain a full-color image, three light sensors are generally required to obtain the information of the three primary colors of red, green, and blue. In order to reduce the cost and size of digital cameras, manufacturers usually use CCD or CMOS image sensors. Usually, the original image output by the CMOS image sensor is in the Bayer domain RGB format. A single pixel contains only one color value. To obtain the grayscale value of the image, it is necessary to first interpolate the color information of each pixel and then calculate the grayscale value of each pixel. In other words, the Bayer domain refers to a raw image format inside a digital camera.

[0178] RAW domain: RAW domain images, that is, original images contain data processed from the image sensor of a digital camera, scanner or film scanner. It is so named because RAW domain images have not been processed, printed or used for editing. RAW domain images contain the most original information of the image and have not been processed by nonlinear processing in the ISP process.

[0179] It should be noted that in the embodiment of the present application, the original image frame captured by the camera of the electronic device is a RAW domain image. On this basis, when the RAW domain image is converted into an image of another format, there will be a phenomenon of image information loss. Based on this, in the above embodiment, when the electronic device receives a user's photo operation, the electronic device reads the first photo queue from the first DDR, and uses a preset RAW domain processing module to perform RAW domain processing on the first photo queue, and then transmits the processed original image frame to the first ISP back-end module to convert it into a photo file in the target image format. In this way, more details in the original image frame can be retained, thereby improving the display effect of the photo file.

[0180] In other embodiments, when the electronic device receives a zoom operation input by the user, and the zoom operation is used to instruct the electronic device to display a photo preview image with a target zoom ratio of 4.5×, such as Fig.11As shown, the main camera uses a 2-exp overlapping exposure method to capture raw image frames. For example, the main camera captures the first image frame L0 within a first exposure time (such as a long exposure time), and captures the second image frame S0 within a second exposure time (such as a short exposure time); then, the main camera inputs the first image frame L0 and the second image frame S0 into the first preset RAW domain processing module. At the same time, the secondary camera uses a 2-exp overlapping exposure method to capture raw image frames. For example, the secondary camera captures the third image frame L1 within a first exposure time (such as a long exposure time), and captures the fourth image frame S1 within a second exposure time (such as a short exposure time); then, the secondary camera inputs the third image frame L1 and the fourth image frame S1 into the second preset RAW domain processing module. On the one hand, the first preset RAW domain processing module performs HDR algorithm synthesis processing on the first image frame L0 and the second image frame S0 to generate a first long exposure image frame L0'. The first preset RAW domain processing module transmits the first long exposure image frame L0' to the first ISP front-end module, and the first ISP front-end module performs "YUV domain" processing on the first long exposure image frame L0', and converts the first long exposure image frame L0' into a first long exposure image frame L0' in YUV format. Then, the first ISP front-end module transmits the first long exposure image frame L0' in YUV format to the first ISP back-end module, and the first ISP back-end module performs image enhancement on the first long exposure image frame L0' in YUV format. Correspondingly, the second preset RAW domain processing module performs HDR algorithm synthesis processing on the third image frame L1 and the fourth image frame S1 to generate a second long exposure image frame L1'. The second preset RAW domain processing module transmits the second long exposure image frame L1' to the second ISP front-end module, and the second ISP front-end module performs "YUV domain" processing on the second long exposure image frame L1', and converts the second long exposure image frame L1' into a second long exposure image frame L1' in YUV format. Then, the second ISP front-end module transmits the second long exposure image frame L1' in YUV format to the second ISP back-end module, and the second ISP back-end module performs image enhancement on the second long exposure image frame L1' in YUV format. Finally, the first ISP back-end module outputs the first long exposure image frame L0' to the display screen, and the second ISP back-end module outputs the second long exposure image frame L1' to the display screen, so that the display screen displays the photo preview image according to the first long exposure image frame L0' and the second long exposure image frame L1'. On the other hand, the first preset RAW domain processing module retains the first image frame L0 and the second image frame S0 in the image cache queue to generate a first photo queue; the second preset RAW domain processing module retains the third image frame L1 and the fourth image frame S1 in the image cache queue to generate a second photo queue.Exemplarily, the first preset RAW domain processing module can transfer the first image frame L0 and the second image frame S0 to the first DDR; the second preset RAW domain processing module can transfer the third image frame L1 and the fourth image frame S1 to the second DDR. Then, when the electronic device receives the user's photo operation, the electronic device reads the first photo queue from the first DDR and reads the second photo queue from the second DDR; the electronic device performs Bayer processing and RAW domain processing on the first photo queue and the second photo queue, respectively. Then, the electronic device transmits the processed first photo queue to the first ISP back-end module, and transmits the processed second photo queue to the second ISP back-end module.

[0181] It should be noted that since the field of view (FOV) of the original image frames captured by the main camera and the original image frames captured by the secondary camera are different, after the electronic device transmits the processed first photographing queue to the first ISP back-end module and transmits the processed second photographing queue to the second ISP back-end module, the electronic device can also perform field of view fusion on the first photographing queue and the second photographing queue, and generate a photographing file in the target image format based on the fused first photographing queue and second photographing queue.

[0182] It should be understood that the first photographing queue is the image frames output by the main camera, and the second photographing queue is the image frames output by the secondary camera. When the electronic device generates a photographing file in the target image format according to the first photographing queue and the second photographing queue, the electronic device can determine the timestamp of each image frame in the first photographing queue output by the main camera, and determine the timestamp of each image frame in the second photographing queue output by the secondary camera, and align the timestamp of each image frame in the first photographing queue output by the main camera with the timestamp of each image frame in the second photographing queue output by the secondary camera.

[0183] For example, when the timestamps of each image frame in the first photographing queue and the timestamps of each image frame in the second photographing queue are the same, the electronic device can perform Bayer processing and RAW domain processing on the image frames with the same timestamps. For another example, when the timestamps of each image frame in the first photographing queue and the timestamps of each image frame in the second photographing queue are different, the electronic device can perform Bayer processing and RAW domain processing on the image frames with two close timestamps as a group of images. Exemplarily, if the difference between the first timestamp in the first photographing queue and the second timestamp in the second photographing queue is less than a preset value, the electronic device can perform Bayer processing and RAW domain processing on the image frames corresponding to the first timestamp and the image frames corresponding to the second timestamp as a group of images.

[0184] Still Fig.11As shown, in the process of generating a photo preview image by the electronic device through the original image frame captured by the main camera and the original image frame captured by the auxiliary camera, when the first preset RAW domain processing module transmits the first long exposure image frame L0' after the first image frame L0 and the second image frame S0 captured by the main camera are fused to the first ISP front-end module, and the second preset RAW domain processing module transmits the second long exposure image frame L1' after the third image frame L1 and the fourth image frame S1 captured by the auxiliary camera are fused to the second ISP front-end module, the 3A statistical algorithm in the first ISP front-end module and the second ISP front-end module can process the first long exposure image frame L0' and the second long exposure image frame L1'. Exemplarily, the 3A statistical algorithm includes an automatic focus (AF) algorithm, an automatic exposure (AE) algorithm, and an automatic white balance (AWB) algorithm.

[0185] In some other embodiments, when the electronic device receives a zoom operation input by the user, and the zoom operation is used to instruct the electronic device to display a photo preview image with a target zoom ratio of 5.0×, such as Fig.12As shown, the secondary camera uses a 2-exp overlapping exposure method to capture raw image frames. For example, the secondary camera captures the third image frame L1 within the first exposure time (such as the long exposure time), and captures the fourth image frame S1 within the second exposure time (such as the short exposure time); then, the secondary camera inputs the third image frame L1 and the fourth image frame S1 into the second preset RAW domain processing module. The second preset RAW domain processing module performs HDR algorithm synthesis processing on the third image frame L1 and the fourth image frame S1 to generate a second long exposure image frame L1'. The second preset RAW domain processing module transmits the second long exposure image frame L1' to the second ISP front-end module, and the second ISP front-end module performs "YUV domain" processing on the second long exposure image frame L1', and converts the second long exposure image frame L1' into a second long exposure image frame L1' in YUV format. Then, the second ISP front-end module transmits the second long exposure image frame L1' in YUV format to the second ISP back-end module, and the second ISP back-end module performs image enhancement on the second long exposure image frame L1' in YUV format. The second ISP back-end module outputs the second long exposure image frame L1' to the display screen, so that the display screen displays the photo preview image according to the second long exposure image frame L1'. On the other hand, the second preset RAW domain processing module retains the third image frame L1 and the fourth image frame S1 in the image cache queue to generate a second photo queue. The second preset RAW domain processing module can transfer the third image frame L1 and the fourth image frame S1 to the second DDR. Then, when the electronic device receives the user's photo operation, the electronic device reads the second photo queue from the second DDR; the electronic device performs Bayer processing and RAW domain processing on the second photo queue. Then, the electronic device transmits the processed second photo queue to the second ISP back-end module; the second ISP back-end module is used to generate a photo file in the target image format (such as a JPEG format file) according to the second photo queue.

[0186] In the above embodiment, when the electronic device is in the photo taking state, the electronic device can retain the original image frames captured by the main camera and / or the auxiliary camera in the image cache queue. When the photo taking control is triggered, the electronic device will find the relevant original image frames from the image cache queue for encoding and display to generate a photo taking file. In other words, when the user triggers the photo taking, the electronic device can find the relevant image frames from the image cache queue at the first time and immediately present them on the display screen of the electronic device, thereby achieving zero-delay fast photo taking.

[0187] Fig.13 A flowchart of an image processing method provided in an embodiment of the present application is shown in FIG. Fig.13 As shown, the method includes S501-S503.

[0188] S501: In response to a user starting an operation of a first camera, the electronic device displays a first preview image.

[0189] The first preview image is generated by a first long exposure image frame; the first long exposure image frame is generated by an electronic device fusing a first image frame with a second image frame; the first image frame and the second image frame are captured by a first camera, and the exposure time of the first image frame is a first exposure time, and the exposure time of the second image frame is a second exposure time. The first exposure time and the second exposure time are different.

[0190] Exemplarily, the first camera may be, for example, the main camera (or main camera) described in the above embodiment. The first preview image may be a video preview image, or a photo preview image.

[0191] In some embodiments, Figure 5b , Figure 7b , Figure 8b , Fig.10b and Fig.11 As shown, the first image frame may be, for example, L0, the second image frame may be, for example, S0, and the first long exposure image frame may be, for example, L0'. In addition, in combination with the above embodiment, the first exposure time is greater than the second exposure time; the first exposure time may be, for example, a long exposure time, and the second exposure time may be, for example, a short exposure time.

[0192] S502: The electronic device determines a target zoom ratio and activates a second camera in response to a zoom operation input by a user.

[0193] For example, Figure 7a and Fig.10a As shown, the zoom operation input by the user may be, for example, a touch operation (such as a sliding operation or a clicking operation) of the user on the control 306 or the control 402. The zoom operation is used to determine the target zoom ratio.

[0194] Among them, the second camera can be, for example, the auxiliary camera (or auxiliary camera) described in the above embodiment.

[0195] S503: The electronic device displays a second preview image corresponding to the target zoom factor.

[0196] Wherein, when the target zoom ratio is greater than or equal to the first preset value and less than or equal to the second preset value, the second preview image is generated by fusing the second long exposure image frame and the third long exposure image frame. The second long exposure image frame is generated by the electronic device fusing the third image frame and the fourth image frame; the third long exposure image frame is generated by the electronic device fusing the fifth image frame and the sixth image frame. The third image frame and the fourth image frame are captured by the first camera; the fifth image frame and the sixth image frame are captured by the second camera. Wherein, the exposure duration of the third image frame and the fifth image frame is the first exposure duration, and the exposure duration of the fourth image frame and the sixth image frame is the second exposure duration.

[0197] For example, Figure 5b , Figure 7b , Figure 8b , Fig.10b and Fig.11 As shown, the third image frame may be, for example, L0, the fourth image frame may be, for example, S0, and the second long exposure image frame may be, for example, L0'. It should be noted that the first image frame and the second image frame and the third image frame and the fourth image frame are collected by the first camera at different times. For example, the first image frame and the second image frame are collected by the first camera at the first moment, and the third image frame and the fourth image frame are collected by the second camera at the second moment.

[0198] For example, Figure 5b , Figure 8b , Fig.11 and Fig.12 As shown, the fifth image frame may be, for example, L1, the sixth image frame may be, for example, S1, and the third long-exposure image frame may be, for example, L1'.

[0199] In some embodiments, in combination with the above embodiments, the first preset value may be, for example, 4.5×, and the second preset value may be, for example, 4.9×. That is, when the target zoom ratio is in the range of [4.5×, 4.9×], the second preview image is generated by fusing the second long exposure image frame and the third long exposure image frame, that is, the second preview image displayed by the electronic device is generated by fusing the original image frames captured by the first camera and the second camera.

[0200] In some embodiments, when the target zoom ratio is greater than or equal to a third preset value and less than or equal to a fourth preset value, the second preview image is generated by the second long exposure image frame, wherein the fourth preset value is less than the first preset value.

[0201] Exemplarily, in combination with the above embodiment, the third preset value may be, for example, 1.0×, and the fourth preset value may be, for example, 4.4×. That is, when the target zoom ratio is in the range of [1.0×, 4.4×], the second preview image is generated by the second long-exposure image frame, that is, the second preview image displayed by the electronic device is generated by fusing the original image frame captured by the first camera.

[0202] It should be noted that, in this embodiment, when the target zoom ratio is greater than or equal to the third preset value and less than or equal to the fourth preset value, the second preview image is generated by the second long exposure image frame; and the second long exposure image frame is generated by the fusion processing of the third image frame and the fourth image frame captured by the first camera. In other words, in this embodiment, if the target zoom ratio is small, the second preview image is generated by the fusion processing of the original image frame captured by the first camera; on this basis, the second preview image can also be generated by the first long exposure image frame. Exemplarily, the third preset value can be, for example, 1×, and the fourth preset value can be, for example, 4.4×. When the target zoom ratio is 1×, the electronic device can generate a second preview image based on the first long exposure image frame.

[0203] In some embodiments, when the target zoom ratio is greater than or equal to a fifth preset value, the second preview image is generated by the third long exposure image frame, wherein the fifth preset value is greater than the second preset value.

[0204] Exemplarily, in combination with the above embodiment, the fifth preset value may be, for example, 5.0×. That is, when the target zoom ratio is greater than or equal to 5.0×, the second preview image is generated by the third long-exposure image frame, that is, the second preview image displayed by the electronic device is generated by fusion processing of the original image frame captured by the second camera.

[0205] In some embodiments, when the target zoom ratio is greater than the first preset zoom ratio, the electronic device activates the second camera. Exemplarily, the first preset zoom ratio may be, for example, 4.4×. That is, when the target zoom ratio is greater than 4.4×, the electronic device activates the second camera. The second camera may be a telephoto camera (such as the secondary camera in the above embodiment).

[0206] In some embodiments, the second preview image is a video preview image; the method also includes: the electronic device displays a first interface; the first interface is a preview interface during shooting, and the first interface includes a recording control; the electronic device generates a video file in response to the user's operation of the recording control; wherein the video file is generated by the electronic device through a fusion process of the second long exposure image frame and the third long exposure image frame.

[0207] For example, Figure 7aAs shown in (1), the first interface may be, for example, interface 301, and the recording control may be, for example, control 302.

[0208] It should be noted that, in this embodiment, the video file generated by the electronic device is also related to the target zoom ratio. For example, when the target zoom ratio is greater than or equal to the third preset value and less than or equal to the fourth preset value, the video file is generated by the electronic device fusing the second long exposure image frame. When the target zoom ratio is greater than or equal to the fifth preset value, the video file is generated by the electronic device fusing the third long exposure image frame.

[0209] In some embodiments, the second preview image is a preview image of the electronic device during video recording.

[0210] Exemplarily, after the user clicks the recording control 302, Figure 7a The second preview image is displayed in the interface 303 shown in (2). It should be noted that during the video recording process, the image displayed in the interface 303 can also be called a preview image. The electronic device will generate a video file only after the user clicks the end recording control 305 again.

[0211] In some embodiments, the second preview image is a photo preview image; the method also includes: the electronic device displays a second interface; the second interface is a preview interface during shooting, and the second interface includes a photo control; the electronic device generates a photo file in response to the user's operation of the photo control; the photo file is generated by the electronic device through a fusion process of the second long exposure image frame and the third long exposure image frame.

[0212] For example, Fig.10a As shown, the second interface may be, for example, interface 401, and the photo taking control may be, for example, control 403.

[0213] It should be noted that, in this embodiment, the photo file generated by the electronic device is also related to the target zoom ratio. For example, when the target zoom ratio is greater than or equal to the third preset value and less than or equal to the fourth preset value, the photo file is generated by the electronic device fusing the second long exposure image frame. When the target zoom ratio is greater than or equal to the fifth preset value, the photo file is generated by the electronic device fusing the third long exposure image frame.

[0214] In some embodiments, before displaying a second preview image corresponding to a target zoom ratio, the method includes: the electronic device performs image conversion processing on a second long exposure image frame and a third long exposure image frame; the image conversion processing includes: the electronic device converts the second long exposure image frame into a second long exposure image frame in a target format, and converts the third long exposure image frame into a third long exposure image frame in a target format; a bandwidth of the second long exposure image frame during transmission is higher than a bandwidth of the second long exposure image frame in the target format during transmission; a bandwidth of the third long exposure image frame during transmission is higher than a bandwidth of the third long exposure image frame in the target format during transmission.

[0215] Exemplarily, in combination with the above embodiments and Figure 7b , Figure 8b , Fig. 9 , Fig.10b , Fig.11 as well as Fig.12 As shown, the electronic device can perform image conversion processing on the second long exposure image frame through the first ISP front-end module, and perform image conversion processing on the third long exposure image frame through the second ISP front-end module. For example, a GTM algorithm module is preset in the first ISP front-end module and the second ISP front-end module, and the GTM algorithm module is used to perform image conversion processing on the second long exposure image frame and the third long exposure image frame. The image conversion processing can be, for example, "YUV domain" processing. On this basis, the second long exposure image frame of the target format obtained after the image conversion processing can be, for example, the second long exposure image frame in the "YUV domain" format; the third long exposure image frame of the target format obtained after the image conversion processing can be, for example, the third long exposure image frame in the "YUV domain" format.

[0216] In some embodiments, the electronic device includes N consecutive second long exposure image frames and M consecutive third long exposure image frames; N≥1, M≥1; the image conversion process also includes: in the second long exposure image frames of the N consecutive frames, if the local information in the second long exposure image frame at the nth moment does not meet the preset conditions, the electronic device repairs it according to the local information in the second long exposure image frames at the n-1th moment and the n+1th moment; n≥2; and / or, in the third long exposure image frames of the M consecutive frames, if the local information in the third long exposure image frame at the mth moment does not meet the preset conditions, the electronic device repairs it according to the local information in the third long exposure image frames at the m-1th moment and the m+1th moment; m≥2; wherein the local information includes at least one of color, texture or shape.

[0217] In combination with the above embodiments, it can be known that the GTM algorithm module is also used to supplement the local information in the second long exposure image frame at the previous moment and the next moment according to the local information in the second long exposure image frame at the current moment when the local information is insufficient; correspondingly, when the local information in the third long exposure image frame at the current moment is insufficient, it is supplemented according to the local information in the third long exposure image frame at the previous moment and the next moment.

[0218] In some embodiments, the method further includes: the electronic device performs a multi-camera smoothing algorithm on the second long exposure image frame in the target format and the third long exposure image frame in the target format; wherein the multi-camera smoothing algorithm is used to reduce noise or distortion of the second long exposure image frame in the target format and the third long exposure image frame in the target format.

[0219] Exemplarily, in combination with the above embodiment, the electronic device may perform multi-shot smoothing algorithm processing on the second long exposure image frame in the target format and the third long exposure image frame in the target format through the SAT algorithm module.

[0220] In some embodiments, the method also includes: the electronic device performs a first preset algorithm processing on the second long exposure image frame in the target format and the third long exposure image frame in the target format; wherein the first preset algorithm processing includes at least one of image simulation transformation processing, multi-frame high dynamic range image processing or gamma processing.

[0221] Exemplarily, the electronic device performs the first preset algorithm processing on the second long exposure image frame in the target format through the first ISP back-end module; and performs the first preset algorithm processing on the third long exposure image frame in the target format through the second ISP back-end module.

[0222] In some embodiments, the method further includes: after the electronic device displays the second preview image, the electronic device caches the image frames captured by the first camera into a first photographing queue, and caches the image frames captured by the second camera into a second photographing queue.

[0223] Exemplarily, the electronic device retains the image frames collected by the first camera into the cache queue through the first preset RAW domain processing module to generate a first photo queue; the electronic device retains the image frames collected by the second camera into the cache queue through the second preset RAW domain processing module to generate a second photo queue. For example, the first preset RAW domain processing module and the second preset RAW domain processing module include DDR. The first preset RAW domain processing module transfers the image frames collected by the first camera to the DDR to generate a first photo queue; the second preset RAW domain processing module transfers the image frames collected by the second camera to the DDR to generate a second photo queue.

[0224] In some embodiments, the electronic device generates a photo file in response to a user's operation of a photo control, including: the electronic device selects a first image from a first photo queue and a second image from a second photo queue in response to the user's operation of the photo control; the first image is an image of the latest frame of all images in the first photo queue, and the second image is an image of the latest frame of all images in the second photo queue; the electronic device performs a second preset algorithm on the first image and the second image to generate a photo file in a target image format; the second preset algorithm is used to retain details in the first image and the second image.

[0225] Exemplarily, the second preset algorithm processing includes Bayer processing and RAW domain processing, wherein the target image format may be JPEG, for example.

[0226] In some embodiments, the method further includes: the electronic device processes the first image and the second image using a third preset algorithm; and the third preset algorithm is used to fuse the field of view of the first image and the second image.

[0227] In some embodiments, the electronic device performs a second preset algorithm processing on the first image and the second image, including: the electronic device performs a second preset algorithm processing on the first target image in the first photographing queue and the second target image in the second photographing queue; wherein the timestamp of the first target image and the timestamp of the second target image are the same; or, the difference between the timestamp of the first target image and the timestamp of the second target image is less than a preset value.

[0228] The present application embodiment provides an electronic device, which may include a display screen, a first camera and a second camera, a memory, and one or more processors; the display screen is used to display images captured by the first camera and the second camera or images generated by the processor; the memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device can perform the various functions or steps performed by the electronic device in the above embodiment. The structure of the electronic device can refer to Figure 3 The structure of the electronic device 100 is shown.

[0229] The present application also provides a chip system, such as Fig.14 As shown, the chip system 1800 includes at least one processor 1801 and at least one interface circuit 1802. The processor 1801 may be Figure 3 The interface circuit 1802 may be, for example, an interface circuit between the processor 110 and the external memory 120 ; or an interface circuit between the processor 110 and the internal memory 121 .

[0230] The processor 1801 and the interface circuit 1802 can be interconnected via lines. For example, the interface circuit 1802 can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit 1802 can be used to send signals to other devices (such as the processor 1801). Exemplarily, the interface circuit 1802 can read instructions stored in the memory and send the instructions to the processor 1801. When the instructions are executed by the processor 1801, the electronic device can execute the various steps performed by the mobile phone 180 in the above embodiment. Of course, the chip system can also include other discrete devices, which is not specifically limited in the embodiments of the present application.

[0231] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes each function or step executed by the electronic device in the above method embodiment.

[0232] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute each function or step executed by the electronic device in the above method embodiment.

[0233] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0234] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0235] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0236] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0237] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0238] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, characterized in that: Applied in an electronic device; the electronic device comprises a first camera and a second camera; the method comprises: In response to a user starting an operation of the first camera, a first preview image is displayed; the first preview image is generated by a first long exposure image frame; the first long exposure frame is generated by a fusion process of a first image frame and a second image frame, the first image frame and the second image frame are collected by the first camera, the exposure time of the first image frame is a first exposure time, the exposure time of the second image frame is a second exposure time, and the first exposure time and the second exposure time are different; wherein the zoom ratio of the first preview image is less than a first preset value; In response to a zoom operation input by a user, a target zoom ratio is determined; when the target zoom ratio is greater than the first preset zoom ratio, the second camera is started to display a second preview image corresponding to the target zoom ratio; wherein the second preview image is generated for images captured by the first camera and / or the second camera.

2. The method according to claim 1, characterized in that When the target zoom ratio is greater than or equal to a first preset value and less than or equal to a second preset value, the second preview image is generated by fusing the second long exposure image frame and the third long exposure image frame; The second long exposure image frame is generated by fusing the third image frame and the fourth image frame; The third long exposure image frame is generated by fusing the fifth image frame and the sixth image frame; The exposure durations of the third image frame and the fifth image frame are the first exposure durations; the exposure durations of the fourth image frame and the sixth image frame are the second exposure durations; and the first exposure duration is different from the second exposure duration; The third image frame and the fourth image frame are captured by the first camera; the fifth image frame and the sixth image frame are captured by the second camera.

3. The method according to claim 2, characterized in that When the target zoom ratio is greater than or equal to a third preset value and less than or equal to a fourth preset value, the second preview image is generated by the second long exposure image frame; The fourth preset value is smaller than the first preset value.

4. The method according to claim 2 or 3, characterized in that: When the target zoom ratio is greater than or equal to a fifth preset value, the second preview image is generated by the third long exposure image frame; Wherein, the fifth preset value is greater than the second preset value.

5. The method according to any one of claims 2 to 4, characterized in that: The second preview image is a video preview image; the method further includes: Display a first interface; the first interface is a preview interface during shooting, and the first interface includes a recording control; In response to the user's operation on the recording control, a video file is generated; wherein the video file is generated by fusing the second long exposure image frame and the third long exposure image frame.

6. The method according to any one of claims 2 to 4, characterized in that: The second preview image is a photo preview image; and the method further includes: Display a second interface; the second interface is a preview interface during shooting, and the second interface includes a photo taking control; In response to the user's operation on the photographing control, a photographing file is generated; the photographing file is generated by fusing the second long-exposure image frame and the third long-exposure image frame.

7. The method according to claim 2, characterized in that Before displaying the second preview image corresponding to the target zoom magnification, the method further includes: performing image conversion processing on the second long exposure image frame and the third long exposure image frame; The image conversion process includes: The second long exposure image frame is converted into a second long exposure image frame in a target format, and the third long exposure image frame is converted into a third long exposure image frame in a target format; the bandwidth of the second long exposure image frame during transmission is higher than the bandwidth of the second long exposure image frame in the target format during transmission, and the bandwidth of the third long exposure image frame during transmission is higher than the bandwidth of the third long exposure image frame in the target format during transmission.

8. The method according to claim 7, characterized in that The electronic device includes N consecutive frames of the second long exposure image frames and M consecutive frames of the third long exposure image frames; N≥1, M≥1; The image conversion process also includes: In the second long exposure image frames of the continuous N frames, if the local information in the second long exposure image frame at the nth moment does not meet the preset condition, repair is performed according to the local information in the second long exposure image frames at the n-1th moment and the n+1th moment; n≥2; and / or In the third long exposure image frame of the continuous M frames, if the local information in the third long exposure image frame at the mth moment does not meet the preset condition, repair is performed according to the local information in the third long exposure image frames at the m-1th moment and the m+1th moment; m≥2; The local information includes at least one of color, texture or shape.

9. The method according to claim 7 or 8, characterized in that: The method further comprises: A multi-camera smoothing algorithm is performed on the second long exposure image frame in the target format and the third long exposure image frame in the target format; the multi-camera smoothing algorithm is used to reduce noise or distortion of the second long exposure image frame in the target format and the third long exposure image frame in the target format.

10. The method according to claim 9, characterized in that The method further comprises: The second long exposure image frame in the target format and the third long exposure image frame in the target format are processed by a first preset algorithm; wherein the first preset algorithm processing includes at least one of image simulation transformation processing, multi-frame high dynamic range image processing or gamma processing.

11. The method according to claim 6, characterized in that The method further comprises: After displaying the second preview image, the image frames captured by the first camera are cached in a first photographing queue; and the image frames captured by the second camera are cached in a second photographing queue.

12. The method according to claim 11, characterized in that In response to the user's operation on the photo control, a photo file is generated, including: In response to the user's operation on the photographing control, a first image is selected from the first photographing queue, and a second image is selected from the second photographing queue; the first image is an image of the latest frame among all images in the first photographing queue; the second image is an image of the latest frame among all images in the second photographing queue; The first image and the second image are processed by a second preset algorithm to generate a photographic file in a target image format; the second preset algorithm is used to retain details in the first image and the second image.

13. The method according to claim 12, characterized in that The method further comprises: The first image and the second image are processed by a third preset algorithm; the third preset algorithm is used to fuse the field of view angles of the first image and the second image.

14. The method according to claim 12 or 13, characterized in that The first image and the second image are processed by a second preset algorithm, including: The first target image in the first photographing queue and the second target image in the second photographing queue are processed by the second preset algorithm; the timestamp of the first target image is the same as the timestamp of the second target image; or, the difference between the timestamp of the first target image and the timestamp of the second target image is less than a preset value.

15. An electronic device, characterized in that: include: A display screen, a first camera and a second camera, a memory, and one or more processors; The display screen is used to display images captured by the first camera and the second camera, or the display screen is used to display images generated by the processor; The processor is used to fuse and process images captured by the first camera and / or the second camera; the memory stores computer program code, and the computer program code includes computer instructions. When the computer instructions are executed by the processor, the electronic device executes the method described in any one of claims 1 to 14.

16. A computer-readable storage medium, characterized in that: The method comprises computer instructions; when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 14.

Images