Image processing method and electronic equipment

Abstract

The invention discloses an image processing method and electronic equipment, and relates to the technical field of image processing, and the method comprises the steps that the electronic equipment displays a preview interface of a shooting scene in response to a starting operation of a camera application, and the preview interface comprises a preview frame image. If the electronic equipment detects that the preview frame image has an overexposure area, when the electronic equipment receives a shooting operation triggered by a user, obtaining a reference frame image and at least one candidate frame image; and the electronic equipment determines a ghosting region corresponding to the short frame image according to the normal exposure image and the short frame image, and performs image fusion processing on the reference frame image and the candidate frame image by adopting an exposure reduction strategy corresponding to the area size of the ghosting region to obtain a first shot image. And if the area of the ghosting region corresponding to the short frame image is greater than the target threshold, the electronic device discards the short frame image. According to the scheme, the influence of the ghosting area in the short-frame image on image fusion can be reduced, and the exposure reduction processing effect of the shot image can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more particularly to an image processing method and an electronic device. Background Technology

[0002] High dynamic range (HDR) is an image or video processing technique that provides a wider range of brightness and color. In dynamic, high-contrast scenes (such as stages, light boards, fireworks, etc., HDR scenes or HDR modes), more details can be captured by increasing the dynamic range of brightness and color. This is achieved by capturing multiple images with different exposures and fusing them to obtain images with a wider dynamic range that are more realistic and vivid.

[0003] Because HDR scenes may contain different lighting fields and object motion conditions, the captured images may have areas of overexposure. Existing technologies use image fusion to reduce the exposure of overexposed areas in the captured images, but ghosting issues appear in the fused image, resulting in unsatisfactory underexposure reduction. Summary of the Invention

[0004] This application provides an image processing method and an electronic device. Before performing exposure reduction processing, the electronic device can acquire the ghosting region of each short frame image corresponding to the normally exposed image. Based on the size of the ghosting region, different exposure reduction strategies can be adopted to perform image fusion processing for images with different exposure values. This can effectively take into account the impact of the ghosting region on the image fusion processing. Especially when the ghosting area is particularly large, the short frame image can be discarded, so that image fusion is performed only based on the short frame image with a smaller ghosting area and the normally exposed image. This reduces the impact of the ghosting region in the short frame image with an excessively large ghosting area on image fusion, reduces the probability of image fusion failure, and can improve the effect of exposure reduction processing on the captured image.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions.

[0006] Firstly, an image processing method is provided, the method comprising:

[0007] In response to the launch of the camera application, the electronic device displays a preview interface of the shooting scene; the preview interface includes preview frame images;

[0008] If the electronic device detects an overexposed area in the preview frame image, when the electronic device receives a shooting operation triggered by the user, the electronic device acquires a reference frame image and at least one candidate frame image; wherein, the reference frame image is a normally exposed image with the same exposure duration as the preview frame image, and the candidate frame image includes at least one short frame image with an exposure duration less than that of the preview frame image;

[0009] The electronic device determines the ghosting region corresponding to the short frame image based on the normal exposure image and the short frame image;

[0010] The electronic device employs an exposure reduction strategy corresponding to the size of the ghost region, performs image fusion processing on images with different exposure values ​​in the reference frame image and the candidate frame image, and obtains the first captured image; wherein, if the area of ​​the ghost region corresponding to the short frame image is greater than the target threshold, the electronic device discards the short frame image.

[0011] In this application, the ghosting region corresponding to the normally exposed image is obtained for each short frame image. Based on the size of the ghosting region, different exposure reduction strategies can be adopted to fuse images with different exposure values. This can effectively take into account the impact of the ghosting region on the image fusion process. Especially when the ghosting area is particularly large, the short frame image can be discarded, so that image fusion is only based on the short frame image with a smaller ghosting area and the normally exposed image. This reduces the impact of the ghosting region in the short frame image with an excessively large ghosting area on the image fusion, reduces the probability of image fusion failure, and can improve the effect of exposure reduction processing on the captured image.

[0012] In one possible implementation of the first aspect, after acquiring the first captured image, the method further includes:

[0013] The electronic device outputs a reminder message to indicate that the preview of the exposure reduction effect has been enabled.

[0014] In this application, after the electronic device performs exposure reduction processing on the first captured image, the electronic device can perform exposure reduction processing on the preview frame image. In the exposure reduction processing strategy of discarding short frames, the image after exposure reduction processing may have poor exposure reduction effect. In order to avoid disappointing users with the final image effect, a reminder message can be output to remind users that the exposure reduction preview effect will be performed, so that users can make a preliminary judgment on the imaging effect.

[0015] In another possible implementation of the first aspect, the exposure reduction strategy includes:

[0016] Using a normally exposed image as a reference frame image, image fusion processing is performed on the reference frame image and at least one candidate frame image.

[0017] Alternatively, a normal exposure image whose acquisition time is closest to that of the short frame image with the smallest ghosting area can be used as a reference frame image, and image fusion processing can be performed on the reference frame image and at least one candidate frame image.

[0018] Alternatively, the shortest frame image with the smallest ghosting region can be used as the reference frame image, and image fusion processing can be performed on the reference frame image and at least one candidate frame image.

[0019] In this application, different exposure reduction strategies are used based on the size of the ghost area to perform exposure reduction processing. This can effectively take into account the impact of the image fusion processing corresponding to the ghost area. In different exposure reduction strategies, the influence of the ghost area is reduced to achieve a balance between overexposure recovery and image quality, so that the first captured image can maintain a good exposure reduction effect as much as possible.

[0020] In another possible implementation of the first aspect, when the candidate frame image includes multiple short frame images, the electronic device employs an exposure reduction strategy corresponding to the area size of the ghost region, performs image fusion processing on the reference frame image and at least one candidate frame image to obtain a first captured image, including:

[0021] If the area of ​​the ghost region corresponding to at least one short frame image is less than or equal to the first threshold, the electronic device uses the normally exposed image as the reference frame image and performs image fusion processing on the reference frame image and the candidate frame image to obtain the first captured image.

[0022] Short frame images in which the area of ​​the candidate frame image that does not contain ghosting regions is greater than the target threshold.

[0023] In this application, different exposure reduction strategies are used based on the size of the ghost area to perform exposure reduction processing. This can effectively take into account the impact of the image fusion processing corresponding to the ghost area. In different exposure reduction strategies, the influence of the ghost area is reduced to achieve a balance between overexposure recovery and image quality, so that the first captured image can maintain a good exposure reduction effect as much as possible.

[0024] In another possible implementation of the first aspect, when the candidate frame image includes multiple short frame images, the electronic device employs an exposure reduction strategy corresponding to the area size of the ghost region, performs image fusion processing on the reference frame image and at least one candidate frame image to obtain a first captured image, including:

[0025] If the area of ​​the ghost region corresponding to all short frame images is greater than the first threshold, and at least one short frame image has a ghost region area less than or equal to the second threshold, the electronic device determines the normal exposure image with the acquisition time closest to the first target short frame image as the reference frame image, and performs image fusion processing on the reference frame image and the candidate frame image to obtain the first captured image.

[0026] Among them, the first target short frame image is the short frame image with the smallest area of ​​the corresponding ghost region among multiple short frame images, and the candidate frame images do not contain short frame images with a ghost region area greater than the target threshold.

[0027] In this application, different exposure reduction strategies are used based on the size of the ghost area to perform exposure reduction processing. This can effectively take into account the impact of the image fusion processing corresponding to the ghost area. In different exposure reduction strategies, the influence of the ghost area is reduced to achieve a balance between overexposure recovery and image quality, so that the first captured image can maintain a good exposure reduction effect as much as possible.

[0028] In another possible implementation of the first aspect, when the candidate frame image includes multiple short frame images, the electronic device employs an exposure reduction strategy corresponding to the area size of the ghost region, performs image fusion processing on the reference frame image and at least one candidate frame image to obtain a first captured image, including:

[0029] If the area of ​​the ghost region corresponding to all short frame images is greater than the second threshold, and at least one short frame image has a ghost region area less than the target threshold, the electronic device uses the second target short frame image as the reference frame image and performs image fusion processing on the reference frame image and the candidate frame image to obtain the first captured image.

[0030] Among them, the second target short frame image is the short frame image with the smallest area of ​​the corresponding ghost region among multiple short frame images, and the candidate frame images do not contain short frame images with a ghost region area greater than the target threshold.

[0031] In this application, different exposure reduction strategies are used based on the size of the ghost area to perform exposure reduction processing. This can effectively take into account the impact of the image fusion processing corresponding to the ghost area. In different exposure reduction strategies, the influence of the ghost area is reduced to achieve a balance between overexposure recovery and image quality, so that the first captured image can maintain a good exposure reduction effect as much as possible.

[0032] In another possible implementation of the first aspect, when the candidate frame image includes multiple short frame images, the electronic device employs an exposure reduction strategy corresponding to the area size of the ghost region, performs image fusion processing on the reference frame image and at least one candidate frame image to obtain a first captured image, including:

[0033] If the area of ​​the ghost region corresponding to all short frame images is greater than the target threshold, the electronic device discards all short frame images, uses the normally exposed image as the reference frame image, and performs image fusion processing on the reference frame image and the remaining candidate frame images to obtain the first captured image.

[0034] The remaining candidate frame images include normally exposed images and / or long frame images with an exposure duration longer than that of the preview frame images.

[0035] In this application, when the ghosting area is particularly large, the short frame image can be discarded, so that image fusion is performed only based on the short frame image with a smaller ghosting area and the normally exposed image. This reduces the impact of the ghosting area in the short frame image with an excessively large ghosting area on image fusion, reduces the probability of image fusion failure, and can improve the effect of underexposure processing on the captured image.

[0036] In another possible implementation of the first aspect, after acquiring the first captured image, the method further includes:

[0037] The electronic device generates and stores exposure reduction strategy adjustment information corresponding to the shooting scene. The exposure reduction strategy adjustment information includes the exposure values ​​of the first preview frame image of the shooting scene and the target short frame image.

[0038] The target short frame image is the short frame image with the smallest corresponding ghosting region among the candidate frame images.

[0039] In this application, the electronic device can generate and store exposure reduction strategy adjustment information corresponding to the shooting scene, and apply the exposure reduction strategy adjustment information to the exposure reduction processing of the preview frame image after the shooting operation, which can improve the efficiency of the electronic device in performing preview exposure reduction processing.

[0040] In another possible implementation of the first aspect, the method further includes:

[0041] In cases where the preview interface is not displayed for the first time after the electronic device launches the camera application, the electronic device acquires a second preview frame image of the current shooting scene;

[0042] The electronic device performs a shooting scene consistency check based on the second preview frame image and the first preview frame image of the current shooting scene;

[0043] When the current shooting scene is consistent with the shooting scene indicated by the exposure reduction strategy adjustment information, the electronic device performs preview exposure reduction processing based on the exposure value of the target short frame image, and displays the preview frame image after exposure reduction processing on the preview interface;

[0044] When the electronic device receives a shooting operation triggered by the user, it performs exposure reduction processing using an exposure reduction strategy corresponding to the preview exposure reduction processing to obtain a second captured image.

[0045] In this application, when the current shooting scene is consistent with the shooting scene indicated by the exposure reduction strategy adjustment information, the electronic device can perform exposure reduction processing on the preview frame image based on the exposure reduction strategy adjustment information, and perform exposure reduction processing on the captured image during subsequent shooting operations, which can improve the efficiency of the electronic device in performing preview exposure reduction processing.

[0046] In another possible implementation of the first aspect, the electronic device performs preview downexposure processing based on the exposure value of the target short frame image, and displays the downexposure-processed preview frame image on a preview interface, including:

[0047] If the exposure value of the target short frame image is less than the preset exposure value, the electronic device will use the exposure value of the target short frame image as the adjustment range for reducing the exposure, thereby reducing the exposure value of the preview frame image in the current shooting scene, and displaying the preview frame image after the exposure value has been reduced on the preview interface.

[0048] In this application, if the exposure value is less than the preset exposure value, it indicates that the target short frame image is a valid short frame image, and the electronic device can perform exposure reduction processing on the preview frame image using the first target exposure value. If the exposure value of the target short frame image is relatively small, then the exposure reduction processing can be performed using the exposure value of the target short frame image. In this case, the first target exposure value is the exposure value of the target short frame image.

[0049] In another possible implementation of the first aspect, when the electronic device receives a user-triggered shooting operation, it performs exposure reduction processing using an exposure reduction strategy corresponding to the preview exposure reduction processing to obtain a second captured image, including:

[0050] When the electronic device receives a shooting operation triggered by the user, the electronic device adopts a zero-delay shooting frame output strategy and obtains the first target reference frame image from the buffer queue corresponding to the zero-delay shooting strategy. The buffer queue corresponding to the zero-delay shooting strategy includes pre-generated normal exposure images and abnormal exposure images generated in response to the shooting operation.

[0051] Based on the distance between each image in the cache queue and the first target reference frame image, a preset number of normally exposed and abnormally exposed images that are closest to each other are obtained from the cache queue and subjected to downexposure processing to obtain the second captured image;

[0052] The first target reference frame image is an image with the same brightness as the preview frame image in the preview interface.

[0053] In this application, when the electronic device receives a shooting operation, it regenerates short-frame images and long-frame images. In this case, according to the image fusion principle, the electronic device sequentially acquires a first number of normally exposed images that are close to the reference frame image, following the order of the image frames in the buffer queue. Then, it acquires a second number of short-frame and long-frame images from the buffer queue, forming a preset number of images for image fusion processing to obtain the second captured image. Since the exposure value of the short-frame images is relatively small, meaning the ghosting area of ​​the short-frame images is relatively small, the probability of image fusion failure due to image fusion based on the short-frame images in the buffer queue is relatively small, ensuring the effect of image underexposure.

[0054] In another possible implementation of the first aspect, the electronic device performs preview downexposure processing based on the exposure value of the target short frame image, and displays the downexposure-processed preview frame image on a preview interface, including:

[0055] If the exposure value of the target short frame image is greater than or equal to the preset exposure value, the electronic device will use the preset exposure value as the adjustment range for reducing the exposure, thereby reducing the exposure value of the preview frame image in the current shooting scene, and displaying the preview frame image after the exposure value has been reduced on the preview interface.

[0056] In this application, if the exposure value is greater than or equal to the preset exposure value, the preview frame image is down-exposed using the second target exposure value. Since the target short frame image has a relatively large exposure value, directly using its exposure value for down-exposing might result in excessive down-exposing and an overly dark preview frame image. In this case, the second target exposure value can be the preset exposure value.

[0057] In another possible implementation of the first aspect, when the electronic device receives a user-triggered shooting operation, it performs exposure reduction processing using an exposure reduction strategy corresponding to the preview exposure reduction processing to obtain a second captured image, including:

[0058] When the electronic device receives a shooting operation triggered by the user, the electronic device adopts a non-zero delay shooting strategy to alternately generate an image sequence containing normally exposed images and abnormally exposed images;

[0059] The electronic device acquires a second target reference frame image from the image sequence, and performs down-exposure processing on a preset number of normally exposed images and abnormally exposed images that are closest to the second target reference frame image to obtain a second captured image;

[0060] The brightness of the second target reference frame image is equal to the difference between the preset exposure value and the exposure value of the target short frame image.

[0061] In this application, since the short frame image and the normally exposed image are close in distance in the non-ZSL frame output strategy, that is, the generation time is close, the displacement difference of the moving object in the short frame image and the normally exposed image is relatively small. Based on this strategy, image fusion can minimize the impact of ghosting regions caused by the displacement difference of moving objects on image fusion and improve the effect of exposure reduction processing.

[0062] In another possible implementation of the first aspect, after displaying the preview interface, the method further includes:

[0063] The electronic device detects whether the shooting scene is a high dynamic range (HDR) scene;

[0064] If the electronic device detects an overexposed area in the preview frame image, upon receiving a user-triggered shooting operation, the electronic device acquires a reference frame image and at least one candidate frame image, including:

[0065] When the electronic device detects that the shooting scene is a high dynamic range (HDR) scene, it performs exposure reduction processing using a preset exposure reduction logic. If the electronic device detects an overexposed area in the preview frame image, it acquires a reference frame image and at least one candidate frame image when it receives a shooting operation triggered by the user.

[0066] In this application, the preview frame image is the image after exposure reduction. That is, after the electronic device performs initial exposure reduction processing on the preview frame image, there are still overexposed areas in the preview frame image, and the exposure reduction effect of the preview frame image is insufficient. In this case, exposure reduction processing on the preview frame image can ensure the exposure reduction effect.

[0067] In another possible implementation of the first aspect, if the electronic device detects an overexposed area in the preview frame image, upon receiving a user-triggered shooting operation, the electronic device acquires a reference frame image and at least one candidate frame image, further comprising:

[0068] If the electronic device detects that the shooting scene is not a high dynamic range (HDR) scene, and the electronic device detects that there is an overexposed area in the preview frame image, when the electronic device receives a shooting operation triggered by the user, the electronic device acquires a reference frame image and at least one candidate frame image.

[0069] In this application, in non-HDR scenes, if no exposure reduction processing is performed, but there are overexposed areas in the preview frame image, in this case, performing exposure reduction processing on the preview frame image can ensure the exposure reduction effect.

[0070] In a second aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in any of the first aspects above.

[0071] Thirdly, a computer-readable storage medium is provided that stores instructions which, when executed by a processor, implement the steps of the method described in any of the first aspects above.

[0072] Fourthly, a computer program product including instructions is provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the method described in any of the first aspects above.

[0073] Fifthly, embodiments of this application provide a chip, the chip including a processor, the processor being configured to invoke a computer program in memory to perform the method as described in any one of the first aspects.

[0074] It is understood that the beneficial effects of the electronic device described in the second aspect, the computer-readable storage medium described in the third aspect, the computer program product described in the fourth aspect, and the chip described in the fifth aspect can be referred to the beneficial effects of the first aspect and any of its possible design embodiments, which will not be repeated here. Attached Figure Description

[0075] Figure 1 Schematic diagrams of images in several HDR scenes where overexposed areas may exist, provided for embodiments of this application;

[0076] Figure 2 A comparative schematic diagram of a luminous sign image before and after exposure reduction, provided as an embodiment of this application;

[0077] Figure 3 This is a schematic diagram illustrating the selection of image frames by an electronic device performing image fusion according to an embodiment of this application.

[0078] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0079] Figure 5 A schematic diagram of a preview interface for a camera application provided in an embodiment of this application;

[0080] Figure 6 A software structure block diagram of an electronic device provided in an embodiment of this application;

[0081] Figure 7 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0082] Figure 8 A schematic diagram of a mobile phone interface provided in an embodiment of this application;

[0083] Figure 9 A schematic diagram of a ghosting region provided in an embodiment of this application;

[0084] Figure 10 A schematic diagram of the ghosting regions corresponding to several short frame images provided in the embodiments of this application;

[0085] Figure 11 A flowchart illustrating another image processing method provided in an embodiment of this application;

[0086] Figure 12 A schematic diagram illustrating a ZSL frame output strategy and a non-ZSL frame output strategy provided for embodiments of this application;

[0087] Figure 13 A possible structural schematic diagram of the electronic device provided in the embodiments of this application;

[0088] Figure 14 This is a schematic diagram of the chip system provided in an embodiment of this application. Detailed Implementation

[0089] In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions "a," "the," "the," "the," and "this" are intended to also include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, "at least one" and "one or more" refer to one or more (including two). The term "and / or" is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can indicate: A exists alone, A and B exist simultaneously, or B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0090] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0091] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0092] Before introducing the image processing method provided in this application, some explanations will be given regarding the terms involved in the embodiments of this application.

[0093] Exposure value (EV, also known as exposure amount / exposure): refers to the intensity and duration of light perceived by the camera. When shooting with a camera, due to the limitation of dynamic range, the exposure value may be too high or too low, resulting in overexposure or underexposure of the subject and background. If overexposed, the image will be too bright; if underexposed, the image will be too dark. When shooting in a high-contrast environment using the camera's normal mode, an underexposed image will show underexposed shadows, failing to reveal details in the shadows; an overexposed image will show overexposed highlights, failing to reveal details in the highlights.

[0094] Exposure parameters include aperture, shutter speed, and ISO. Electronic devices can obtain a suitable exposure value (EV) by controlling one or more of these three parameters.

[0095] Understandably, electronic devices adjust exposure time by controlling shutter speed. The faster the shutter speed, the shorter the exposure time, the smaller the exposure value, and the less bright the image; conversely, the slower the shutter speed, the longer the exposure time, the larger the exposure value, and the brighter the image.

[0096] Aperture size also affects image brightness. Specifically, a larger aperture and a smaller aperture value (e.g., F2.8) result in a higher exposure value and increased image brightness. Conversely, a smaller aperture and a larger aperture value (e.g., F16) result in a lower exposure value and reduced image brightness.

[0097] ISO sensitivity measures how sensitive a photosensitive element is to light. Specifically, higher ISO sensitivity means stronger light resolution, more light is sensed, and image brightness increases; conversely, lower ISO sensitivity means weaker light resolution, less light is sensed, and image brightness decreases. ISO sensitivity is expressed as a sensitivity value (ISO). ISO can be divided into several levels; for example, if adjacent exposure values ​​differ by a factor of two, ISO can be divided into levels such as 50, 100, 200, 400, 800, 1600, and 3200. A higher ISO sensitivity value indicates a stronger light-sensing capability of the photosensitive element.

[0098] A normal exposure image (norm 1-frame image, also known as an N-frame or N-frame image) refers to an image captured by a camera with an exposure value of 0EV. In other words, the exposure value of a normal exposure image is 0EV. It's important to understand that 0EV here is a relative value, not an actual exposure of zero. For example, exposure value = exposure time (T) * ISO. Assuming a normal exposure image is captured at ISO 200 and exposure time T is 50 milliseconds, the actual exposure value corresponding to 0EV is the product of 200 and 50 milliseconds.

[0099] Short frame images (also known as S-frames or S-frame images) are images captured by a camera with an exposure value less than 0 EV. In other words, the exposure value of a short frame image is less than the exposure value (0 EV) of a normally exposed image; for example, the exposure value of a short frame image can be -2 EV, -4 EV, etc.

[0100] Long frame images (also known as L-frames or L-frame images) refer to images captured by a camera with an exposure value greater than 0 EV. In other words, the exposure value of a long frame image is greater than the exposure value (0 EV) of a normally exposed image. For example, the exposure value of a long frame image can be 2 EV, 4 EV, etc.

[0101] It is understandable that both short-frame and long-frame images can be referred to as abnormally exposed images.

[0102] High dynamic range (HDR) is a processing technique that enhances the brightness and contrast of images. Compared to ordinary images, HDR images offer a greater dynamic range and more image detail. Electronic devices can use images with the best detail corresponding to each exposure time to synthesize the final HDR image, thereby improving the image's representation of the visual effects of the real environment.

[0103] Dynamic range (DR) value: used to represent the proportion of overexposed areas in the entire image in the acquired preview image.

[0104] Zero shutter lag (ZSL) refers to a technique where, when a user presses the shutter button, the electronic device captures several images as auxiliary frames and uses a buffered preview frame as a reference frame. The resulting image is then fused with the auxiliary and reference frames to improve image quality (e.g., sharpness, dynamic range). The preview frame is the image displayed on the screen before the user presses the shutter button. The auxiliary frames captured by the phone can be any of the images captured by one or more cameras. The ZSL strategy corresponds to a buffer sequence containing multiple images, including sequentially captured preview frames and short and long frames re-captured for image fusion in HDR scenarios.

[0105] Ghosting refers to abnormal areas that appear in an image of a scene that does not exist in the scene being filmed, creating hollow spaces. For example, due to camera shake or movement of the object being filmed, the content of multiple images may differ significantly. By fusing multiple images to obtain a fused image, abnormal areas that do not exist in the scene being filmed appearing in the fused image are called ghosting.

[0106] In dynamic, high-contrast scenes (such as stages, light boards, fireworks, etc. in HDR scenes / with HDR enabled), more details can be captured by increasing the dynamic range of brightness and color. Image fusion is performed based on normally exposed and abnormally exposed images to obtain images with a wider dynamic range, which are more realistic and vivid.

[0107] Understandably, image fusion processing based on normally exposed and abnormally exposed images can achieve effects such as underexposure reduction. In HDR scenes where some images may have overexposed areas, electronic devices need to perform image fusion based on normally exposed and abnormally exposed images to achieve the effect of underexposure reduction.

[0108] For example, Figure 1 Several illustrative images of HDR scenes where overexposure may exist are provided. For example, Figure 1 In (a), when photographing people moving on a stage, the stage lighting may cause overexposed areas on the faces of the people in the image; for example, Figure 1 In (b), when photographing the illuminated sign, there may be overexposed areas in the luminous part of the sign in the image; for example, Figure 1 In (c), when photographing high-brightness objects such as fireworks and / or campfires, the areas of fireworks and campfires in the image may have overexposed areas; for example, Figure 1 In (d), when shooting outdoor scenery, due to strong sunlight, the part of the image where the sun is located may have overexposed areas, etc.

[0109] For example, when photographing an illuminated sign, the electronic device can capture both a normally exposed image and an abnormally exposed image of the sign. The abnormally exposed image may include short frame images. The electronic device can then perform image fusion processing using the normally exposed and abnormally exposed images to obtain a fused image, thereby reducing the exposure of overexposed areas in the illuminated sign image. For example, Figure 2 A comparative illustration of an illuminated sign image before and after underexposure is provided. Figure 2 Image 1, which includes an overexposed area, is shown in (a). Figure 2 (b) shows a schematic of image 2 after the overexposed areas of image 1 have been downexposed.

[0110] In a single shooting session, the electronic device may acquire at least one short-frame image, and at least one normally exposed image; this application does not limit this. When there are multiple short-frame images, the exposure values ​​between the short-frame images may be the same or different. That is, the electronic device can capture multiple short-frame images with the same exposure value, such as capturing two -2EV short-frame images, or it can capture short-frame images with different exposure values, such as capturing one -2EV short-frame image and one -8EV short-frame image. In some embodiments, when the electronic device captures images using the HDR function, it may not capture long-frame images, or it may capture at least one long-frame image.

[0111] For example, refer to Figure 3 A schematic diagram is given of an electronic device selecting image frames when performing image fusion. Figure 3 The paper presents a cache queue corresponding to the ZSL strategy.

[0112] An electronic device can capture multiple preview frame images (normal exposure images) and store them in a buffer queue in the order they were captured. In response to a shooting operation in an HDR scene, the electronic device selects a preview frame image from the buffer sequence as a reference frame image. Simultaneously, the electronic device recaptures abnormal exposure images (e.g., short frame images) and stores them in the buffer queue. There is a time delay between the electronic device determining the reference frame image and the electronic device recapturing the abnormal exposure image. Following the ZSL strategy, where frame images are arranged in the buffer queue in the order of capture, the recaptured short frame image is stored at the end of the buffer queue, such as... Figure 3 The preview frame image and short frame image are shown. When an electronic device performs image fusion based on a reference frame image, theoretically, image frames (normal exposure images and abnormal exposure images) need to be selected from the buffer queue in ascending order of their acquisition time from the reference frame image. The newly acquired short frame image (abnormal exposure image) is stored at the end of the queue, further away from the reference frame image. If there are moving objects in the captured image, the longer the time interval between two frames, the greater the differences in content / brightness, etc., between the captured images. Therefore, when based on... Figure 3 When the short frame image and the preview frame image shown are fused, there may be misaligned edge areas between multiple frames in the fused image. In other words, ghosting may appear in the fused image, causing fusion failure and poor exposure reduction effect.

[0113] This application provides an image processing method. When an electronic device detects that an overexposed area still exists in the preview frame image, the electronic device can determine the area of ​​the ghost region based on the overexposed areas of the preview frame image and the short frame image. Thus, when a shooting operation is received, different exposure reduction strategies can be adopted to fuse images with different exposure values ​​based on the size of the ghost region. This method can effectively take into account the impact of the ghost region on the image fusion process. Especially when the ghost area is particularly large, the short frame image can be discarded, so that image fusion is performed only based on the short frame image with a smaller ghost area and the normally exposed image. This reduces the impact of the ghost region in the short frame image with an excessively large ghost area on the image fusion, reduces the probability of image fusion failure, and can improve the effect of exposure reduction processing on the captured image.

[0114] The electronic device can be a portable computer (such as a mobile phone), tablet computer, laptop computer, personal computer (PC), wearable electronic device (such as a smartwatch), augmented reality (AR) / virtual reality (VR) device, or other device with a camera function. The following embodiments do not impose any special restrictions on the specific form of the electronic device.

[0115] Figure 4 A schematic diagram of the electronic device is shown.

[0116] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, sensor module 180, camera 193, display screen 194, etc.

[0117] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0118] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0119] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0120] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0121] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0122] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0123] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0124] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0125] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0126] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0127] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0128] In this embodiment, after the electronic device launches the camera application, a preview interface of the camera application can be displayed on the screen. (See reference) Figure 5 A schematic diagram of a camera application preview interface is provided. For example, as shown below... Figure 5 (a) The camera app's preview interface may include preview frame images, function components, mode components, a gallery button, and a shooting button. Function components may include buttons to turn HDR on / off, flash on / off, AI shooting on / off, and settings buttons, etc. Mode components may include slow motion options, video options, photo options, portrait options, and professional options, etc.

[0129] When an electronic device displays a preview of a camera application, it can detect whether HDR needs to be enabled for the current shooting scene; that is, the electronic device can automatically enable or disable HDR. Alternatively, the electronic device can enable or disable HDR in response to a user's action on the HDR on / off button. In some embodiments, if the electronic device automatically enables HDR, it can also output a reminder message indicating that HDR is enabled on the display interface. For example, such as... Figure 5 (b) The notification message for enabling HDR can be “HDR scene detected, HDR is enabled”. The notification message for enabling HDR can be displayed in the preview interface in the form of pop-up window, floating box, etc.

[0130] Electronic devices can acquire images in response to a user's action on the shutter button. For example, when HDR is enabled, the electronic device, in response to the user's action on the shutter button, acquires at least one normally exposed image and at least one abnormally exposed image, then performs image fusion based on the normally and abnormally exposed images to achieve an underexposure processing effect in HDR scenes. Alternatively, when HDR is disabled (non-HDR scene), the electronic device, in response to the user's action on the shutter button, acquires a normally exposed image. The electronic device can store the fused image obtained in the HDR scene, or the normally exposed image obtained in the non-HDR scene, in its image library.

[0131] In this embodiment, after detecting an HDR scene, the electronic device, upon receiving a user's action on the shutter button, can use the image processing method provided in this embodiment to determine whether image exposure reduction processing is needed, and perform exposure reduction processing if necessary. Simultaneously, the electronic device can generate an exposure reduction strategy corresponding to the current HDR scene. Furthermore, the electronic device can display a reminder on the screen that the exposure reduction strategy has been enabled, and that exposure reduction processing will be applied to both the preview frame image and the captured image. The preview frame image in the preview interface at this time represents the effect after exposure reduction processing. (Reference) Figure 5 (c) After the electronic device completes the exposure reduction processing of the image, the electronic device displays a reminder message on the screen. The reminder message is "Exposure reduction preview enabled".

[0132] In response to a user's action on the gallery button, an electronic device may display a gallery interface on its screen. An example of a gallery interface can be found here. Figure 5 As shown in (d), the gallery interface can include thumbnails of multiple stored images, and the gallery interface can also include function options such as pictures, albums, moments, and creation.

[0133] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0134] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0135] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0136] In this embodiment, the electronic device can capture images with different exposure values ​​through one or more cameras. For example, it can capture normal exposure images and abnormal exposure images (short frame images and / or long frame images) through one or more cameras. Specifically, when the electronic device displays the preview interface of the camera application, it can continuously capture preview frame images through one or more cameras. The preview frame images are normal exposure images. When the electronic device receives a user's action on the shutter button, it can acquire a captured image. The captured image can be a fusion of the preview frame images and the captured short frame images, or it can be the preview frame images themselves. The electronic device can capture at least one normal exposure image through camera 1 at ISO 400 and an exposure time of 20ms, and at least one short frame image through camera 2 at ISO 100 and an exposure time of 5ms. Image fusion processing is then performed based on the normal exposure images and the short frame images to achieve the effect of reducing exposure.

[0137] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP performs Fourier transforms on the frequency energy.

[0138] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0139] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0140] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0141] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0142] Electronic device 100 can implement audio functions such as music playback and recording through audio module 170, speaker, receiver, microphone, headphone jack, and application processor.

[0143] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses a layered architecture. Taking the system as an example, the software structure of electronic device 100 is illustrated.

[0144] Figure 6 This is a software structure block diagram of an electronic device 100 according to an embodiment of the present invention. The layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, [the following is omitted as the text is incomplete and requires further context]. The system is divided into an application layer, an application framework layer, a hardware abstraction layer (HAL), and a hardware layer.

[0145] The application layer can include a series of application packages. For example... Figure 6 As shown, the application package may include applications such as camera apps / third-party apps that can be used to take pictures, as well as gallery apps for storing the captured images.

[0146] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes predefined interface functions, such as the camera module interface.

[0147] In some embodiments, the application framework layer may also include modules for implementing other functionalities. For example, a window manager, a content provider, a view system, a resource manager, and so on.

[0148] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0149] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0150] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0151] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0152] The HAL layer includes the camera HAL. The camera HAL provides an interface for unified management of other underlying devices that can be accessed by the camera or camera application. An image processing module is also deployed in the HAL layer. This image processing module may include a scene detection unit, an exposure reduction unit, and a feedback unit, among others.

[0153] The hardware layer includes underlying components such as the ISP, camera (same as image sensor), and display screen.

[0154] The camera is used to acquire image data upon receiving a shooting command. The data acquired by the camera is RAW data. The ISP in the electronic device can convert the RAW data acquired by the camera into YUV data that can be processed by the HAL layer / CPU / GPU, and perform pre-processing on the YUV image, such as denoising, cropping, enhancement, and sharpening.

[0155] In some embodiments, a kernel layer is further included between the HAL layer and the hardware layer. The kernel layer includes drivers corresponding to various devices in the hardware layer, used to enable the devices and issue instructions and upload data corresponding to each device. For example, the kernel layer includes camera drivers, display drivers, etc.

[0156] Images processed through preprocessing can be stored in a buffer queue. This buffer queue can correspond to a ZSL (Zero-Site Frame) strategy, storing images sequentially according to their acquisition time. Alternatively, it can be a queue not corresponding to a ZSL strategy, which can be viewed as a random storage space where images can be stored randomly.

[0157] For example, when an electronic device launches a camera application and displays the camera application's preview interface, the scene detection unit of the image processing module can be triggered to detect whether the current shooting scene is an exposure-reducing scene. If the shooting scene is an exposure-reducing scene, the exposure-reducing unit is triggered to perform exposure-reducing processing on the preview frame image based on traditional exposure-reducing logic. If there are still overexposed areas in the preview frame image after exposure-reducing processing, the exposure-reducing unit is triggered to perform exposure-reducing processing on the image using the first exposure-reducing logic when responding to the user's operation of pressing the shutter button (taking a picture). Alternatively, if the shooting scene is not an exposure-reducing scene, and there are overexposed areas in the preview frame image, the exposure-reducing unit is triggered to perform exposure-reducing processing on the image using the first exposure-reducing logic when responding to the taking a picture operation.

[0158] After reducing the exposure, the exposure reduction unit can output a captured image to the gallery. Simultaneously, the feedback unit can interact with the camera app, outputting a notification message reminding the user to enter the exposure reduction preview. (See reference...) Figure 5(c) Furthermore, the feedback unit can generate an exposure reduction strategy adjustment information (taglist), which characterizes the exposure reduction required in the shooting scene and the corresponding reduction magnitude. The exposure reduction strategy adjustment suggestion may also include a preview frame image and a target short frame image for the shooting scene. The target short frame image can be the short frame image with the smallest ghosting area (less than a target threshold). The feedback unit can transmit this tag to the scene detection unit. When the scene detection unit determines that the shooting scene for the next photo capture operation is consistent with the current shooting scene, it performs exposure reduction processing based on the reduction magnitude indicated by the tag.

[0159] Specifically, in addition to detecting whether the shooting scene is a de-exposure scene, the scene detection unit can also detect whether the shooting scenes corresponding to different shooting operations are consistent. Here, different shooting operations refer to multiple shooting operations after the electronic device launches the camera application. For example, after the electronic device launches the camera and receives the first shooting operation, it acquires and captures image 1 in response to the shooting operation. While the camera application is running continuously (either in the foreground or background, i.e., the background application has not been killed), it receives a second shooting operation. In response to this shooting operation, the scene detection unit can perform consistency verification between the shooting scene of the second shooting operation and the shooting scene of the first shooting operation. If the shooting scene is consistent, that is, no change in the shooting scene occurs between the two shooting operations, in this case, the electronic device can perform de-exposure processing on the preview frame image based on the de-exposure range indicated by the taglist; when a shooting operation is received, the corresponding de-exposure logic is used to perform de-exposure processing on the captured image according to the de-exposure range.

[0160] In this embodiment, no restrictions are placed on the detection of exposure reduction scenes or the specific algorithms for exposure reduction processing based on traditional exposure reduction strategies.

[0161] This embodiment provides an image processing method that discards short-frame images when the ghosting region is determined to be too large. This avoids the problem of image fusion failure during exposure reduction processing caused by such short-frame images, minimizing ghosting in the post-exposure reduction image and ensuring the best possible exposure reduction effect. An example is provided using a mobile phone for shooting. (See reference...) Figure 7 , Figure 7 A flowchart of an image processing method is given, combined with... Figure 8 The provided diagram of the mobile phone interface is used for illustration, including:

[0162] S101. In response to the operation of launching the camera application, the electronic device displays a preview interface on the screen.

[0163] In some embodiments, before the electronic device displays the shooting preview interface on the display screen, the electronic device also includes receiving an action from the user to launch the camera application.

[0164] In other words, after a user launches the camera app by clicking the camera app icon, or by voice or other means, the electronic device displays the camera app's launch screen. Here, the preview screen is the camera app's launch screen.

[0165] In some embodiments, the application used for taking photos may also be a third-party application. The preview interface is the shooting interface of a third-party application.

[0166] The preview interface includes preview frames of the captured image from the camera. For example, refer to... Figure 8 Image (a) shows the preview interface of the phone's camera application. The preview interface includes a preview frame image. The preview frame image is an image of the shooting scene containing a illuminated sign.

[0167] In some embodiments, the specific content of the preview interface can be found in [reference]. Figure 8 The given illustration. For example, Figure 8 As shown in (a), the preview interface may also include a shooting button, a camera switch button, a gallery button, a mode component, and a function component. The mode component includes options for photo, video, slow motion, portrait, and professional modes. The function component includes options for HDR mode (on / off button), flash (on / off button), AI photography (on / off button), and settings.

[0168] S102, The electronic device determines that there is an overexposed area in the preview frame image.

[0169] If the electronic device determines that there is an overexposed area in the preview frame image, it executes S103. If there is no overexposed area, there is no need to perform exposure reduction processing. In this case, the exposure reduction execution logic can be exited, the preview frame image can be output in normal mode, and a shooting operation can be performed in normal mode when a shooting operation is received.

[0170] Overexposed areas refer to regions in an image that are too bright and lack detail due to excessive light or prolonged exposure. Overexposed areas typically appear as excessive brightness. Electronic devices can determine overexposed areas in a preview frame image based on the brightness values ​​of individual pixels. For example, in a 14-bit image, the maximum brightness of a pixel might be 2. 14 -1 represents 16383. Based on 16383, a brightness threshold can be set, and areas formed by pixels with brightness values ​​greater than this threshold are defined as overexposed areas. For example, the maximum brightness of a pixel in an 8-bit image is 2. 8-1 equals 255. Based on 255, a brightness threshold can be set, and the area formed by pixels with brightness values ​​greater than this brightness threshold is identified as an overexposed area.

[0171] In this embodiment, the presence of overexposed areas in the preview frame image can include several scenarios. One scenario is that the preview frame image is an image that has already undergone exposure reduction; that is, even after the electronic device performs initial exposure reduction processing on the preview frame image, overexposed areas still exist, indicating insufficient exposure reduction. Another scenario is that the preview frame image is an image that has not undergone exposure reduction and still contains overexposed areas.

[0172] In this embodiment, the initial exposure reduction performed by the electronic device refers to the electronic device performing exposure reduction processing on the preview frame image based on traditional exposure reduction logic when it detects that the shooting scene is an HDR scene. In this embodiment, the specific methods for detecting HDR scenes and performing exposure reduction processing based on traditional exposure reduction logic are not limited.

[0173] For example, in one feasible approach, after launching a camera app or a third-party photography app, the electronic device triggers preview detection when displaying the preview interface. Preview detection may include the electronic device acquiring a preview frame image and determining whether it is an exposure reduction scene based on the preview frame image. If the electronic device detects an exposure reduction scene, it performs initial exposure reduction processing on the preview frame image using conventional exposure reduction logic. If the electronic device does not detect an exposure reduction scene, or detects a non-exposure reduction scene, it will not perform exposure reduction processing on the preview frame image. However, in this scenario, the preview frame image may contain overexposed areas.

[0174] Specifically, after executing S101, the process by which the electronic device determines that there are overexposed areas in the preview frame image includes:

[0175] S201. The electronic device performs a preview detection. If it is determined that the current shooting scene is a reduced exposure scene, then S202 is executed; if it is determined that the current shooting scene is not a reduced exposure scene, then S203 is executed.

[0176] In some embodiments, when the electronic device determines that the current shooting scene is an underexposed scene, the HDR mode button can also be switched from the off state (e.g., ...). Figure 8 (a) is updated to be enabled (see reference). Figure 8 (b)

[0177] S202. The electronic device uses traditional exposure reduction logic to perform initial exposure reduction processing on the preview frame image.

[0178] If overexposed areas still exist in the preview frame image after the exposure is reduced, then the conclusion of S102 is obtained, and S102 and subsequent steps are executed.

[0179] If the preview frame image after underexposure does not have overexposed areas, the electronic device can perform multi-frame image fusion based on the normally exposed image to obtain the fused captured image.

[0180] S203. If the electronic device does not perform exposure reduction processing on the preview frame image, and there are overexposed areas in the preview frame image, then the conclusion of S102 is obtained, and S102 and subsequent steps are executed.

[0181] That is, in this embodiment, regardless of whether the shooting scene of the electronic device is a de-exposure scene, as long as the electronic device detects that there are still overexposed areas in the preview frame image, it will reach the conclusion of S102 and execute the subsequent steps to perform de-exposure processing. The image processing method provided in this application embodiment can be applied to de-exposure processing of overexposed areas in de-exposure scenes or non-de-exposure scenes.

[0182] In this embodiment, whether the electronic device enters a de-exposure scene for initial de-exposure processing or does not enter a de-exposure scene and does not perform de-exposure processing, overexposed areas may still exist in the preview frame image. The electronic device can detect and identify these scenes with overexposed areas, and when it receives a user's shooting operation, it can perform further de-exposure processing to improve the de-exposure quality of the output image. This enables de-exposure processing of overexposed areas in any scene, meeting the quality requirements of users for the output image in more scenarios and optimizing the user's photography experience.

[0183] S103, In response to the shooting operation, the electronic device acquires a reference frame image and at least one candidate frame image.

[0184] In this embodiment, the electronic device can acquire multiple frames of images in response to a shooting operation. Therefore, the electronic device can determine a reference frame image and at least one candidate frame image from the multiple frames.

[0185] One of the reference frame images can be a normally exposed image. In this embodiment, the normally exposed image can serve as a reference frame in the image fusion process, and it can also be a preview frame image. Candidate frame images can include normally exposed images and abnormally exposed images, wherein abnormally exposed images can include short frame images and long frame images. An abnormally exposed image can be at least one short frame image re-captured by the electronic device after receiving a shooting operation. The exposure values ​​of at least one abnormally exposed image can be the same or different.

[0186] S104. The electronic device determines the ghosting region corresponding to the short frame image based on the normal exposure image and the short frame image.

[0187] Ghosting refers to areas of double exposure or ghosting identified based on overexposed areas in normally exposed images and overexposed areas in short-frame images. Ghosting typically manifests as unwanted, blurry, repeating images or light spots in an image.

[0188] Considering that down-exposure processing of normally exposed images requires data from overexposed areas in short-frame images and dark area data from long-frame images, the overexposed areas in short-frame images are relatively small due to their short exposure time, while the overexposed areas in normally exposed images are relatively large. Therefore, overexposed areas in a portion of the normally exposed image can be restored based on the short-frame image. However, because there may be moving objects in the shooting scene, there is a deviation in object displacement between the short-frame image and the normally exposed image. This deviation results in a blank area that cannot cover the overexposed area in the short-frame image and the area that can be restored (the recoverable area), compared to the overexposed area in the normally exposed image. This blank area is referred to as a ghosting area in this technical field. For example, refer to... Figure 9 The diagram illustrates the ghosting region. During the fusion of short-frame images and normally exposed images, the size of the ghosting region directly affects the probability of image fusion anomalies during the exposure reduction process. Therefore, setting different exposure reduction strategies based on the size of the ghosting region can improve the exposure reduction effect and increase the success rate of image fusion.

[0189] An overexposed area refers to a region comprised of pixels whose brightness value is greater than or close to the maximum brightness value of the image. For example, in a 14-bit image, the maximum brightness value of a pixel is 16383. Therefore, the largest connected region formed by pixels with a brightness value greater than or close to 16383 is an overexposed area. The brightness value being close to the maximum brightness value can be defined as the difference between the brightness value and the maximum brightness value being less than a preset value. For example, the difference between the brightness value and the maximum brightness value is less than 10. In other words, the largest connected region formed by pixels with a brightness value greater than or equal to 16373 is an overexposed area. The preset value can be a natural number greater than 1 and can be determined based on various factors such as the amount of light entering the lens and the shooting scene.

[0190] In this embodiment, the ghosting region corresponding to each short frame image can be determined based on the overexposed areas of the normally exposed image and the overexposed areas of the short frame image.

[0191] In some embodiments, if the candidate frame image includes multiple short frame images, the electronic device can calculate the ghost region corresponding to each short frame image based on the normally exposed image. In this embodiment, the ghost region corresponding to each short frame image can be determined based on various traditional ghost calculation methods.

[0192] For example, by performing histogram matching between a normally exposed image and a short-frame image, a difference region can be obtained. This difference region may contain ghosting and can be considered a ghosting region. For instance, by performing histogram matching between a short-frame image and a normally exposed image, and then performing ghosting detection using structural similarity (SSIM), a mask image of the ghosting region can be obtained.

[0193] Alternatively, after unifying the brightness of short-frame images to the brightness level of normally exposed images (performing brightness alignment), differential methods can be used to detect ghost and non-ghost regions. Brightness alignment is performed to eliminate brightness differences caused by varying exposures, thus providing a consistent brightness basis for ghost detection.

[0194] Brightness alignment can include, for example, both the normal exposure image N and the short frame image S being 14-bit images, with the maximum pixel value being 2. 14 -1 is 16383. The amount of light entering the normally exposed image N is E1, and the amount of light entering the short frame image S is E2. By obtaining the ratio of the brightness value of each pixel in the normally exposed image N to the amount of light entering E1 (N / E1), and obtaining the ratio of the brightness value of each pixel in the short frame image S to the amount of light entering E2 (S / E2), the normally exposed image N and the short frame image S are made to be within the same brightness range, thus achieving brightness alignment.

[0195] After aligning the brightness of the normally exposed image and the short-frame image, the ghosting region can be determined based on the first overexposed area in the normally exposed image and the second overexposed area in the short-frame image. Specifically, the intersection of the first and second overexposed areas can be taken; the area formed by the pixels outside the intersection is the ghosting region. For example, refer to... Figure 10 The diagrams illustrating ghosting regions corresponding to several short frame images are provided. Generally, when the reference frame image is a normally exposed image, the exposure time of the normally exposed image is longer than that of the short frame image. In the same shooting scene, the area / number of pixels of the first overexposed region in the normally exposed image is greater than the area / number of pixels of the second overexposed region in the short frame image. The difference between the first overexposed region of the normally exposed image and the second overexposed region of the short frame image in the same shooting scene constitutes the ghosting region.

[0196] After obtaining the ghosting regions corresponding to each short frame image, the electronic device can determine the corresponding exposure reduction strategy based on the size of the ghosting regions. In this embodiment, the exposure reduction strategy can also be called a fusion strategy. One way to reduce the exposure of a normally exposed image is essentially to fuse the normally exposed image with the short frame images to obtain the fused image.

[0197] In one feasible implementation method, determining the corresponding exposure reduction strategy based on the size of the ghost region can be achieved by comparing the ghost region with a preset area value to determine the exposure reduction strategy corresponding to different sizes of ghost regions. For example, if the area of ​​the ghost region is less than or equal to a first area value, a normally exposed image is used as the reference frame image, and the reference frame image is fused with candidate frame images (including abnormally exposed images and normally exposed images). Alternatively, if the area of ​​the ghost region is greater than the first area value and less than or equal to a second area value, a short frame image is used as the reference frame image, and the reference frame image is fused with candidate frame images (including abnormally exposed images and normally exposed images). Finally, if the area of ​​the ghost region is greater than the second area value, a normally exposed image is used as the reference frame image, and the reference frame image is fused with candidate frame images (including normally exposed images and remaining abnormally exposed images). Here, the first area value is less than the second area value.

[0198] Alternatively, the underexposure strategy for different sizes of ghost regions can be determined based on the proportion of the ghost region's area to the image area. For example, the calculation method could be: Area ratio s = Ghost region area S1 / First overexposed region area S2 / Image area S. Or, the calculation method could be: Area ratio s = Ghost region area S1 / Image area S.

[0199] Different ratios of ghosting areas correspond to different exposure reduction strategies. For example, if the area ratio of the ghosting area is less than or equal to the first ratio, the first exposure reduction strategy is used; if the area ratio of the ghosting area is greater than the first ratio and less than or equal to the second ratio, the second exposure reduction strategy is used; if the area ratio of the ghosting area is greater than the second ratio and less than or equal to the third ratio, the third exposure reduction strategy is used; and if the area ratio of the ghosting area is greater than the third ratio, the fourth exposure reduction strategy is used. Here, the first ratio is less than the second ratio, and the second ratio is less than the third ratio.

[0200] The first exposure reduction strategy can be to use a normally exposed image as the reference frame image and perform normal fusion processing with candidate frame images (including abnormally exposed images and normally exposed images). The second strategy can be to use the normally exposed image closest to the short frame image as the reference frame image and perform normal fusion processing with candidate frame images (including abnormally exposed images and normally exposed images). The third exposure reduction strategy can be to use a short frame image as the reference frame image and perform fusion processing with candidate frame images (including short frame images and normally exposed images). The fourth exposure reduction strategy can be to discard short frame images with a ratio greater than the third ratio, use a normally exposed image as the reference frame image, and perform fusion processing with candidate frame images (including normally exposed images and remaining abnormally exposed images).

[0201] The size of the ghosting area determines the effect of the exposure reduction process. Different exposure reduction strategies based on the size of the ghosting area can improve the exposure reduction effect.

[0202] Specifically, please refer to the following steps:

[0203] S105. If the first condition is met, the electronic device adopts the first exposure reduction strategy to perform exposure reduction processing and acquire the first captured image.

[0204] In this embodiment, meeting the first condition means that there exists at least one ghost region corresponding to a short frame image that is less than or equal to a first threshold. A ghost region being less than or equal to the first threshold may include: the area of ​​the ghost region being less than or equal to a first area value, or the area ratio of the ghost region being less than or equal to a first ratio. The judgment conditions for ghost regions are similar in the following embodiments and will not be repeated.

[0205] If a candidate frame image includes a short frame image, then meeting the first condition means that the ghosting region of the short frame image is less than or equal to the first threshold. If a candidate frame image includes multiple short frame images, then meeting the first condition means that, among these short frame images, one or more short frame images have ghosting regions less than or equal to the first threshold.

[0206] In this embodiment, a ghosting region less than or equal to a first threshold indicates the existence of a ghosting region between the normally exposed image and the short-frame image, but the area of ​​this region is very small. Based on this, the probability of ghosting in the fused image causing fusion anomalies is relatively low. Therefore, as long as the ghosting region of one short-frame image is less than or equal to the first threshold, image downexposure processing can be performed on that short-frame image to obtain the downexposure-processed first captured image. Correspondingly, the first downexposure strategy refers to maintaining the selected normally exposed image as the reference frame image, performing normal fusion processing between the reference frame image and the candidate frame image, using the overexposed areas of the short-frame image to recover overexposure from the overexposed areas of the reference frame image, and using the dark areas of the long-frame image to recover dark areas from the dark areas of the reference frame image, thus obtaining the first captured image. The candidate frame images include abnormally exposed images and normally exposed images. The image fusion method involved in this embodiment can be executed using traditional image fusion algorithms, and there are no limitations on this.

[0207] In this embodiment, if there is a short frame image in the candidate frame image whose corresponding ghost region is greater than the third threshold, the short frame image is discarded to avoid the influence of this type of short frame image on the image fusion processing.

[0208] S106. If the second condition is met, the electronic device uses the second exposure reduction strategy to perform exposure reduction processing to obtain the first captured image.

[0209] In this embodiment, meeting the second condition means that, in the absence of a ghost region corresponding to a short frame image that is less than or equal to the first threshold, there exists at least one ghost region corresponding to a short frame image that is less than or equal to the second threshold.

[0210] If a candidate frame image includes a short frame image, then the second condition is met if the ghosting region of the short frame image is greater than the first threshold and less than or equal to the second threshold. If a candidate frame image includes multiple short frame images, then the second condition is met if, among these short frame images, one or more short frame images have ghosting regions greater than the first threshold and less than or equal to the second threshold.

[0211] In this embodiment, a ghosting region greater than a first threshold and less than or equal to a second threshold indicates the existence of a ghosting region between the normally exposed image and the short-frame image, but this ghosting region is small. Based on this, the probability of the fused image exhibiting ghosting and fusion anomalies is low. Therefore, as long as the ghosting region of one short-frame image is greater than the first threshold and less than or equal to the second threshold, exposure reduction processing can be performed on that short-frame image to obtain the first captured image after exposure reduction. To further reduce the impact of the ghosting region on image fusion processing, the second exposure reduction strategy refers to using the normally exposed image closest to the target short-frame image as the reference frame image, performing normal fusion processing between the reference frame image and the candidate frame image, using the overexposed area of ​​the short-frame image to recover overexposure from the overexposed area of ​​the reference frame image, and using the dark area of ​​the long-frame image to recover dark areas from the dark area of ​​the reference frame image, to obtain the first captured image. The target short-frame image can be the short-frame image corresponding to the smallest ghosting region. Candidate frame images include abnormally exposed images and normally exposed images. The image fusion method involved in this embodiment can be executed using traditional image fusion algorithms, and there is no limitation on it.

[0212] In this embodiment, if there is a short frame image in the candidate frame image whose corresponding ghost region is greater than the third threshold, the short frame image is discarded to avoid the influence of this type of short frame image on the image fusion processing.

[0213] S107. If the third condition is met, the electronic device adopts the third exposure reduction strategy to perform exposure reduction processing and obtain the first captured image.

[0214] In this embodiment, meeting the third condition means that, in the absence of a ghost region corresponding to a short frame image that is less than or equal to the second threshold, there is at least one ghost region corresponding to a short frame image that is less than or equal to the third threshold (same as the target threshold).

[0215] If a candidate frame image includes a short frame image, then meeting the third condition means that the ghosting region of the short frame image is greater than the second threshold and less than or equal to the third threshold. If a candidate frame image includes multiple short frame images, then meeting the third condition means that among these short frame images, one or more short frame images have ghosting regions greater than the second threshold and less than or equal to the third threshold.

[0216] In this embodiment, a ghosting region greater than the second threshold and less than or equal to the third threshold indicates the existence of a ghosting region between the normally exposed image and the short-frame image that may affect the exposure reduction effect. Based on this, the probability of the fused image exhibiting ghosting and fusion anomalies due to the presence of ghosting is high. In this case, to improve the exposure reduction effect while utilizing the effective data from the overexposed area of ​​the short-frame image, it is still possible to retain the short-frame image for further exposure reduction to obtain the first image after exposure reduction. Correspondingly, the third exposure reduction strategy refers to using the target short-frame image as a reference frame image, performing dark area recovery processing on the reference frame image based on candidate frame images to restore the dark area signal-to-noise ratio of the reference frame image, thereby obtaining the first image after exposure reduction. The target short-frame image can be the short-frame image corresponding to the smallest ghosting region. Candidate frame images include other abnormally exposed images (long-frame images) and normally exposed images. The image fusion method involved in this embodiment can be executed using traditional image fusion algorithms, and there are no limitations on this.

[0217] In this embodiment, in the first captured image obtained by underexposing the target short frame image as a reference frame image, it is not necessary to restore the overexposed areas in the short frame image, but to restore the dark areas in the short frame image, thus balancing the issues of underexposing and dark area restoration.

[0218] In this embodiment, if there is a short frame image in the candidate frame image whose corresponding ghost region is greater than the third threshold, the short frame image is discarded to avoid the influence of this type of short frame image on the image fusion processing.

[0219] S108. If the fourth condition is met, the electronic device adopts the fourth exposure reduction strategy to perform exposure reduction processing and obtain the first captured image.

[0220] In this embodiment, meeting the fourth condition means that the ghosting region corresponding to all short frame images in the candidate frame portrait is greater than the third threshold. In this embodiment, a ghosting region greater than the third threshold means that there is a ghosting region between the normally exposed image and the short frame image, and this ghosting region is relatively large, which will affect the effect of image down-exposure processing. Based on this, the probability of the fused image having ghosting and fusion abnormality is very high. Therefore, when performing fusion processing of the reference frame image and the candidate frame image, it is necessary to discard short frame images with ghosting regions greater than the fourth threshold, and perform down-exposure processing based on the remaining candidate frame images and the reference frame image to obtain the first captured image after down-exposure processing. Correspondingly, the fourth down-exposure strategy refers to discarding short frame images with corresponding ghosting regions greater than the third threshold, keeping the selected normally exposed image as the reference frame image, and performing normal fusion processing of the reference frame image and the candidate frame image to obtain the first captured image. Among them, the candidate frame image includes long frame images and normally exposed images. The image fusion method involved in this embodiment can be executed using traditional image fusion algorithms, and there is no limitation on this.

[0221] In this embodiment, in the first captured image obtained by image fusion processing using a selected normally exposed image as a reference frame image after discarding short frame images with corresponding ghosting regions greater than the third threshold, since there are no effective short frame images for exposure reduction processing, the overexposed areas in the first captured image may still have overexposure problems.

[0222] Based on S105-S108 above, it can be understood that if any short frame image in the candidate frame image contains a ghost region larger than the third threshold, that short frame image is discarded. If, after discarding the discarded short frame image, there are still remaining short frame images in the candidate frame image, and there are short frame images with ghost regions smaller than the first threshold, then the first exposure reduction strategy is used for exposure reduction processing. If none of the remaining short frame images have ghost regions smaller than the first threshold, but there are short frame images with ghost regions smaller than the second threshold, then the second exposure reduction strategy is used for exposure reduction processing. If none of the remaining short frame images have ghost regions smaller than the second threshold, but there are short frame images with ghost regions smaller than the third threshold, then the third exposure reduction strategy is used for exposure reduction processing. If, after discarding short frame images larger than the third threshold, there are no remaining short frame images in the candidate frame image, then the fourth exposure reduction strategy is used for exposure reduction processing.

[0223] Different exposure reduction strategies are used based on the size of the ghost area to reduce exposure. This can effectively take into account the impact of the ghost area on the image fusion processing. In different exposure reduction strategies, the influence of the ghost area is reduced to achieve a balance between overexposure recovery and image quality, so that the first captured image can maintain a good exposure reduction effect as much as possible.

[0224] S109. The electronic device outputs a reminder message on the display screen.

[0225] In some embodiments, after the electronic device performs exposure reduction processing on the captured image based on the aforementioned exposure reduction strategy, a first captured image is obtained. The electronic device can store the first captured image in a gallery. Simultaneously, to remind the user that the electronic device has enabled the exposure reduction processing logic, while the electronic device is still displaying the preview interface, it can output a reminder message in the preview interface. This reminder message is used to inform the user that the electronic device has enabled the exposure reduction strategy and will perform exposure reduction processing on both the preview frame image and the captured image; the preview frame image in the preview interface at this time is the effect after exposure reduction processing. For example, refer to... Figure 8 (c) The electronic device displays a reminder message on the preview interface, which reads "Underexposure preview has been enabled".

[0226] In some embodiments, the electronic device may display a gallery interface in response to a user's action on the gallery button. (See reference) Figure 8 (d) The electronic device displays a gallery interface on the screen. The gallery interface can display images taken at the most recent time based on the shooting time by default. Figure 8 The image shown in (d) is the first image after the exposure was reduced.

[0227] If, after storing the first captured image to the gallery, the electronic device detects that the currently displayed interface is not the preview interface—meaning the electronic device may have switched to displaying another interface in response to the user's interface switching action—then, since the displayed interface may be the interface of another application or the electronic device's system interface, which are unrelated to the preview interface, the electronic device may not output a notification message to reduce the user's perception of the notification and avoid unnecessary ambiguity.

[0228] In this embodiment, using short frames as reference frames may result in poor dark area recovery, and the output image may have smearing and color noise. To avoid disappointing users with the final image, a reminder message can be output to remind users that an exposure reduction preview will be performed, allowing users to make a preliminary judgment on the imaging effect.

[0229] Steps S101-S109 above describe how, after an electronic device launches a camera or other shooting application, it displays a preview interface. Upon detecting an overexposed area in the preview frame image, and upon receiving the first shooting operation, it employs a corresponding exposure reduction strategy based on the size of the ghosting area to perform exposure reduction processing. In other words, this embodiment is applicable to both the preview detection scenario after an electronic device launches a camera application and the exposure reduction processing scenario in response to the first shooting operation after launching the camera application.

[0230] In some embodiments, to improve the efficiency of exposure reduction processing by the electronic device, after performing exposure reduction processing, the electronic device can also generate a taglist of exposure reduction strategies corresponding to the current shooting scene. For example, the taglist of the exposure reduction strategy corresponding to the current shooting scene is a first value. The first value can be any pre-defined numerical value. In some embodiments, the taglist can indicate a preview frame image or a captured image corresponding to the shooting scene, and it can also indicate a valid short frame image and an exposure reduction strategy corresponding to the shooting scene. The electronic device can store the taglist in a designated storage space. Alternatively, after performing exposure reduction processing, the electronic device can also generate exposure reduction strategy adjustment information corresponding to the current shooting scene. For example, the exposure reduction strategy adjustment information can include a preview frame image or a captured image corresponding to the shooting scene, and it can also include the exposure value of a valid short frame image corresponding to the shooting scene. The electronic device can store the exposure reduction strategy adjustment information in a designated storage space.

[0231] After the electronic device responds to the shooting operation and outputs the first captured image, the electronic device displays a preview interface. If the shooting scene corresponding to the preview interface is consistent with the shooting scene corresponding to the shooting operation, the exposure reduction strategy and / or effective short frame images corresponding to the shooting scene can be used to perform exposure reduction processing for the preview of the shooting scene and exposure reduction processing for the first captured image, which can further improve the efficiency of exposure reduction processing.

[0232] If the shooting scene corresponding to the preview interface is the same as the shooting scene corresponding to the shooting operation, it means that the shooting scene has not changed. The electronic device can still use the short frame image acquired in the above-mentioned exposure reduction processing to perform preview exposure reduction processing and / or respond to the shooting operation to perform exposure reduction processing on the captured image, thereby improving the efficiency of the electronic device's exposure reduction processing. If the shooting scene corresponding to the preview interface is inconsistent with the shooting scene corresponding to the shooting operation, that is, if the current shooting scene has changed when the electronic device displays the preview interface after outputting the first captured image, in this case, the electronic device can re-execute S101-S109 to perform exposure reduction processing on the preview frame image and the captured image.

[0233] Situations where the shooting scene may not have changed include: When the camera app is running continuously, the shooting scene may remain unchanged between the first and second shooting operations; or, after the first shooting operation, the user switches to the gallery to view the captured images, then switches back to the camera app to perform a second shooting operation, and the shooting scene may remain unchanged between these two operations. In these cases where the shooting scene may not have changed, the electronic device can determine whether the shooting scene after the first shooting operation is consistent with the shooting scene before the first shooting operation by detecting scene consistency when displaying the preview interface. This allows it to determine whether the captured short-frame image can be used for underexposure processing. In other words, as long as the camera app remains running in the foreground or background and is not killed, the electronic device can perform a shooting scene consistency check when displaying the preview interface.

[0234] If the camera app is killed and then restarted to display the preview interface, it is assumed that the shooting scene has likely changed. In this case, the electronic device does not perform the shooting scene consistency judgment, and the shooting count is also reset to 0.

[0235] In scenarios where the electronic device displays a preview interface after the first shot following the initial camera application launch, or when the electronic device switches the camera application from the background to the foreground and displays a preview interface, the process of the electronic device performing underexposure processing for the Nth shot using multiple short-frame images can be found in [reference needed]. Figure 11 A flowchart illustrating another image processing method is provided, including:

[0236] S301. When the electronic device displays the preview interface, a consistency check of the shooting scene is performed.

[0237] In this embodiment, when the electronic device re-displays the preview interface after completing a shot in response to a shooting operation, it can perform a consistency check of the shooting scene. This can be understood as follows: if the preview interface is displayed when the camera application is launched, the electronic device may not perform a consistency check of the shooting scene.

[0238] The electronic device performs a shooting scene consistency check by retrieving the first preview frame image or captured image of the first shooting scene indicated by the last stored identifier, and the second preview frame image of the current shooting scene (second shooting scene), according to the storage time of the identifier. Based on the first and second preview frame images, image consistency is judged. If the similarity between the two images is greater than or equal to a preset similarity value, the shooting scenes are considered consistent; if the similarity is less than the preset similarity value, the shooting scenes are considered inconsistent. The similarity between the two images can be obtained through image feature extraction and calculated based on image feature vectors. Alternatively, the first and second preview frame images can be input into a preset similarity model to calculate the similarity between the two images. This embodiment does not limit the specific similarity calculation method.

[0239] If the second shooting scene is different from the first shooting scene, then the exposure reduction process is re-performed based on the preview frame image, that is, the above S102 and S201-S203 are executed.

[0240] S302. If the second shooting scene is the same as the first shooting scene, then perform exposure reduction processing based on the exposure reduction strategy corresponding to the first shooting scene.

[0241] In some embodiments, the identifier of the first shooting scene indicates a valid short-frame image and a corresponding exposure reduction strategy. The electronic device can perform exposure reduction processing on the preview frame image according to the exposure reduction strategy. For example, the exposure reduction strategy indicated by the identifier is the first exposure reduction strategy, where the valid short-frame image is image 1. The electronic device can use a preview frame image as a reference frame image, and use image 1 and other normally exposed images and long-frame images acquired in the current shooting scene as candidate frame images, and perform exposure reduction processing to obtain the exposure-reduced preview frame image. The exposure-reduced preview frame image is displayed in the preview interface.

[0242] When a shooting operation is received, the electronic device can still use the corresponding exposure reduction strategy to reduce the exposure of the captured image, thereby obtaining a second captured image, as described in S103-S108 above.

[0243] To further improve the effect of exposure reduction processing of electronic devices, among other feasible methods, electronic devices can also determine the exposure reduction strategy for the current shooting scene based on the exposure value of the valid short frame image indicated by the identifier.

[0244] S303. The electronic device determines whether the exposure value of the target short frame image is less than the preset exposure value.

[0245] In this embodiment, the target short frame image is the valid short frame image indicated by the identifier. The target short frame image is also the short frame image with the smallest corresponding ghosting area. The electronic device acquires the exposure value (EV) of the target short frame image. The exposure value of the target short frame image also signifies the degree of exposure reduction in the exposure reduction process. The electronic device can also determine the degree of exposure reduction for the preview frame image of the current shooting scene based on the exposure value of the target short frame image to ensure the exposure reduction effect of the preview frame image of the current shooting scene.

[0246] If the exposure value is less than the preset exposure value, the preview frame image is down-exposed using the first target exposure value. If the exposure value of the target short frame image is relatively low, the exposure value of the target short frame image can be used for down-exposing processing. In this case, the first target exposure value is the exposure value of the target short frame image. For example, the preset exposure value can be 2EV. The preset exposure value can be determined according to the actual situation, and this embodiment does not limit it.

[0247] If the exposure value is greater than or equal to the preset exposure value, the preview frame image is down-exposed using the second target exposure value. If the target short frame image has a relatively high exposure value, directly using its exposure value for down-exposing might result in excessive down-exposing and an overly dark preview frame image. In this case, the second target exposure value can be the preset exposure value.

[0248] By reducing the exposure of the preview frame image based on a first target exposure value or a second target exposure value, the exposure value of the entire preview frame image can be directly reduced by the first target exposure value or the second target exposure value. For example, if the first target exposure value is 2EV, the electronic device can directly reduce the exposure value of each pixel in the entire preview frame image by 2EV, achieving rapid exposure reduction of the preview frame image.

[0249] In scenarios where the exposure value is lower than the preset exposure value, if the electronic device receives a shooting command, it will execute:

[0250] S304. When the electronic device responds to the shooting operation, it performs underexposure processing using the first frame output strategy to obtain the second captured image.

[0251] In this approach, since the exposure value of a short frame image is less than the preset exposure value, it means that the exposure value of the short frame image is relatively close to that of a normally exposed image. Even short frame images that are far from the reference frame image can be used as candidate frame images for image fusion processing. In this case, the electronic device can employ a ZSL (Zero-Side Frame Output) strategy, using the brightness of the preview frame image as the brightness of the reference frame image to obtain a first target reference frame image. Based on the first target reference frame image and a preset number of candidate frame images, exposure reduction processing is performed to obtain the second captured image. In the ZSL strategy, when the electronic device responds to the shooting operation, it regenerates the short frame image and the long frame image, and stores them sequentially in a buffer queue that already stores normally exposed images. Figure 12 (a) provides a schematic diagram of the buffer queue in a ZSL frame output strategy.

[0252] Specifically, the electronic device uses the ZSL buffer queue to retrieve an image with the same brightness as the preview frame image as a reference frame image. According to the image fusion principle, following the order of the image frames in the buffer queue, a predetermined number of images that are close to the reference frame image are sequentially retrieved and fused to obtain the second captured image. When the electronic device receives a shooting operation, it does not regenerate short-frame or long-frame images; in this case, the images retrieved from the buffer queue include normally exposed images. The electronic device then performs image fusion processing based on multiple normally exposed images to obtain the second captured image.

[0253] In HDR scenarios, when an electronic device receives a shooting command, it regenerates short-frame and long-frame images. In this case, according to the image fusion principle, the electronic device sequentially acquires a first number of normally exposed images that are close to the reference frame image, according to the order of the image frames in the buffer queue. Then, it acquires a second number of short-frame and long-frame images from the buffer queue, forming a preset number of images for image fusion processing to obtain the second captured image.

[0254] The preset quantity is the number of images required for image fusion processing, and this preset quantity can be set in advance according to the device performance.

[0255] In scenarios where the exposure value is greater than or equal to the preset exposure value, if the electronic device receives a shooting command, it will execute:

[0256] S305. When the electronic device responds to the shooting operation, it performs exposure reduction processing using the second frame output strategy to obtain the second captured image.

[0257] In this case, since the exposure value of the short frame image is greater than or equal to the preset exposure value, it means that the exposure value difference between the short frame image and the normally exposed image is significant. Therefore, the short frame image obtained by the electronic device using the ZSL (Zero-Screen Latency) frame output strategy may differ significantly from the normally exposed image, potentially resulting in a large ghosting area. Thus, in this embodiment, the electronic device uses a non-ZSL frame output strategy to perform exposure reduction processing to obtain the second captured image. In the non-ZSL frame output strategy, normally exposed images, short frame images, and long frame images are generated alternately. (Reference) Figure 12 (b) provides a schematic diagram of a buffer queue in a non-ZSL frame output strategy.

[0258] Because the exposure values ​​of short-frame images and normally exposed images differ significantly, directly using the brightness of the normally exposed image (preview frame image) as the brightness of the reference frame image may result in excessive underexposure. In the underexposure processing of this embodiment, the electronic device can use the difference between the exposure value of the short-frame image and a preset exposure value as the target brightness of the reference frame image to obtain a second target reference frame image. Based on the second target reference frame image and a preset number of candidate frame images, underexposure processing is performed to obtain the second captured image. The reference frame image with the target brightness is obtained.

[0259] Specifically, the electronic device acquires an image with the same brightness as the target brightness as a reference frame image, acquires a first number of normally exposed images, and then acquires a second number of short and long frame images to form a preset number of images for image fusion processing to obtain the second captured image. Since the short frame images and normally exposed images are close in distance in the non-ZSL frame output strategy (i.e., their generation times are close), the displacement difference of moving objects in the short frame images and normally exposed images is relatively small. Based on this strategy, image fusion can minimize the impact of ghosting regions caused by the displacement difference of moving objects on image fusion, thereby improving the effect of the down-exposure processing.

[0260] In this embodiment, after performing exposure reduction processing, the electronic device can generate an exposure reduction strategy identifier corresponding to the shooting scene to guide exposure reduction processing under the same shooting scene. If the shooting scene corresponding to the preview interface is the same as the shooting scene corresponding to the shooting operation, the exposure reduction strategy corresponding to that shooting scene and / or an effective short frame image can be used to perform preview exposure reduction processing and second shooting image exposure reduction processing for that shooting scene. The exposure reduction strategy under the same scene can be applied to the next preview exposure reduction processing and shooting image exposure reduction processing, which can further improve the efficiency and effect of exposure reduction processing.

[0261] Figure 13 A possible structural schematic diagram of the electronic device involved in the above embodiments is shown. Figure 13The electronic device 1300 shown includes a processing module 1301, an image acquisition module 1302, a display module 1303, and a storage module 1304.

[0262] The processing module 1301 may be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The processor may include an application processor and a baseband processor. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0263] For example, the processing module 1301 can be as follows: Figure 4 The processor 110 shown; the image acquisition module 1302 can be as follows: Figure 4 The camera 193 shown; the display module 1303 can be as follows Figure 4 The display screen 194 shown; the storage module 1304 can be as follows: Figure 4 The internal memory 121 shown. The electronic device provided in this application embodiment can be Figure 4 The electronic device 100 shown.

[0264] This application also provides a chip system (e.g., a system-on-a-chip (SoC)). Figure 14 As shown, the chip system includes at least one processor 1401 and at least one interface circuit 1402. The processor 1401 and the interface circuit 1402 are interconnected via lines. For example, the interface circuit 1402 can be used to receive signals from other devices (e.g., the memory of an electronic device). As another example, the interface circuit 1402 can be used to send signals to other devices (e.g., the processor 1401 or the camera of an electronic device). Exemplarily, the interface circuit 1402 can read instructions stored in the memory and send those instructions to the processor 1401. When the instructions are executed by the processor 1401, the electronic device can perform the steps in the above embodiments. Of course, the chip system may also include other discrete components, which are not specifically limited in this application embodiment.

[0265] This application also provides a computer-readable storage medium including computer instructions that, when executed on the electronic device, cause the electronic device to perform various functions or steps performed by the electronic device 100 in the above method embodiment.

[0266] This application also provides a computer program product that, when run on a computer, causes the computer to perform the various functions or steps performed by the electronic device 100 in the above method embodiments. For example, the computer may be the aforementioned electronic device 100.

[0267] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0268] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0269] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0270] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0271] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0272] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, Applied to electronic devices, the method includes: The electronic device responds to the launch operation of the camera application by displaying a preview interface of the shooting scene; the preview interface includes preview frame images; If the electronic device detects an overexposed area in the preview frame image, when the electronic device receives a shooting operation triggered by the user, the electronic device acquires a reference frame image and at least one candidate frame image; wherein, the reference frame image is a normal exposure image with the same exposure duration as the preview frame image, and the candidate frame image includes at least one short frame image with an exposure duration shorter than that of the preview frame image; The electronic device determines the ghosting region corresponding to the short frame image based on the normal exposure image and the short frame image; The electronic device employs an exposure reduction strategy corresponding to the size of the ghost region to perform image fusion processing on images with different exposure values ​​in the reference frame image and the candidate frame image to obtain a first captured image; wherein, if the area of ​​the ghost region corresponding to the short frame image is greater than a target threshold, the electronic device discards the short frame image.

2. The method according to claim 1, characterized in that, After acquiring the first captured image, the method further includes: The electronic device outputs a reminder message, which is used to remind users that the preview of the exposure reduction effect has been enabled.

3. The method according to claim 1, characterized in that, The exposure reduction strategy includes: Using the normally exposed image as the reference frame image, image fusion processing is performed on the reference frame image and the at least one candidate frame image; or, The reference frame image is the normally exposed image whose acquisition time is closest to that of the short frame image with the smallest ghost area. Image fusion processing is then performed on the reference frame image and the at least one candidate frame image. or, The short frame image with the smallest ghost region area is used as the reference frame image, and image fusion processing is performed on the reference frame image and the at least one candidate frame image.

4. The method according to claim 3, characterized in that, When the candidate frame image includes multiple short frame images, the electronic device employs an exposure reduction strategy corresponding to the area size of the ghost region to perform image fusion processing on the reference frame image and the at least one candidate frame image to obtain a first captured image, including: If the area of ​​the ghost region corresponding to at least one short frame image is less than or equal to the first threshold, the electronic device uses the normally exposed image as the reference frame image and performs image fusion processing on the reference frame image and the candidate frame image to obtain the first captured image. The candidate frame image is a short frame image in which the area of ​​the ghost region is greater than the target threshold.

5. The method according to claim 3, characterized in that, When the candidate frame image includes multiple short frame images, the electronic device employs an exposure reduction strategy corresponding to the area size of the ghost region to perform image fusion processing on the reference frame image and the at least one candidate frame image to obtain a first captured image, including: If the area of ​​the ghost region corresponding to all the short frame images is greater than the first threshold, and at least one of the short frame images has a ghost region area less than or equal to the second threshold, the electronic device determines the normal exposure image with the acquisition time closest to the first target short frame image as the reference frame image, and performs image fusion processing on the reference frame image and the candidate frame image to obtain the first captured image. The first target short frame image is the short frame image with the smallest area of ​​the corresponding ghost region among the plurality of short frame images, and the candidate frame images do not contain short frame images with a ghost region area greater than the target threshold.

6. The method according to claim 3, characterized in that, When the candidate frame image includes multiple short frame images, the electronic device employs an exposure reduction strategy corresponding to the area size of the ghost region to perform image fusion processing on the reference frame image and the at least one candidate frame image to obtain a first captured image, including: If the area of ​​the ghost region corresponding to all the short frame images is greater than the second threshold, and at least one of the short frame images has a ghost region area smaller than the target threshold, the electronic device uses the second target short frame image as the reference frame image and performs image fusion processing on the reference frame image and the candidate frame image to obtain the first captured image. The second target short frame image is the short frame image with the smallest area of ​​the corresponding ghost region among the plurality of short frame images, and the candidate frame images do not contain short frame images with a ghost region area greater than the target threshold.

7. The method according to claim 3, characterized in that, When the candidate frame image includes multiple short frame images, the electronic device employs an exposure reduction strategy corresponding to the area size of the ghost region to perform image fusion processing on the reference frame image and the at least one candidate frame image to obtain a first captured image, including: If the area of ​​the ghost region corresponding to all the short frame images is greater than the target threshold, the electronic device discards all the short frame images, uses the normally exposed image as the reference frame image, and performs image fusion processing on the reference frame image and the remaining candidate frame images to obtain the first captured image. The remaining candidate frame images include normally exposed images and / or long frame images with an exposure duration longer than that of the preview frame images.

8. The method according to any one of claims 1-7, characterized in that, After acquiring the first captured image, the method further includes: The electronic device generates and stores exposure reduction strategy adjustment information corresponding to the shooting scene, the exposure reduction strategy adjustment information including the exposure values ​​of the first preview frame image and the target short frame image of the shooting scene; The target short frame image includes the short frame image with the smallest corresponding ghosting region among the candidate frame images.

9. The method according to any one of claims 1-7, characterized in that, The method further includes: When the preview interface is not displayed for the first time after the electronic device launches the camera application, the electronic device acquires a second preview frame image of the current shooting scene; The electronic device performs a shooting scene consistency check based on the second preview frame image of the current shooting scene and the first preview frame image; When the current shooting scene is consistent with the shooting scene indicated by the exposure reduction strategy adjustment information, the electronic device performs preview exposure reduction processing based on the exposure value of the target short frame image, and displays the preview frame image after exposure reduction processing on the preview interface; When the electronic device receives a shooting operation triggered by the user, it performs exposure reduction processing using an exposure reduction strategy corresponding to the preview exposure reduction processing to obtain the second captured image.

10. The method according to claim 9, characterized in that, The electronic device performs preview downexposure processing based on the exposure value of the target short frame image, and displays the downexposure-processed preview frame image on the preview interface, including: If the exposure value of the target short frame image is less than the preset exposure value, the electronic device uses the exposure value of the target short frame image as the adjustment range for reducing the exposure, reduces the exposure value of the preview frame image in the current shooting scene, and displays the preview frame image after reducing the exposure value on the preview interface.

11. The method according to claim 9 or 10, characterized in that, When the electronic device receives a user-triggered shooting operation, it performs exposure reduction processing using an exposure reduction strategy corresponding to the preview exposure reduction processing to obtain the second captured image, including: When the electronic device receives a shooting operation triggered by the user, the electronic device adopts a zero-delay shooting frame output strategy and obtains a first target reference frame image from the buffer queue corresponding to the zero-delay shooting strategy; the buffer queue corresponding to the zero-delay shooting strategy includes a pre-generated normal exposure image and an abnormal exposure image generated in response to the shooting operation; Based on the distance between each image in the cache queue and the first target reference frame image, a preset number of normally exposed and abnormally exposed images that are closest to each other are obtained from the cache queue and subjected to downexposure processing to obtain the second captured image; The first target reference frame image is an image with the same brightness as the preview frame image in the preview interface.

12. The method according to claim 9, characterized in that, The electronic device performs preview downexposure processing based on the exposure value of the target short frame image, and displays the downexposure-processed preview frame image on the preview interface, including: If the exposure value of the target short frame image is greater than or equal to the preset exposure value, the electronic device uses the preset exposure value as the adjustment range for reducing the exposure, reduces the exposure value of the preview frame image in the current shooting scene, and displays the preview frame image after reducing the exposure value on the preview interface.

13. The method according to claim 9 or 12, characterized in that, When the electronic device receives a user-triggered shooting operation, it performs exposure reduction processing using an exposure reduction strategy corresponding to the preview exposure reduction processing to obtain the second captured image, including: When the electronic device receives a shooting operation triggered by the user, the electronic device adopts a non-zero delay shooting strategy to alternately generate an image sequence containing normally exposed images and abnormally exposed images; The electronic device acquires a second target reference frame image from the image sequence, and performs down-exposure processing on a preset number of normally exposed images and abnormally exposed images that are closest to the second target reference frame image to obtain the second captured image; The brightness of the second target reference frame image is equal to the difference between the preset exposure value and the exposure value of the target short frame image.

14. The method according to any one of claims 1-13, characterized in that, After displaying the preview interface, the method further includes: The electronic device detects whether the shooting scene is a high dynamic range (HDR) scene; If the electronic device detects an overexposed area in the preview frame image, upon receiving a user-triggered shooting operation, the electronic device acquires a reference frame image and at least one candidate frame image, including: When the electronic device detects that the shooting scene is a high dynamic range (HDR) scene, after the electronic device performs exposure reduction processing using a preset exposure reduction logic, if the electronic device detects that there is an overexposed area in the preview frame image, when the electronic device receives a shooting operation triggered by the user, the electronic device acquires a reference frame image and at least one candidate frame image.

15. The method according to claim 14, characterized in that, If the electronic device detects an overexposed area in the preview frame image, upon receiving a user-triggered shooting operation, the electronic device acquires a reference frame image and at least one candidate frame image, further comprising: If the electronic device detects that the shooting scene is a non-high dynamic range (HDR) scene, and the electronic device detects that there is an overexposed area in the preview frame image, when the electronic device receives a shooting operation triggered by the user, the electronic device acquires a reference frame image and at least one candidate frame image.

16. An electronic device comprising a display screen, a camera, a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-15.

17. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1-15.

18. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1-15.

Images