Image processing method, shooting method, storage medium, device and program product

Abstract

The invention relates to the technical field of image processing, and discloses an image processing method, a shooting method, a storage medium, equipment and a program product, electronic equipment collects a collection scene to obtain multiple frames of to-be-fused images, and when the judgment result shows that the multiple frames of to-be-fused images comprise a first shooting element of which the position changes in different to-be-fused images, a second shooting element of which the position changes in the different to-be-fused images, and a third shooting element of which the position changes in the different to-be-fused images; when the brightness of the first shooting element meets the first brightness condition and the brightness of the second shooting element meets the first brightness condition, the electronic equipment improves the brightness of the dark frame with the low exposure value in the multiple frames of images to be fused, so that the brightened dark frame serves as an output image. Due to the fact that the exposure value of the dark frame is low, the brightness of the second shooting element with the high brightness in the dark frame can be reduced, overexposure cannot occur, and the abnormal phenomenon of overexposure of the output image can be avoided; moreover, the electronic equipment does not perform fusion processing on the multiple frames of images to be fused, but takes the brightened dark frame as the output image, so that the abnormal phenomenon of ghosting of the output image can be avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image processing method, a photographing method, a storage medium, an apparatus and a program product. BACKGROUND

[0002] When a user uses an electronic device such as a mobile phone to take a picture, the electronic device usually collects multiple frames of to-be-fused images (hereinafter referred to as multiple frames of to-be-fused images) with different exposure values (EV), and then fuses the multiple frames of to-be-fused images to generate an image with a higher dynamic range (DR).

[0003] However, in the process of collecting the multiple frames of to-be-fused images by the electronic device, if there is a region with a higher brightness (for example, the photographed scene includes an electronic display screen with a higher brightness, hereinafter referred to as a highlight region) and a region with a changed position (for example, the position of the electronic display screen included in the photographed scene changes, hereinafter referred to as a changed region) in the photographed scene, the electronic device may have abnormal phenomena such as ghosting and overexposure when fusing the multiple frames of to-be-fused images. SUMMARY

[0004] The present application provides an image processing method, a photographing method, a storage medium, an apparatus and a program product.

[0005] In a first aspect, the present application provides an image processing method applied to an electronic device, the method comprising: acquiring multiple frames of to-be-fused images, wherein the multiple frames of to-be-fused images include the same photographed elements; determining that the multiple frames of to-be-fused images include a first region and a second region, wherein the first region includes a first photographed element with a changed position in different to-be-fused images, the second region includes a second photographed element with a brightness satisfying a first brightness condition, and the first photographed element and the second photographed element are the same or different; selecting a first image from the multiple frames of to-be-fused images, and increasing the brightness of the first image to obtain a first target image, wherein the exposure value of the first image is less than the exposure value of a second image in the multiple frames of to-be-fused images.

[0006] In the embodiment of the present application, the electronic device collects a plurality of frames of to-be-fused images in a collection scene, and when it is determined that the plurality of frames of to-be-fused images include a first shooting element whose position changes in different to-be-fused images and a second shooting element whose brightness meets a first brightness condition, the electronic device increases the brightness of a dark frame with a lower exposure value in the plurality of frames of to-be-fused images, and takes the brightened dark frame as a first target image. Since the exposure value of the dark frame is lower, the brightness of the second shooting element with higher brightness will be reduced in the dark frame without overexposure, so that the abnormal phenomenon of overexposure of the first target image can be avoided. Moreover, since the electronic device does not perform fusion processing on the plurality of frames of to-be-fused images, but takes the brightened dark frame as the first target image, the abnormal phenomenon of ghosting of the first target image can be avoided.

[0007] In a possible implementation of the first aspect, the first brightness condition includes at least one of the following: the brightness of each pixel in the second region is greater than or equal to a first brightness threshold; and the proportion of pixels in the second region with brightness greater than or equal to the first brightness threshold is greater than a first proportion.

[0008] In a possible implementation of the first aspect, selecting a first image from the plurality of frames of to-be-fused images and increasing the brightness of the first image to obtain a first target image includes: selecting a to-be-fused image with the lowest exposure value from the plurality of frames of to-be-fused images as the first image; increasing the brightness of the first image to obtain a first intermediate image; performing morphing alignment processing on a second image based on the first image to obtain a second intermediate image, wherein each pixel in the second intermediate image corresponds to each pixel at a corresponding position in the first image; and performing noise reduction processing on the first intermediate image based on the second intermediate image to obtain the first target image.

[0009] In a possible implementation of the first aspect, performing noise reduction processing on the first intermediate image based on the second intermediate image to obtain the first target image includes: performing noise reduction processing on the first intermediate image based on pixel information of the second intermediate image to obtain a third intermediate image, wherein the pixel information includes a pixel value and / or brightness; and performing tone mapping processing on the third intermediate image to obtain the first target image, wherein the bit depth of the first target image is less than the bit depth of the third intermediate image.

[0010] In a possible implementation of the first aspect, selecting a first image from the plurality of frames of to-be-fused images and increasing the brightness of the first image to obtain a first target image includes: when the size of the first region is greater than or equal to a first region threshold and the region feature of the second region meets a first feature condition, selecting the first image from the plurality of frames of to-be-fused images and increasing the brightness of the first image to obtain the first target image, wherein the first feature condition includes that the size of the second region is greater than or equal to a second region threshold, and / or the region type of the second region is a preset type.

[0011] In a possible implementation of the first aspect, the method further includes: performing fusion processing on the plurality of frames of to-be-fused images to obtain a second target image, in response to a size of a region corresponding to the first region being less than a first region threshold.

[0012] In a possible implementation of the first aspect, the method further includes: obtaining a third target image based on the second image, in response to a region feature corresponding to the second region not satisfying a first feature condition.

[0013] In a second aspect, an embodiment of the present application provides a photographing method applied to an electronic device, including: detecting a photographing instruction of a user, and collecting a plurality of frames of to-be-fused images; performing processing on the plurality of frames of to-be-fused images based on the image processing method provided in the first aspect to obtain a processing result image, and displaying the processing result image.

[0014] In a third aspect, an embodiment of the present application provides a computer readable storage medium, which stores instructions. When the instructions are executed on an electronic device, the electronic device implements any one of the image processing methods provided in the first aspect and possible implementations of the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, which includes: a memory configured to store instructions executed by one or more processors of the electronic device; and a processor, which is one of the processors of the electronic device, and is configured to execute the instructions stored in the memory to implement any one of the image processing methods provided in the first aspect and possible implementations of the first aspect.

[0016] In a fifth aspect, an embodiment of the present application provides a program product, which includes instructions. When the instructions are executed on an electronic device, the electronic device can implement any one of the image processing methods provided in the first aspect and possible implementations of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 According to some embodiments of the present application, a schematic diagram of a photographing scene 10 is shown;

[0018] Figure 2 According to some embodiments of the present application, an example schematic diagram of an image 20 is shown;

[0019] Figure 3 According to some embodiments of the present application, an example schematic diagram of an image 30 is shown;

[0020] Figure 4 According to some embodiments of the present application, an example schematic diagram of an image 40 is shown;

[0021] Figure 5 According to some embodiments of the present invention, a schematic diagram of a data processing method is shown;

[0022] Figure 6 According to some embodiments of the present invention, an example schematic diagram of brightening a dark frame is shown;

[0023] Figure 7 According to some embodiments of the present invention, a schematic diagram of another data processing method is shown;

[0024] Figure 8 According to some embodiments of the present invention, a flowchart of a data processing method is shown;

[0025] Figure 9 According to some embodiments of the present invention, an example schematic diagram of a mobile phone 100 is shown. Detailed Implementation

[0026] The illustrative embodiments of the present invention include, but are not limited to, an image processing method, an image capturing method, a storage medium, an apparatus, and a program product.

[0027] The following is in conjunction with the appendix Figures 1 to 9 The technical solution of the present invention will be described.

[0028] Figure 1 According to some embodiments of this application, a schematic diagram of a shooting scene 10 is shown.

[0029] In some embodiments, such as Figure 1 As shown, a user can use an electronic device, such as a mobile phone 100, to take a picture of, for example, a friend. For example, Figure 1 The shooting scene 10 corresponding to the mobile phone 100 shown is a scene where a friend holds an electronic display screen and displays, for example, a photo through the electronic display screen. In shooting scene 10, the display screen has a high brightness, that is, the area where the electronic display screen is located can be regarded as the highlight area of ​​shooting scene 10; at the same time, due to the movement of the person's hand (such as up and down or left and right shaking, not shown in the figure), the position of the electronic display screen changes accordingly with the movement of the person's hand, that is, the area where the person's hand and the electronic display screen are located can be regarded as the changing area of ​​shooting scene 10.

[0030] See also Figure 1As shown, during the process that the user uses the mobile phone 100 to take a photo of a friend, when the user clicks the button 100A of the mobile phone 100, the mobile phone 100 usually collects multiple frames of to-be-fused images of the shooting scene 10 through the camera of the mobile phone 100, and then fuses the multiple frames of to-be-fused images to generate a shooting image with a higher dynamic range, for example, a high dynamic range (HDR) image. It can be understood that the multiple frames of to-be-fused images collected by the mobile phone 100 of the shooting scene 10 include the same shooting elements. For example, for the shooting scene 10, the shooting elements that can be collected by the mobile phone 100 at least include the shooting elements of a person, an electronic display screen, and the like, that is, the multiple frames of to-be-fused images collected by the mobile phone 100 all include the shooting elements of the person, the electronic display screen, and the like in the shooting scene 10.

[0031] It can be understood that the shooting elements can be divided according to the people and objects in the image, or can be divided according to the foreground and background in the image, or can be divided according to the pixel positions in the image, and the specific division is not limited.

[0032] It can be understood that the shooting elements can be divided according to the people and objects in the image, or can be divided according to the foreground and background in the image, or can be divided according to the pixel positions in the image, and the specific division is not limited. Figure 1 The mobile phone 100 shown is only an example of an electronic device in the embodiments of the present application; in other embodiments, the electronic device can also be a wearable device, a tablet computer, or any device with an image processing function.

[0033] It can be understood that the mobile phone 100 shown is only an example of an electronic device in the embodiments of the present application; in other embodiments, the electronic device can also be a wearable device, a tablet computer, or any device with an image processing function. Figure 1 The example that the mobile phone 100 collects multiple frames of to-be-fused images through a photographing function in the shooting scene 10 to generate a shooting image is only an example of an application scenario in the embodiments of the present application; in other embodiments, the electronic device can also collect multiple frames of to-be-fused images in a shooting scene through a video recording function to generate a cover image of a corresponding video file, or the electronic device can also collect multiple frames of to-be-fused images in a monitoring scene, a third-party application download image scene, a web display image scene, or any scene to generate a corresponding image with a higher dynamic range, and the like, and the specific application scenario is not limited.

[0034] It can be understood that the relative positional relationship between the highlight region and the change region in the shooting scene 10 shown is only an example in the embodiments of the present application; in other embodiments, the relative positional relationship between the highlight region and the change region can also be a completely overlapping (for example, the highlight region and the change region are the same region) or completely non-overlapping (for example, the highlight region and the change region are different regions) relative positional relationship, and the specific relative positional relationship is not limited. Figure 1 The relative positional relationship between the highlight region and the change region in the shooting scene 10 shown is only an example in the embodiments of the present application; in other embodiments, the relative positional relationship between the highlight region and the change region can also be a completely overlapping (for example, the highlight region and the change region are the same region) or completely non-overlapping (for example, the highlight region and the change region are different regions) relative positional relationship, and the specific relative positional relationship is not limited. Figure 1 The relative positional relationship between the highlight region and the change region in the shooting scene 10 shown is only an example in the embodiments of the present application; in other embodiments, the relative positional relationship between the highlight region and the change region can also be a completely overlapping (for example, the highlight region and the change region are the same region) or completely non-overlapping (for example, the highlight region and the change region are different regions) relative positional relationship, and the specific relative positional relationship is not limited.

[0035] The embodiments of the present application will be described below through the example of the shooting scene 10 shown. Figure 1 The embodiments of the present application will be described below through the example of the shooting scene 10 shown. The embodiments of the present application will be described below through the example of the shooting scene 10 shown.

[0036] In some embodiments, as shown in FIG. 1A, the mobile phone 100 acquires a plurality of frames of to-be-fused images of the shooting scene 10, and fuses the plurality of frames of to-be-fused images to generate a shooting image. However, due to the existence of a changed area (e.g., a hand of a person and an area where an electronic display screen is located) in the shooting scene 10, the shooting image generated by the mobile phone 100 may have a ghosting abnormal phenomenon. Figure 1 In some embodiments, as shown in FIG. 1A, the mobile phone 100 acquires a plurality of frames of to-be-fused images of the shooting scene 10, and fuses the plurality of frames of to-be-fused images to generate a shooting image. However, due to the existence of a changed area (e.g., a hand of a person and an area where an electronic display screen is located) in the shooting scene 10, the shooting image generated by the mobile phone 100 may have a ghosting abnormal phenomenon.

[0037] For example, as shown in FIG. 1A, the mobile phone 100 acquires a plurality of frames of to-be-fused images of the shooting scene 10, and fuses the plurality of frames of to-be-fused images to generate a shooting image. However, due to the existence of a changed area (e.g., a hand of a person and an area where an electronic display screen is located) in the shooting scene 10, the shooting image generated by the mobile phone 100 may have a ghosting abnormal phenomenon. Figure 2 For example, as shown in FIG. 1A, the mobile phone 100 acquires a plurality of frames of to-be-fused images of the shooting scene 10, and fuses the plurality of frames of to-be-fused images to generate a shooting image. However, due to the existence of a changed area (e.g., a hand of a person and an area where an electronic display screen is located) in the shooting scene 10, the shooting image generated by the mobile phone 100 may have a ghosting abnormal phenomenon. Figure 2 For example, as shown in FIG. 1A, the mobile phone 100 acquires a plurality of frames of to-be-fused images of the shooting scene 10, and fuses the plurality of frames of to-be-fused images to generate a shooting image. However, due to the existence of a changed area (e.g., a hand of a person and an area where an electronic display screen is located) in the shooting scene 10, the shooting image generated by the mobile phone 100 may have a ghosting abnormal phenomenon. Figure 1 For example, as shown in FIG. 1A, the mobile phone 100 acquires a plurality of frames of to-be-fused images of the shooting scene 10, and fuses the plurality of frames of to-be-fused images to generate a shooting image. However, due to the existence of a changed area (e.g., a hand of a person and an area where an electronic display screen is located) in the shooting scene 10, the shooting image generated by the mobile phone 100 may have a ghosting abnormal phenomenon. Figure 2 For example, as shown in FIG. 1A, the mobile phone 100 acquires a plurality of frames of to-be-fused images of the shooting scene 10, and fuses the plurality of frames of to-be-fused images to generate a shooting image. However, due to the existence of a changed area (e.g., a hand of a person and an area where an electronic display screen is located) in the shooting scene 10, the shooting image generated by the mobile phone 100 may have a ghosting abnormal phenomenon. Figure 2 For example, as shown in FIG. 1A, the mobile phone 100 acquires a plurality of frames of to-be-fused images of the shooting scene 10, and fuses the plurality of frames of to-be-fused images to generate a shooting image. However, due to the existence of a changed area (e.g., a hand of a person and an area where an electronic display screen is located) in the shooting scene 10, the shooting image generated by the mobile phone 100 may have a ghosting abnormal phenomenon.

[0038] For example, as shown in FIG. 1A, the mobile phone 100 acquires a plurality of frames of to-be-fused images of the shooting scene 10, and fuses the plurality of frames of to-be-fused images to generate a shooting image. However, due to the existence of a changed area (e.g., a hand of a person and an area where an electronic display screen is located) in the shooting scene 10, the shooting image generated by the mobile phone 100 may have a ghosting abnormal phenomenon. Figure 1 For example, as shown in FIG. 1A, the mobile phone 100 acquires a plurality of frames of to-be-fused images of the shooting scene 10, and fuses the plurality of frames of to-be-fused images to generate a shooting image. However, due to the existence of a changed area (e.g., a hand of a person and an area where an electronic display screen is located) in the shooting scene 10, the shooting image generated by the mobile phone 100 may have a ghosting abnormal phenomenon.

[0039] For example, as shown in FIG. 1A, the mobile phone 100 acquires a plurality of frames of to-be-fused images of the shooting scene 10, and fuses the plurality of frames of to-be-fused images to generate a shooting image. However, due to the existence of a changed area (e.g., a hand of a person and an area where an electronic display screen is located) in the shooting scene 10, the shooting image generated by the mobile phone 100 may have a ghosting abnormal phenomenon. Figure 3 For example, as shown in FIG. 1A, the mobile phone 100 acquires a plurality of frames of to-be-fused images of the shooting scene 10, and fuses the plurality of frames of to-be-fused images to generate a shooting image. However, due to the existence of a changed area (e.g., a hand of a person and an area where an electronic display screen is located) in the shooting scene 10, the shooting image generated by the mobile phone 100 may have a ghosting abnormal phenomenon. Figure 3Image 30 is a captured image of the corresponding shooting scene 10 output by mobile phone 100 based on a bright frame, and image 31A is a magnified view of region 31 in image 30. For example, combined with... Figure 1 and Figure 3 As shown, because the electronic display screen in shooting scene 10 has a high brightness, the pixel brightness of the shooting elements on the electronic display screen will be higher in bright frames, resulting in an overexposure anomaly; for example, Figure 3 The electronic display screen shown in image 31A has high brightness, causing a loss of detail in the photograph displayed on the screen; for example, combined with... Figure 1 and Figure 3 As shown, relative Figure 1 , Figure 3 The photo displayed on the electronic display screen only retains the dark part of the person's hair, while the light-colored parts of the person's face and torso appear white due to overexposure.

[0040] As mentioned earlier, during the process of electronic devices acquiring multiple frames of images to be fused, if there are highlight areas and changing areas in the acquisition scene (such as the aforementioned shooting scene 10, monitoring scene, third-party application download image scene, web page display image scene, etc.), the electronic devices may exhibit abnormal phenomena such as ghosting and overexposure when processing multiple frames of images to be fused.

[0041] To address this, the present invention proposes an image processing method. In this method, after an electronic device acquires multiple frames of images to be fused from a scene, it determines whether the multiple frames include a first region and a second region based on the pixel information of the images. The first region includes a first shooting element whose position changes in different images (e.g., when the scene includes a changing region, the multiple frames include the first region). The second region includes a second shooting element whose brightness meets a first brightness condition (e.g., when the scene includes a highlight region, the multiple frames include the second region). When the determination is yes, the electronic device brightens the dark frames with lower exposure values ​​in the multiple frames (hereinafter referred to as brightening) to use the brightened dark frames as the output image (corresponding to an instance of the first target image). It can be understood that since the exposure value of the dark frames is low, the brightness of the second shooting element with higher brightness in the dark frames will be reduced without overexposure, thus avoiding the abnormal phenomenon of overexposure in the output image. Furthermore, since the electronic device does not perform fusion processing on the multiple frames but uses the brightened dark frames as the output image, the abnormal phenomenon of ghosting in the output image can be avoided.

[0042] The following example illustrates the process by which an electronic device determines whether a multi-frame image to be fused includes a first region based on pixel information of the multi-frame image to be fused.

[0043] In some embodiments, the electronic device can determine whether the multiple frames to be fused include a first region based on the misaligned pixels between them. For example, when the electronic device determines that the multiple frames to be fused include misaligned pixels, or that the pixel area of ​​the misaligned pixels is greater than a preset first area threshold, it determines that the multiple frames to be fused include the first region. That is, the acquisition scene corresponding to the multiple frames to be fused acquired by the electronic device includes a changing region.

[0044] For example, an electronic device can determine the non-aligned pixels between multiple frames of images to be fused based on pixel information of the red, green, and blue (RGB) values ​​of pixels at the same pixel coordinates in the multiple frames of images to be fused.

[0045] For example, an electronic device acquires two images from multiple frames to be merged: image A with the highest exposure value and image B with the lowest exposure value. When the electronic device determines that the RGB value ratio of pixel A1 at pixel coordinate 01 in image A is different from the RGB value ratio of pixel B1 at pixel coordinate 01 in image B, it determines that pixels A1 and B1 are misaligned pixels. That is, the multiple frames to be merged include a first region, and the region position of the first region in each frame is the coordinate position corresponding to pixel coordinate 01 in each frame. For example, the pixels included in the first region in each frame can be used as the first captured element in the corresponding frame.

[0046] It is understood that the image sizes of the multiple frames to be fused are usually the same, and the pixels in the multiple frames to be fused correspond one-to-one. That is, pixels at the same pixel coordinate in the multiple frames to be fused correspond to the same location in the acquisition environment. For example, pixels A1 and B1 at pixel coordinate 01 in images A and B to be fused correspond to the same location in the acquisition environment. Exemplarily, images A and B to be fused can also be any two different images from the multiple frames to be fused.

[0047] For example, when an electronic device determines that the RGB value ratios of pixels at the same pixel coordinate in a predetermined number of images to be merged are different, it determines that the images to be merged include a first region. The predetermined number could be one-half, one-third, etc., without specific limitations. Alternatively, the electronic device can also determine the differences between pixels at the same pixel coordinate in the images to be merged based on methods such as the RGB absolute difference method or the RGB Euclidean distance method, to identify misaligned pixels between the images to be merged, without specific limitations.

[0048] In other embodiments, the electronic device may also determine the misaligned pixels between multiple frames of images to be fused based on optical flow, feature point detection methods (scale-invariant feature transform (SIFT), speeded up robust features (SURF), etc.), inter-frame difference detection, etc., without any specific limitations.

[0049] In other embodiments, the electronic device may also perform feature recognition on multiple frames of images to be fused using feature recognition methods such as deep learning and object segmentation to identify the shooting elements included in the multiple frames of images to be fused. Based on the pixel coordinates of the same shooting elements included in the multiple frames of images to be fused, it can determine whether the multiple frames of images to be fused include the first region. For example, when the electronic device determines that the pixel coordinates of the same shooting elements included in the multiple frames of images to be fused are different, or that the difference in pixel coordinates is greater than a difference threshold, it determines that the multiple frames of images to be fused include the first region.

[0050] For example, for Figure 1 The mobile phone 100 shows multiple frames of images to be fused captured from shooting scene 10. These multiple frames may include the same shooting elements such as people and electronic displays. Further, the electronic device determines the pixel coordinates of shooting elements, such as electronic displays, in the multiple frames. When the shooting elements of the electronic displays have completely or partially different pixel coordinates in different frames to be fused, it indicates that the position of the shooting elements of the electronic displays has changed in the different frames to be fused. Therefore, it is determined that the multiple frames to be fused include a first region. The region position of the first region in each frame to be fused is the coordinate position corresponding to the pixel coordinates of the shooting elements of the electronic displays in each frame to be fused. Furthermore, the shooting elements of the electronic displays in each frame to be fused can be used as the first shooting element in the corresponding frame to be fused.

[0051] The following example illustrates the process by which an electronic device determines whether a multi-frame image to be fused includes a second region based on pixel information of the multi-frame image to be fused.

[0052] In some embodiments, the electronic device can determine whether the brightness of pixels in multiple frames of images to be fused meets a first brightness condition through methods such as histogram analysis, threshold detection, RGB saturation detection, color space conversion, and deep learning, and then determine whether the multiple frames of images to be fused include a second region. For example, when the electronic device determines that the multiple frames of images to be fused include pixels whose brightness meets the first brightness condition, or that the pixel area of ​​the pixels whose brightness meets the first brightness condition is greater than a preset second area threshold, it determines that the multiple frames of images to be fused include a second region. That is, the acquisition scene corresponding to the multiple frames of images to be fused acquired by the electronic device includes a highlight region.

[0053] For example, an electronic device can perform brightness threshold detection on the brightest frame with the highest exposure value in multiple frames of images to be merged. When the brightness of a pixel in the bright frame is greater than or equal to a preset first brightness threshold, that is, when the pixel brightness meets the first brightness condition, it is determined that the bright frame includes a second region, and the region position of the second region in the bright frame is the pixel coordinate position corresponding to the pixel whose brightness is greater than or equal to the first brightness threshold. Simultaneously, the electronic device determines the region position of the second region in each frame of images to be merged based on the region position of the second region in the bright frame. For example, the region position of the second region in each frame of images to be merged is the same as the region position of the second region in the bright frame. Exemplarily, the pixels included in the second region in each frame of images to be merged can be used as the second shooting elements in the corresponding frames of images to be merged. It can be understood that the brightness of each pixel in the second region is greater than or equal to the first brightness threshold.

[0054] For example, after identifying the shooting elements included in multiple frames of images to be fused, the electronic device performs brightness threshold detection on the brightness of the pixels corresponding to the shooting elements included in the bright frame. When it is determined that the brightness of all or a portion of the pixels corresponding to the shooting element (e.g., a first proportion of pixels such as 95%, 90%, 80%, etc.) is greater than or equal to the corresponding preset first brightness threshold, that is, when the brightness of all or a portion of the pixels meets the first brightness condition, it is determined that the bright frame includes a second region, and the region position of the second region in the bright frame is: the pixel coordinate position corresponding to the shooting element whose brightness is greater than or equal to the preset brightness threshold. Similarly, the region position of the second region in each frame of images to be fused is the same as the region position of the second region in the bright frame. For example, the shooting elements included in the second region in each frame of images to be fused can be used as the second shooting elements in the corresponding frames of images to be fused. It can be understood that when all pixels corresponding to the shooting element meet the first brightness condition, the brightness of each pixel in the second region is greater than or equal to the first brightness threshold; when a portion of the pixels corresponding to the shooting element meet the first brightness condition, the brightness of a first proportion of pixels in the second region is greater than or equal to the first brightness threshold. The first percentage can also be other percentages, such as 98%, 87%, etc., and there are no specific restrictions.

[0055] Alternatively, the electronic device can preset different brightness thresholds based on the exposure values ​​of multiple frames of images to be fused, and detect pixels in each frame of images to be fused that are greater than or equal to the corresponding brightness threshold based on the different brightness thresholds, so as to determine the second region of each frame of images to be fused, etc. That is, the location of the second region in multiple frames of images to be fused may be different.

[0056] It is understandable that the first region and the second region in a frame of an image to be fused can be two completely overlapping regions. For example, when the changing region and the highlight region in the scene corresponding to the multiple frames of images to be fused captured by the electronic device completely overlap, the first region and the second region completely overlap. That is, the pixel coordinates corresponding to the first shooting element in a frame of an image to be fused completely overlap with the pixel coordinates corresponding to the second shooting element, that is, the first shooting element and the second shooting element are the same shooting element.

[0057] Alternatively, the first and second regions in a frame of an image to be fused can be two completely non-overlapping regions. For example, when the changing regions and highlight regions in the scene corresponding to the multiple frames of images to be fused captured by an electronic device do not overlap at all, the first and second regions do not overlap at all. That is, the pixel coordinates corresponding to the first shooting element and the pixel coordinates corresponding to the second shooting element in a frame of an image to be fused do not overlap at all, meaning that the first shooting element and the second shooting element are different shooting elements.

[0058] Alternatively, a frame of image to be fused may include multiple first regions and multiple second regions, where some first regions and some second regions completely overlap, while other first regions and other second regions do not overlap at all. For example, the change areas and highlight areas in the scene corresponding to the acquisition of multiple frames of images to be fused by an electronic device partially overlap; that is, a frame of image to be fused includes multiple first shooting elements and multiple second shooting elements, where some first shooting elements and some second shooting elements are the same shooting elements, while other first shooting elements and other second shooting elements are different shooting elements.

[0059] The following example illustrates the process by which an electronic device determines the output image based on the dark frames in multiple images to be fused.

[0060] In some embodiments, the electronic device may first determine the processing method of the multi-frame images to be fused by determining whether the multi-frame images to be fused include a first region.

[0061] For example, when an electronic device determines that a first region is not included in the multi-frame images to be fused, it means that pixels at the same pixel coordinate position in the multi-frame images can be perfectly aligned, meaning that ghosting will not occur during fusion; that is, the scene captured by the electronic device for the multi-frame images to be fused does not include changing areas. In this way, the electronic device can generate an output image by fusing multiple frames, ensuring a high dynamic range, improving the brightness and contrast of the output image, and thus ensuring that the output image can present more image details.

[0062] In some embodiments, when the electronic device determines that the multi-frame images to be fused include a first region, the electronic device may further determine the processing method of the multi-frame images to be fused by determining the size of the first region.

[0063] For example, when an electronic device determines that the multi-frame images to be fused include a first region, and the size of the first region is less than a preset first region threshold, it indicates that the pixel area of ​​the pixels that cannot be aligned in the multi-frame images to be fused is small. That is, the area where ghosting may occur during fusion is small, resulting in less impact on visual effects. Thus, to ensure that the output image has a high dynamic range, the electronic device can also generate the output image (corresponding to an example of the second target image) by fusing the multi-frame images to be fused.

[0064] For example, when an electronic device determines that multiple frames of images to be fused include a first region, and the size of this first region is greater than or equal to a first region threshold, it indicates that the pixel areas of the multiple frames to be fused that cannot be aligned are large, meaning that the area of ​​ghosting in the multiple frames to be fused during fusion is large. Thus, the electronic device can generate an output image without fusing the multiple frames to be fused to avoid the abnormal phenomenon of ghosting during fusion. The specific processing procedure is described below.

[0065] It is understood that when determining the size of the first region, the electronic device can base its judgment on the first region included in any frame of the multiple frames to be fused. For example, a frame of the image to be fused may include multiple first regions. When the size of any first region is greater than or equal to a first region threshold, the electronic device will not fuse the multiple frames of the image to be fused. Alternatively, the electronic device may also base its judgment on the size of the largest first region included in the multiple frames of the image to be fused, without any specific limitation.

[0066] In some embodiments, when the electronic device determines that the multi-frame images to be fused include a first region and the size of the first region is greater than or equal to a first region threshold, the electronic device may continue to determine the processing method of the multi-frame images to be fused by determining whether the multi-frame images to be fused include a second region.

[0067] For example, when an electronic device determines that the multiple frames to be merged do not include the second region, it means that the multiple frames to be merged will not exhibit overexposure anomalies. In this way, the electronic device can select the brightest frame with the highest exposure value from the multiple frames to be merged as the output image to improve the shadow details, color vibrancy, brightness, etc. of the output image. At the same time, since the electronic device does not perform fusion processing, it can avoid the appearance of ghosting anomalies in the output image.

[0068] In some embodiments, when an electronic device determines that multiple frames of images to be fused include a first region, and the size of the first region is greater than or equal to a first region threshold, and the multiple frames of images to be fused also include a second region, the electronic device may further determine the processing method for the multiple frames of images to be fused by determining whether the regional features of the second region satisfy a first feature condition. For example, the regional features of the second region include at least region size and region type.

[0069] For example, when the electronic device determines that the multi-frame image to be fused includes a second region, and the size of the second region is greater than or equal to a second region threshold, it determines that the region features of the second region satisfy a first feature condition, indicating that an overexposure anomaly may occur in the multi-frame image to be fused, and that the overexposure area is relatively large. Thus, the electronic device can select the darkest frame with the lowest exposure value from the multi-frame image to be fused, brighten the dark frame, and use the brightened dark frame as the output image (corresponding to an instance of the first target image). Because the dark frame has a low exposure value, i.e., the dark frame reduces brightness, thus avoiding the overexposure anomaly. Simultaneously, because the electronic device does not perform fusion processing, it can avoid the ghosting anomaly in the output image.

[0070] For example, when the electronic device determines that the multi-frame image to be fused includes a second region, and the region type of the second region is a preset type, it determines that the region features of the second region satisfy the first feature condition, indicating that the second region is a region of interest. For instance, the electronic device can determine this by including the second region of the darkest frame with the lowest exposure value in the multi-frame image to be fused. For example, if the gradient variance of the second region of the dark frame is greater than or equal to the first variance threshold, it indicates that the texture information in the second region is relatively rich, and the region type of the second region is determined to be the high gradient variance type in the preset types, to avoid the loss of image details due to overexposure. Or, for example, if the second shooting element in the second region of the dark frame is a preset element of interest such as a person or a face, the region type of the second region is determined to be the element of interest type in the preset types, to avoid the loss of image details due to overexposure. In this way, the electronic device can select the darkest frame with the lowest exposure value from the multi-frame image to be fused, brighten the dark frame, and use the brightened dark frame as the output image (an example corresponding to the first target image). Because dark frames have lower exposure values, meaning they reduce brightness and thus avoid overexposure, and because the electronic device does not perform fusion processing, ghosting in the output image is avoided. It is understood that high gradient variance type and interest element type are merely examples of preset types; in other embodiments, preset types may include more types, such as user-defined types, etc., without specific limitations.

[0071] In some embodiments, when the electronic device determines that the size of the second region is less than a second region threshold and the region type of the second region is not a preset type, it determines that the region features of the second region do not meet the first feature condition. This indicates that the area of ​​potential overexposure in the multi-frame images to be fused is small, and the overexposure area is a region of no interest. For example, the second region has little texture information, or the second shooting element included in the second region is a non-interest element such as white paper or white dots. In this way, the electronic device can select the brightest frame with the highest exposure value from the multi-frame images to be fused as the output image (corresponding to an instance of the third target image) to improve the dark details, color vibrancy, brightness, etc. of the output image. At the same time, since the electronic device does not perform fusion processing, it can avoid the abnormal phenomenon of ghosting in the output image.

[0072] In other embodiments, when the electronic device determines that the size of the second region is greater than or equal to the threshold of the second region, or the region type of the second region is a preset type, or the size of the second region is greater than or equal to the threshold of the second region, and the region type of the second region is a preset type, the electronic device determines that the region features of the second region satisfy the first feature condition, and the electronic device can use the brightened dark frame as the output image (an instance corresponding to the first target image).

[0073] This is understandable, because while dark frames have lower brightness, they retain more detail in the bright areas compared to bright frames, thus avoiding overexposure. Conversely, while bright frames have higher brightness, they retain more detail in the dark areas compared to dark frames. Therefore, when an electronic device outputs a brightened dark frame, it can perform noise reduction processing on the brightened dark frame based on the bright frame to supplement the dark details in the output image and improve the signal-to-noise ratio.

[0074] In some embodiments, the electronic device can use the darkest frame with the lowest exposure value among multiple frames to be fused as the target frame, and other frames with exposure values ​​higher than the target frame as non-target frames. Subsequently, the electronic device performs deformation alignment on the non-target frames based on the target frame to obtain the deformed and aligned non-target frames (hereinafter referred to as aligned frames), ensuring that each pixel in the aligned frame is aligned with the corresponding pixels in the target frame. For example, the electronic device can perform deformation alignment on the non-target frames based on the target frame using image alignment techniques such as feature point detection algorithms, optical flow methods, and image interpolation matching (e.g., affine transformation), without any specific limitations.

[0075] Furthermore, the electronic device can first brighten the target frame as a whole, increasing its overall brightness and restoring shadow details to compensate for the lack of shadow brightness caused by insufficient exposure. However, this overall brightening may increase shadow noise in the brightened target frame. Therefore, the electronic device can perform noise reduction on the brightened target frame based on pixel information from the aligned frame. For example, it can fuse the shadow pixels in the aligned frame with the corresponding shadow pixels in the brightened target frame, or weight the shadow pixel values ​​in the brightened target frame based on the shadow pixel values ​​of the aligned frame. This replenishes shadow details in the output image, resulting in a high bit width output image and improving its dynamic range. In essence, the noise reduction processing of the brightened target frame based on the aligned frame by the electronic device can compensate for the shadow noise introduced by the overall brightening of the target frame, improving the signal-to-noise ratio of the output image.

[0076] In some embodiments, for Figure 1 For example, after capturing multiple frames of images to be fused from the shooting scene 10, when the mobile phone 100 determines that the multiple frames to be fused include a first region and a second region, and the size of the first region is greater than or equal to the first region threshold, and the region features of the second region meet the first feature condition, the mobile phone 100 determines the target frame and non-target frames in the multiple frames to be fused. Based on the target frame, it performs deformation alignment on the non-target frames to obtain an aligned frame, brightens the target frame as a whole, and can perform noise reduction processing on the brightened target frame based on the pixel information (such as pixel values ​​and / or brightness) in the aligned frame to obtain the output image. For example, as... Figure 4 As shown, Figure 4Image 40 is the output image of mobile phone 100 based on the dark frame, and image 41A is a magnified view of region 41 in image 40. Clearly, for Figure 2 and Figure 3 , Figure 4 Image 40 shown not only avoids Figure 2 The abnormal phenomenon of ghosts and shadows was avoided. Figure 3 The problem of overexposure causing loss of detail in photos displayed on electronic screens.

[0077] Combination Figure 1 Example, Figure 5 According to some embodiments of this application, a process diagram of an image processing method is shown. For example... Figure 1 and Figure 5 As shown, after acquiring multiple frames of images to be fused from the shooting scene 10, the mobile phone 100 performs multi-frame noise reduction on the multiple dark frames and multiple bright frames in the multiple frames to be fused, such as S51 dark frame multi-frame noise reduction and S52 bright frame multi-frame noise reduction. For example, the noise reduction process can be implemented using filtering (such as bilateral filtering, wavelet transform, etc.), sparse representation, deep learning and other noise reduction algorithms. The dark frames and bright frames can use the same or different noise reduction processes, and there is no specific limitation.

[0078] Furthermore, the mobile phone 100 performs S53 calculation on the non-aligned area of ​​the multi-frame images obtained by noise reduction and determines whether the area is greater than a threshold. For example, the aforementioned electronic device determines whether the multi-frame images to be fused include the first region and whether the area of ​​the first region is greater than the threshold of the first region.

[0079] In some embodiments, when the mobile phone 100 determines that the area is less than a threshold, for example, when the aforementioned electronic device determines that the multi-frame images to be fused include a first region and the size of the first region is less than the first region threshold, it indicates that the pixel area of ​​the pixels that cannot be aligned in the multi-frame images to be fused is small, that is, the area of ​​the multi-frame images to be fused that may produce ghosting during fusion is small, and the impact on visual effects is small. Thus, to ensure that the output image has a high dynamic range, the mobile phone 100 performs S54 HDR fusion of the multi-frame images to be fused based on bright frames to generate a high bit width image. After performing tone mapping and color brightness adjustment on the high bit width image in S55, an output image with a preset bit width adapted to the bit width of the mobile phone 100 is generated. Exemplarily, tone mapping can be implemented through tone mapping algorithms such as histogram equalization, global tone mapping (e.g., S-curve, adaptive logarithmic, etc.), and local tone mapping (e.g., bilateral filtering), and is not specifically limited. It can be understood that since the area of ​​the multi-frame images to be fused that may produce ghosting during fusion is small, the impact on visual effects is small, that is, the output image is normal.

[0080] In other embodiments, when the mobile phone 100 determines that the area is greater than or equal to a threshold, for example, when the aforementioned electronic device determines that the multi-frame images to be fused include a first region and the size of the first region is greater than or equal to the threshold of the first region, the mobile phone 100 can generate an output image by not fusion of the multi-frame images to be fused, so as to avoid the abnormal phenomenon of ghosting in the fusion.

[0081] Furthermore, when the mobile phone 100 determines that the area is greater than or equal to the threshold, the mobile phone 100 performs S56 to determine whether the area of ​​the overexposed area is greater than the threshold; and / or, determines whether the gradient variance of the overexposed area is greater than the threshold, for example, the aforementioned electronic device determines the regional features such as the size and type of the second region included in the multi-frame image to be fused.

[0082] In some embodiments, when the mobile phone 100 determines that the area of ​​the overexposed region is less than a threshold and the gradient variance of the overexposed region is less than a threshold, for example, when the aforementioned electronic device determines that the regional features of the second region do not meet the first feature condition, it indicates that the area of ​​potential overexposure in the multi-frame image to be fused is small, and the overexposed region is a region of no interest. Thus, the mobile phone 100 performs S57 (non-fusion of dark frames), selects the brightest frame with the highest exposure value from the multi-frame image to be fused, and then performs S58 (tone mapping and color brightness adjustment) to generate an output image with a preset bit width adapted to the bit width of the mobile phone 100. It can be understood that since the multi-frame image to be fused includes the second region, the output image output has an overexposed area.

[0083] In other embodiments, when the mobile phone 100 determines that the area of ​​the overexposed region is greater than or equal to a threshold and / or determines that the gradient variance of the overexposed region is greater than or equal to a threshold, for example, when the aforementioned electronic device determines that the regional features of the second region satisfy the first feature condition, it indicates that the overexposed area in the multi-frame image to be fused is large and / or the second region is a region of interest, and it is necessary to avoid the second region from losing image details due to overexposure. Thus, the mobile phone 100 performs S59 to determine the target frame and non-target frames in the multi-frame image to be fused, performs deformation alignment on the non-target frames based on the target frame to obtain an aligned frame, then performs S510 to brighten the target frame as a whole, and performs noise reduction processing on the low-brightness areas in the brightened target frame based on the pixel information in the aligned frame to obtain a high bit width image. Then, after performing tone mapping and color brightness adjustment on the high bit width image in S511, an output image with a preset bit width adapted to the bit width of the mobile phone 100 is generated. For example, the output image is... Figure 4 Image 40 in the image, obviously, for Figure 2 and Figure 3 , Figure 4 Image 40 shown not only avoids Figure 2 The abnormal phenomenon of ghosts and shadows was avoided. Figure 3 The problem of overexposure causing loss of detail in photos displayed on electronic screens.

[0084] In some embodiments, such as Figure 6 As shown, the horizontal axis represents the brightness value. Figure 6 In the process, the HDR multi-frame fusion frame is as follows: When the mobile phone 100 determines that the area is less than a threshold, it performs HDR fusion on multiple frames of images to be fused based on the bright frame to generate a high bit width image. For example, based on the bright frame, the bright part of the bright frame is supplemented by the bright part of the dark frame (the overexposed area of ​​the bright frame) to generate an HDR multi-frame fusion frame. The process of brightening dark frames is as follows: When the mobile phone 100 determines that the area is greater than or equal to a threshold, and the area of ​​the overexposed area is greater than or equal to a threshold and / or the gradient variance of the overexposed area is greater than or equal to a threshold, the mobile phone 100 uses the dark frame as a reference, takes the dark frame as the target frame, performs deformation alignment on the non-target frames based on the target frame to obtain an aligned frame, directly brightens the target frame, and performs noise reduction processing on the low-brightness area in the brightened target frame based on the pixel information in the aligned frame to obtain a high bit width image. It is understandable that the brightened dark frame obtained based on the dark frame has the same or similar high bit width as the HDR multi-frame fusion frame obtained based on the bright frame; and, since the exposure value of the dark frame is low, the brightened dark frame will not have an overexposure abnormality; at the same time, since the brightened dark frame does not undergo multi-frame fusion, the brightened dark frame will also have a ghosting abnormality.

[0085] The following example illustrates the process by which an electronic device determines the output image based on bright frames in multiple images to be fused.

[0086] Combination Figure 1 Example, Figure 7 According to some embodiments of this application, a process diagram of another image processing method is shown. For example... Figure 1 and Figure 7 As shown, after the mobile phone 100 acquires multiple frames of images to be fused from the shooting scene 10, it performs S71 dark frame multi-frame noise reduction and S72 bright frame multi-frame noise reduction on the multiple dark frames and multiple bright frames in the multiple frames to be fused. Then, it performs S73 to calculate the non-aligned area on the multi-frame images obtained by noise reduction and determines whether the area is greater than the threshold.

[0087] In some embodiments, when the mobile phone 100 determines that the area is less than a threshold, it indicates that the pixel area of ​​the pixels that cannot be aligned in the multi-frame images to be fused is small. The mobile phone 100 performs S74 to fuse the multi-frame images to be fused based on the bright frame to generate a high bit width image. After performing tone mapping on the high bit width image in S75, an output image with a preset bit width adapted to the bit width of the mobile phone 100 is generated. It can be understood that since the area of ​​the ghosting that may appear in the multi-frame images to be fused during fusion is small, the impact on the visual effect is small, that is, the output image is normal.

[0088] In other embodiments, when the mobile phone 100 determines that the area is greater than or equal to a threshold, it indicates that the pixel area of ​​the pixels that cannot be aligned in the multi-frame images to be fused is large. The mobile phone 100 performs S76 to not fuse dark frames, selects the bright frame with the higher or highest exposure value from the multi-frame images to be fused, performs S77 tone mapping, and generates an output image with a preset bit width adapted to the bit width of the mobile phone 100. It can be understood that, relative to Figure 5 , Figure 7 Without considering the regional features of overexposed areas (second region) in the multi-frame images to be fused, if the multi-frame images to be fused include overexposed areas and the area of ​​the overexposed areas is greater than or equal to a threshold and / or the gradient variance of the overexposed areas is greater than or equal to a threshold, if the mobile phone 100 determines the output image based on the bright frames in the multi-frame images to be fused, it will lead to the output image exhibiting, for example... Figure 3 The overexposure anomaly shown is illustrated.

[0089] Combination Figure 1 , Figures 4 to 6 , Figure 8 A flowchart of an image processing method is shown according to some embodiments of this application. It can be understood that... Figure 8 The processes shown are all executed by electronic devices. For simplicity, the following description... Figure 8 The execution entity of the process will not be described again when the process is shown. For example... Figure 8 As shown, the image processing method includes, but is not limited to, the following process:

[0090] S801: Acquire multiple frames of images to be fused.

[0091] In some embodiments, the electronic device can capture multiple frames of images with different exposure values ​​of the scene to be fused using a built-in camera or an external camera. For example, such as Figure 1 As shown, the mobile phone 100 can capture multiple frames of images with different exposure values ​​from the shooting scene 10 through its rear camera, which are then to be fused. In other embodiments, the electronic device can also acquire multiple frames of images to be fused through other electronic devices, and there are no specific limitations.

[0092] For example, the image format of the multiple frames to be fused is RAW format, and the multiple frames to be fused typically have a high bit width, such as 12 bits or 14 bits.

[0093] S802: The multi-frame image to be fused is determined to include the first region and the second region.

[0094] In some embodiments, the electronic device determines whether the multi-frame images to be fused include a first shooting element whose position changes in different images to be fused, and a second shooting element whose brightness meets a first brightness condition. For example, when the electronic device determines that the first shooting element is included, it determines that a first region is included, and the region position of the first region corresponds to the pixel position of the first shooting element in the corresponding image to be fused; when the electronic device determines that the second shooting element is included, it determines that a second region is included, and the region position of the second region corresponds to the pixel position of the second shooting element in the corresponding image to be fused. For details, please refer to the foregoing description of the first region and the second region, which will not be repeated here.

[0095] For example, an electronic device can identify whether multiple frames of images to be fused include a first region and a second region, based on the fact that the multiple frames of images to be fused are in RAW format and have a high bit width.

[0096] In some embodiments, when the electronic device determines that the multi-frame image to be fused includes a first region and a second region, it may further determine the first region and the second region.

[0097] For example, if the size of the first region is greater than or equal to the threshold of the first region, and the region features of the second region meet the first feature condition, then S803 continues to be executed.

[0098] For example, when the size of the first region is smaller than a threshold, it indicates that the pixel area where the pixels in the multiple frames to be fused cannot be aligned is small. In other words, the area where ghosting may occur during fusion is small, resulting in less visual impact. Thus, electronic devices can obtain a second target image by performing fusion processing, such as HDR fusion, on the multiple frames to be fused, ensuring that the second target image has a high dynamic range.

[0099] For example, if the regional features of the second region do not meet the first feature condition, it indicates that the area of ​​potential overexposure in the multi-frame image to be fused is small, and the overexposure area is a region of no interest. In this way, the electronic device can avoid fusing dark frames and select the brightest frame with the highest exposure value from the multi-frame image to be fused, obtaining the third target image based on this bright frame, thus improving the dark details, color vibrancy, and brightness of the third target image. For details, please refer to the aforementioned process of determining the output image based on bright frames, which will not be repeated here.

[0100] S803: Select the first image from multiple frames of images to be fused, and increase the brightness of the first image to obtain the first intermediate image.

[0101] In some embodiments, the electronic device selects the image with the lowest exposure value from multiple frames of images to be fused as the first image, and then brightens the first image as a whole to obtain a first intermediate image. It can be understood that the brightness value used by the electronic device to brighten the first image as a whole can be set with reference to the brightness value of the image with a normal exposure value (e.g., an exposure value of 0) among the multiple frames to be fused. For example, the first image can be brightened to the same brightness value as the normal exposure value of the first intermediate image; or the brightness value can be determined according to the acquisition scene corresponding to the first image. For example, when the acquisition scene is dark, the overall brightness value is higher; when the acquisition scene is bright, the overall brightness value is lower, etc., and there are no specific limitations.

[0102] For example, in S803, the first image is usually converted from RAW format to linear RGB format, that is, the image format of the first intermediate image is linear RGB format and has a higher bit width, such as 16 bits or 32 bits; or the first image may not be converted, that is, the image format of the first intermediate image is the same as the image format of the first image, etc., and there is no specific limitation.

[0103] S804: Based on the first image, deform and align the second image in the multiple frames to be fused to obtain the second intermediate image.

[0104] In some embodiments, the electronic device selects at least one or more second images from multiple frames of images to be fused, each with an exposure value greater than that of a first image. Based on the first image, it performs deformation alignment on the second image to obtain a second intermediate image. This ensures that each pixel in the second intermediate image is aligned with the corresponding pixels in the first image, avoiding ghosting caused by misaligned pixels. For example, for... Figure 1 In the shooting scene 10 shown, due to the change in the position of the electronic display screen in the shooting scene 10, the pixel positions of the shooting elements of the electronic display screen in the first image and the second image are different. Thus, the second intermediate image is obtained by deforming and aligning the second image based on the first image. For example, the pixel positions of the shooting elements of the electronic display screen in the second image are adjusted so that the pixel positions of the shooting elements of the electronic display screen in the second intermediate image are aligned one by one with the pixel positions of the shooting elements of the electronic display screen in the first image.

[0105] For example, in S803, the second image is usually converted from RAW format to linear RGB format, that is, the image format of the second intermediate image is linear RGB format and has a higher bit width, such as 16 bits or 32 bits; or the second image may not be converted, that is, the image format of the second intermediate image is the same as the image format of the first image, etc., and there is no specific limitation.

[0106] S805: Based on the first intermediate image and the second intermediate image, obtain the first target image.

[0107] In some embodiments, the electronic device may perform noise reduction processing on the first intermediate image based on pixel information in the second intermediate image, such as filtering, sparse representation, deep learning, etc., to obtain a third intermediate image. For example, the pixel values ​​of dark pixels in the first intermediate image may be weighted and adjusted based on the pixel values ​​of dark pixels in the second intermediate image to obtain the third intermediate image. This compensates for dark noise introduced by overall brightening of the first image, supplements dark details in the first intermediate image obtained by overall brightening, and improves the dynamic range and signal-to-noise ratio of the third intermediate image. Further, the electronic device performs tone mapping on the third intermediate image to reduce the high bit width (e.g., 16-bit) image to a low bit width (e.g., 8-bit) image, and converts the low bit width image to a target format (e.g., JPEG, PNG, HEIF, etc.) to obtain a first target image, so that the electronic device can display and store the first target image. It is understood that the bit depth of the first target image is less than the bit depth of the third intermediate image.

[0108] It is understandable that, since the first image has a low exposure value, the first intermediate image obtained by brightening the first image will not exhibit overexposure. Simultaneously, using pixel information from the second intermediate image with a higher exposure value to perform noise reduction on the first intermediate image can compensate for the dark noise introduced by the overall brightening of the first image, and improve the dynamic range and signal-to-noise ratio of the third intermediate image obtained through noise reduction. Furthermore, Figure 8 The image processing method shown does not perform multi-frame fusion processing on the multiple frames of images to be fused. This means that the final first target image will not exhibit overexposure or ghosting anomalies. For example, the first target image is... Figure 4 Image 40 in the image, obviously, for Figure 2 and Figure 3 , Figure 4 Image 40 shown not only avoids Figure 2 The abnormal phenomenon of ghosts and shadows was avoided. Figure 3 The problem of overexposure causing loss of detail in photos displayed on electronic screens.

[0109] In some embodiments, this application also provides a shooting method. In this method, when an electronic device detects a user's shooting command, it acquires multiple frames of images to be fused, processes the multiple frames of images to be fused based on the aforementioned image processing method to obtain a processed result image, and then displays the processed result image.

[0110] For example, such as Figure 1As shown, when the mobile phone 100 detects that the user clicks button 100A, it captures multiple frames of images to be fused from the shooting scene 10. Subsequently, the mobile phone 100 determines that the multiple frames of images to be fused include a first region and a second region; and, when the mobile phone 100 determines that the size of the first region is greater than or equal to a first region threshold, and the region features of the second region satisfy a first feature condition, it selects the first image from the multiple frames of images to be fused, increases the brightness of the first image to obtain the first target image, and displays the first target image, for example... Figure 4 Image 40 in the image.

[0111] It is understandable that since the exposure value of the first image (such as the dark frame) is low, the brightness of the brighter pixels (such as the second shooting element) in the first image will be reduced without overexposure, thus avoiding the abnormal phenomenon of overexposure in the first target image; and since the mobile phone 100 does not perform fusion processing on the multiple frames to be fused, but uses the brightened first image as the first target image, the abnormal phenomenon of ghosting in the first target image can be avoided.

[0112] For example, when the mobile phone 100 determines that the size of the first region is less than the threshold of the first region, the mobile phone 100 can obtain the second target image by performing a fusion process such as HDR fusion on multiple frames of images to be fused, so as to ensure that the second target image has a high dynamic range and display the second target image.

[0113] For example, when the mobile phone 100 determines that the regional features of the second region do not meet the first feature condition, the mobile phone 100 may not fuse the dark frames, but select the bright frame with the higher or highest exposure value from the multiple frames to be fused, and obtain the third target image based on the bright frame, so as to improve the dark details, color vividness, brightness, etc. of the third target image, and display the third target image.

[0114] Figure 9 According to some embodiments of this application, a hardware structure diagram of a mobile phone 100 is shown.

[0115] like Figure 9 As shown, the mobile phone 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal USB interface 130, a charging management module 140, a power management module 141, a battery module 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0116] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, etc. These different processing units may be independent devices or integrated into one or more processors.

[0117] The controller can serve as the central nervous system and command center of the mobile phone 100. The controller can generate operation control signals based on the instruction operation code and timing signals to control the fetching and execution of instructions. Specifically, the image processing or shooting method executed by the mobile phone 100 in this embodiment can be executed by the processor 110.

[0118] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include inter-integrated circuit (I2C) interfaces, inter-integrated circuit sound (I2S) interfaces, pulse code modulation (PCM) interfaces, etc. It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation of the mobile phone 100.

[0119] The charging management module 140 receives charging input from the charger. The power management module 141 connects to the battery module 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery module 142 and / or the charging management module 140 to power the processor 110, internal memory 121, external memory, display 194, etc.

[0120] The wireless communication function of mobile phone 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0121] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Mobile communication module 150 can provide wireless communication solutions for mobile phone 100, including 2G / 3G / 4G / 5G. Wireless communication module 160 can provide wireless communication solutions for mobile phone 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), and global navigation satellite system (GNSS).

[0122] The mobile phone 100 uses a GPU, a display screen 194, and an application processor to achieve its display function. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor.

[0123] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the mobile phone 100 may include one or N displays screens 194, where N is a positive integer greater than 1.

[0124] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the mobile phone 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions.

[0125] The internal memory 121 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the mobile phone 100 by running the instructions stored in the internal memory 121.

[0126] Mobile phone 100 can achieve audio functions through audio module 170, such as music playback and recording.

[0127] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. A motor 191 can generate vibration feedback. An indicator 192 can be an indicator light. A SIM card interface 195 can be used to connect a SIM card.

[0128] It is understood that the structure illustrated in this application does not constitute a specific limitation on the mobile phone 100. In other embodiments, the mobile phone 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0129] In some embodiments, this application also provides a computer-readable medium storing program code that, when run on a computer, causes the computer to perform the methods described above.

[0130] In some embodiments, this application also provides a computer program product comprising: computer program code that, when run on a computer, causes the computer to perform the methods described above.

[0131] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0132] It should be noted that the units / modules mentioned in the various device embodiments of the present invention are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problem proposed by the present invention. Furthermore, to highlight the innovative aspects of the present invention, the above-described device embodiments of the present invention have not introduced units / modules that are not closely related to solving the technical problem proposed by the present invention. This does not mean that the above-described device embodiments do not contain other units / modules.

[0133] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0134] Although the invention has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made therein without departing from the scope of the invention.

Claims

1. An image processing method applied to electronic devices, characterized in that, include: Acquire multiple frames of images to be fused, wherein the multiple frames of images to be fused include the same captured elements; The multi-frame images to be fused are determined to include a first region and a second region. The first region includes a first shooting element whose position changes in different images to be fused. The second region includes a second shooting element whose brightness meets a first brightness condition. The first shooting element and the second shooting element may be the same or different. A first image is selected from the multiple frames to be fused, and the brightness of the first image is increased to obtain a first target image, wherein the exposure value of the first image is less than the exposure value of the second image in the multiple frames to be fused.

2. The method according to claim 1, characterized in that, The first brightness condition includes at least one of the following: The brightness of each pixel in the second region is greater than or equal to the first brightness threshold; The proportion of pixels with brightness greater than or equal to the first brightness threshold in the second region is greater than the first proportion.

3. The method according to claim 1, characterized in that, The step of selecting a first image from the multiple frames to be fused and increasing the brightness of the first image to obtain a first target image includes: The image with the lowest exposure value among the multiple frames to be fused is selected as the first image. Increase the brightness of the first image to obtain the first intermediate image; Based on the first image, the second image is subjected to deformation alignment processing to obtain a second intermediate image, wherein each pixel in the second intermediate image corresponds one-to-one with each pixel at the corresponding position in the first image; The first intermediate image is denoised based on the second intermediate image to obtain the first target image.

4. The method according to claim 3, characterized in that, The step of performing noise reduction processing on the first intermediate image based on the second intermediate image to obtain the first target image includes: The first intermediate image is denoised based on the pixel information of the second intermediate image to obtain a third intermediate image, wherein the pixel information includes pixel value and / or brightness; The third intermediate image is subjected to tone mapping processing to obtain the first target image, wherein the bit depth of the first target image is less than the bit depth of the third intermediate image.

5. The method according to any one of claims 1 to 4, characterized in that, The step of selecting a first image from the multiple frames to be fused and increasing the brightness of the first image to obtain a first target image includes: If the size of the region corresponding to the first region is greater than or equal to the first region threshold, and the region features of the second region satisfy the first feature condition, then the first image is selected from the multiple frames of images to be fused, and the brightness of the first image is increased to obtain the first target image. The first feature condition includes: the size of the second region is greater than or equal to the threshold of the second region, and / or the region type of the second region is a preset type.

6. The method according to claim 5, characterized in that, The method further includes: If the size of the region corresponding to the first region is less than the threshold of the first region, the multi-frame images to be fused are fused to obtain the second target image.

7. The method according to claim 5, characterized in that, The method further includes: If the regional features corresponding to the second region do not satisfy the first feature condition, a third target image is obtained based on the second image.

8. A shooting method applied to an electronic device, characterized in that, include: The user's shooting command is detected, and multiple frames of images to be fused are acquired. Based on the image processing method according to any one of claims 1 to 7, the multi-frame images to be fused are processed to obtain a processed result image; The image showing the processing result is displayed.

9. A computer-readable storage medium, characterized in that, The readable storage medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 8.

10. An electronic device, characterized in that, include: Memory is used to store instructions executed by one or more processors of an electronic device; And a processor, one of the processors of the electronic device, for executing instructions stored in the memory to implement the method of any one of claims 1 to 8.

11. A program product, characterized in that, The program product includes instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 8.

Images