Image processing method and device, and storage medium

Abstract

The invention discloses an image processing method and device, and a storage medium. The method comprises the steps of obtaining a to-be-processed image; identifying a target area of the to-be-processed image to obtain a first weight map, the first weight map comprising a first weight corresponding to each pixel in the to-be-processed image, and the first weight representing a probability of whether the corresponding pixel belongs to the target area; performing noise reduction processing on the to-be-processed image to obtain a first image; and performing fusion processing on the to-be-processed image, the first weight map and the first image to obtain a target image.

Description

Technical Field

[0001] The present application relates to image processing technology, and in particular to an image processing method and device, and a storage medium. Background Art

[0002] In the related art, the following method is used to deal with the problems of purple and green edges, highlight edge artifacts and low-frequency color noise in the image: for the original image input in YUV format, the pixel position to be processed is first detected based on the sampled small-size original image, and then the corresponding pixel position of the original image is processed by filtering and other methods, so as to eliminate the problems of purple and green edges, highlight edge artifacts and low-frequency color noise in the original image. The above detection process is performed on the sampled small-scale image. After the detection, the detection result is upsampled to obtain the pixel position to be processed in the original image. However, due to the limitation of the detection accuracy on the small scale, the detection is usually not accurate enough, which will cause some pixels that do not need to be processed to be processed incorrectly or pixels that need to be processed are not detected, and the pixels are either processed or not processed, resulting in a lack of transition, which will lead to a serious decline in the quality of the processed image, and even worse than the unprocessed effect in terms of naked eye perception. Summary of the invention

[0003] The embodiments of the present application provide an image processing method and device, and a storage medium, which can improve image quality.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides an image processing method, the method comprising:

[0006] Get the image to be processed;

[0007] Identify the target area of ​​the image to be processed, and obtain a first weight map, wherein the first weight map includes a first weight corresponding to each pixel in the image to be processed, and the first weight represents the probability of whether the corresponding pixel belongs to the target area;

[0008] Performing noise reduction processing on the image to be processed to obtain a first image;

[0009] The image to be processed is fused with the first weight map and the first image to obtain a target image.

[0010] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0011] An acquisition unit, configured to acquire an image to be processed;

[0012] an identification unit configured to identify a target area of ​​the image to be processed and obtain a first weight map, wherein the first weight map includes a first weight corresponding to each pixel in the image to be processed, and the first weight represents a probability of whether the corresponding pixel belongs to the target area;

[0013] A noise reduction unit, configured to perform noise reduction processing on the image to be processed to obtain a first image;

[0014] The fusion unit is configured to fuse the image to be processed with the first weight map and the first image to obtain a target image.

[0015] An embodiment of the present application provides an electronic device, including a processor, wherein the processor is configured to:

[0016] Get the image to be processed;

[0017] Identify the target area of ​​the image to be processed, and obtain a first weight map, wherein the first weight map includes a first weight corresponding to each pixel in the image to be processed, and the first weight represents the probability of whether the corresponding pixel belongs to the target area;

[0018] Performing noise reduction processing on the image to be processed to obtain a first image;

[0019] The image to be processed is fused with the first weight map and the first image to obtain a target image.

[0020] An embodiment of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the above-mentioned image processing method are implemented.

[0021] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned image processing method is implemented.

[0022] The chip provided in the embodiment of the present application is used to implement the above-mentioned image processing method, and the chip includes: a processor, which is used to call and run a computer program from a memory, so that a device equipped with the chip executes the above-mentioned image processing method.

[0023] The image processing method, device, and storage medium provided in the embodiments of the present application identify the target area of ​​the image to be processed, determine a first weight map representing the position of the target area in the image to be processed, and perform noise reduction processing on the image to be processed to obtain a first image, and fuse the first image with the image to be processed according to the first weight map; since the target image is a fusion of the image to be processed and the first image after noise reduction based on the first weight map, that is, the content of each pixel is a fusion of the image to be processed and the corresponding pixel in the first image based on the value of the corresponding pixel in the first weight map, the influence on the pixels outside the target area that do not need to be processed is better reduced, thereby smoothly eliminating the image noise included in the image to be processed and improving the image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is an optional flowchart of the image processing method provided in the embodiment of the present application;

[0025] Figure 2 is a schematic diagram of image fusion provided in an embodiment of the present application;

[0026] Figure 3 is a schematic diagram of the relationship between smoothing and upsampling provided in an embodiment of the present application;

[0027] Figure 4 is a schematic diagram of the relationship between smoothing and upsampling provided in an embodiment of the present application;

[0028] Figure 5 It is an optional flowchart of the image processing method provided in the embodiment of the present application;

[0029] Figure 6 It is an optional flowchart of the image processing method provided in the embodiment of the present application;

[0030] Figure 7 This is a schematic diagram of the comparison before and after eliminating the purple fringing effect provided in the embodiment of the present application;

[0031] Figure 8 This is a schematic diagram of the comparison before and after the green edge elimination effect provided in the embodiment of the present application;

[0032] Fig. 9 It is an optional flowchart of the image processing method provided in the embodiment of the present application;

[0033] Fig.10 is a schematic diagram of comparison before and after highlight artifact processing provided in an embodiment of the present application;

[0034] Fig.11 It is an optional flowchart of the image processing method provided in the embodiment of the present application;

[0035] Fig.12 It is a schematic diagram of the comparison of an image before and after low-frequency color noise removal provided in an embodiment of the present application;

[0036] Fig.13 is an optional schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0037] Fig.14 is an optional schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0038] Fig.15 is an optional schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0039] Fig.16 It is an optional schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0041] The embodiments of the present application can provide an image processing method, device and storage medium. In practical applications, the image processing method can be implemented by an electronic device, and each functional entity in the electronic device can be collaboratively implemented by hardware resources of a computer device (such as a terminal device, a network device, etc.), such as computing resources such as a processor, and communication resources (such as for supporting various communication methods such as optical cables and cellular).

[0042] Of course, the embodiments of the present application are not limited to being provided as methods and hardware, and there may be multiple implementation methods, such as being provided as a storage medium (storing instructions for executing the image processing method provided by the embodiments of the present application).

[0043] Below, various embodiments of the image processing method, device, and storage medium provided in the embodiments of the present application are described.

[0044] The image processing method provided in the embodiment of the present application can be as follows Figure 1 As shown, the method may include:

[0045] S101: The electronic device obtains an image to be processed.

[0046] The image to be processed is an image that has been pre-processed. In this case, the image processing method provided in the embodiment of the present application can be understood as post-processing of the image to be processed.

[0047] The image to be processed may be an RGB image or a YUV image, or an image processed by high dynamic range, tone mapping, etc.

[0048] In an embodiment of the present application, the image to be processed may be an image directly captured by the electronic device or a frame of image in a captured video stream, or it may be an image selected from multiple images stored in the electronic device based on a user's operation or a frame of image selected from a video stream stored in the electronic device, wherein the multiple images or video streams stored in the electronic device are captured by the electronic device itself or received through a network.

[0049] S102, the electronic device identifies the target area of ​​the image to be processed and obtains a first weight map, where the first weight map includes a first weight corresponding to each pixel in the image to be processed, and the first weight represents the probability of whether the corresponding pixel belongs to the target area.

[0050] The electronic device determines the target area based on the image problem to be solved and identifies the target area in the image to be processed. The target area in the embodiment of the present application may include a chroma noise area, a brightness noise area, a frequency noise area, etc., wherein the chroma noise area includes the area where the chroma noise caused by chroma such as green edge, purple edge, blue edge, etc. is located, the brightness noise area includes the area where the brightness noise caused by brightness such as highlight artifact, highlight edge jagged, highlight edge false color is located, and the frequency noise includes the area where the frequency noise caused by frequency such as low-frequency color noise and polar night scene color noise is located.

[0051] In an embodiment of the present application, the target area may include one or more of the following: a green edge area, a purple edge area, a blue edge area, a highlight edge artifact area, a highlight edge jagged area, a highlight edge false color area, a low-frequency color noise area, a polar night scene color noise area with extremely low signal-to-noise ratio, etc.

[0052] After the electronic device identifies the target area in the image to be processed, it obtains a first weight map. The size of the first weight map is the same as the size of the image to be processed. The first weight map includes the first weight corresponding to each pixel in the image to be processed. The first weight corresponding to each pixel is the probability of whether the corresponding pixel in the image to be processed belongs to the target area, that is, the first probability. It can be understood that for the first weight corresponding to a pixel in the first weight map, the value of the first weight increases as the possibility that the pixel belongs to the target area increases. In one example, the value range of the first weight is 0 to 1, 0 indicates that the corresponding pixel does not belong to the target area, 1 indicates that the corresponding pixel belongs to the target area, and 0.5 indicates that the corresponding pixel may belong to the target area.

[0053] In an embodiment of the present application, the electronic device can directly identify the probability that each pixel in the image to be processed belongs to the target area, or it can downsample the image to be processed and identify the probability that each pixel in the image to be processed belongs to the target area based on the downsampled small-size image.

[0054] The electronic device can identify the target area in the image to be processed based on the image feature of the target area, that is, the first image feature. For example, for purple edge, green edge or blue edge, the first image feature used to identify the target area in the image to be processed is purple color information, green color information or blue color information. For another example, for highlight edge artifacts, highlight edge jaggedness, and highlight edge false color, the first image feature used to identify the target area in the image to be processed is the image grayscale value. For another example, for low-frequency color noise, polar night scene color noise, etc., the first image feature used to identify the target area in the image to be processed is a frequency feature, that is, a frequency feature.

[0055] In S102, the electronic device recognizes the target area on the image to be processed. The target area can be recognized directly on the image to be processed, or the target area can be recognized based on the second image after downsampling the image to be processed to obtain a small-sized second image.

[0056] S103: The electronic device performs noise reduction processing on the image to be processed to obtain a first image.

[0057] In addition to identifying the target area of ​​the image to be processed, the electronic device also performs noise reduction processing on the image to be processed. Here, the image content of the image that affects the image quality in the target area can be regarded as image noise that affects the image quality. Through the noise reduction processing, the image noise in the image to be processed is reduced to obtain a first image, that is, the first image can be understood as the image of the image to be processed after the image noise is eliminated.

[0058] In the embodiments of the present application, different problems are solved, that is, different target areas are eliminated, and different image noises are eliminated. For the green edge area, the noise reduced is the green edge; for the purple edge area, the noise eliminated is the purple edge; for the blue edge area, the noise eliminated is the blue edge; for the highlight edge artifact area, the noise eliminated is the highlight edge artifact; for the highlight edge aliasing area, the noise eliminated is the highlight edge aliasing; for the highlight edge false color area, the noise eliminated is the highlight edge false color; for the low-frequency color noise area, the noise eliminated is the low-frequency color noise; for the polar night scene color noise area with extremely low signal-to-noise ratio, the noise eliminated is the polar night scene color noise.

[0059] In an embodiment of the present application, the denoising method used for denoising the image to be processed may include one or more of the following: guided filtering, Gaussian filtering, etc. The embodiment of the present application does not impose any limitation on the denoising method used for denoising the image to be processed.

[0060] In S103, the electronic device performs noise reduction processing on the image to be processed. The noise reduction processing may be performed directly on the image to be processed, or the image to be processed may be downsampled to obtain a second image of a small size and the noise reduction processing may be performed based on the second image.

[0061] It should be noted that when the electronic device performs noise reduction processing, the noise reduction is performed on the entire area of ​​the image to be processed, so that there may be locations where noise reduction is not required.

[0062] S104: The electronic device fuses the image to be processed, the first weight map and the first image to obtain a target image.

[0063] Here, the electronic device fuses the first image with the image to be processed according to the first weight map to obtain a target image.

[0064] The weight corresponding to each pixel in the first weight map represents the probability that the corresponding pixel belongs to the target area, so that when the first image after the denoising process and the image to be processed that has not been denoised are fused, for a pixel, the proportion of the content of the corresponding pixel in the first image in the fusion result, i.e., the target image, is determined based on the value of the weight of the pixel in the first weight map. For example: for a pixel A, the value of pixel A in the first weight map is 0, which represents that pixel A does not belong to the target area. Then, in the image content of pixel A in the target image, the content of pixel A in the first image accounts for 0, and the content of pixel A in the image to be processed accounts for 1, that is, pixel A in the target image does not include the content of pixel A in the first image, but only includes the image content of pixel A in the image to be processed. For another example: for a pixel B, the value of pixel B in the first weight map is 1, which represents that pixel B belongs to the target area. Then, in the image content of pixel B in the target image, the content of pixel B in the first image accounts for 1, and the content of pixel B in the image to be processed accounts for 0, that is, pixel B in the target image only includes the content of pixel B in the first image, but does not include the image content of pixel B in the image to be processed. For another example: for a pixel C, the value of pixel C in the first weight map is 0.5, which represents that pixel C belongs to the target area. Then, in the image content of pixel C in the target image, the content of pixel C in the first image accounts for 0.5, and the content of pixel C in the image to be processed accounts for 0.5, that is, pixel C in the target image only includes the content of pixel C in the first image, and does not include the image content of pixel C in the image to be processed.

[0065] In practical applications, the matching weight map corresponding to the first weight map can be determined, the size of the first weight map is the same as the size of the matching weight map, and the sum of the weights of the corresponding positions in the first weight map and the matching weight map is 1. The electronic device uses the first weight map as the weight map of the first image and the matching weight map as the weight map of the image to be processed, and fuses the first image and the image to be processed.

[0066] In one example, if Figure 2 As shown, the weight map 201 is a first weight map for the image 203 , and the image 202 is a first image after the noise of the image 203 is reduced. The weight map 201 is fused with the image 202 and the image 203 to obtain the target image 204 .

[0067] The image processing method provided in the embodiment of the present application can process one or more problems existing in the image. When the electronic device processes multiple problems, it can process multiple problems in series, that is, after one problem is processed, it continues to process the next problem based on the target image obtained after the problem is processed. It can also process multiple problems in parallel, that is, simultaneously identify the areas of multiple problems and perform noise reduction on multiple problems, and fuse the recognition results (first weight map) and the noise reduction results (first image) with the image to be processed. For example: if the image to be processed has both green edge and blue edge problems, in the case of serial processing, S102 to S104 are executed for the green edge problem, and S102 to S104 are executed for the blue edge problem. For another example, if the image to be processed has both green and blue edges, and parallel processing is performed, the target area identified when determining the first weight map in S102 includes the green edge area and the blue edge area, that is, a weight in the first weight map represents the probability that the pixel belongs to the green edge or the blue edge, and denoising of the blue and green edges is performed in S103 to obtain the first image, and the first weight map obtained is fused with the first image obtained in S103 and the image to be processed.

[0068] The image processing method provided by the embodiment of the present application obtains an image to be processed; identifies the target area of ​​the image to be processed, obtains a first weight map, the first weight map includes a first weight corresponding to each pixel in the image to be processed, and the first weight represents the probability of whether the corresponding pixel belongs to the target area; performs noise reduction processing on the image to be processed to obtain a first image; and fuses the image to be processed with the first weight map and the first image to obtain a target image. Since the target image is a fusion of the image to be processed and the first image after noise reduction based on the first weight map, that is, the content of each pixel is a fusion of the image to be processed and the corresponding pixel in the first image based on the value of the corresponding pixel in the first weight map, the influence on the pixels outside the target area that do not need to be processed is better reduced, thereby smoothly eliminating the image noise included in the image to be processed and improving the image quality.

[0069] In some embodiments, the step of identifying the target area of ​​the image to be processed and obtaining the first weight map in step S102 includes:

[0070] S1021, detecting a target area of ​​a second image to obtain a second weight map, wherein the second image is obtained by downsampling the image to be processed, and the second weight map includes a second weight corresponding to each pixel in the second image, and the second weight represents a probability of whether the corresponding pixel belongs to the target area;

[0071] S1022: Perform upsampling and smoothing processing on the second weight map to obtain the first weight map, wherein a size of the first weight map is the same as a size of the image to be processed.

[0072] Here, the electronic device downsamples the image to be processed to obtain a small-sized second image, and identifies the target area based on the second image. It can be understood that the small size of the second image is small relative to the image to be processed, that is, the size of the second image is smaller than the size of the image to be processed.

[0073] The downsampling method used by the electronic device for downsampling may include first-rate downsampling or Gaussian pyramid downsampling, etc. The first rate may be 3, 4, 5, etc., determined according to the size of the image to be processed.

[0074] After the electronic device obtains the second image, it detects the target area of ​​the second image and obtains a second weight map (mask map) of the same size as the second image, wherein the second weight map includes a second weight corresponding to each pixel in the second image, and the second weight represents the probability that the corresponding pixel in the second image is the target area, i.e., the second probability. The electronic device can detect the target area of ​​the second image through the first image feature, wherein different first image features are used based on different target areas to be detected.

[0075] After the electronic device determines the second weight map corresponding to the second image, it performs upsampling and smoothing on the second weight map. The order of upsampling and smoothing is not limited. Figure 3 As shown, the electronic device may upsample the second weight map 301 to obtain the third weight map 302, and perform smoothing on the third weight map 302 to obtain the first weight map 303. In one example, Figure 4 As shown, the electronic device may perform smoothing processing on the second weight map 301 to obtain a fourth weight map 304 , and upsample the fourth weight map 304 to obtain the first weight map 303 .

[0076] In the embodiment of the present application, the amount of computation required for smoothing the upsampled image is much greater than the amount of computation required for upsampling after smoothing. Therefore, if power consumption is considered, the embodiment of the present application can obtain the first weight map by smoothing first and then upsampling.

[0077] The upsampling method of the electronic device may include: an upsampling method of a first magnification, Laplacian pyramid reconstruction, etc. In the embodiment of the present application, the upsampling method adopted by the electronic device may be determined based on the downsampling method. In one example, the electronic device performs upsampling based on the upsampling method of the first magnification to obtain a second image by downsampling based on the downsampling method of the first magnification. In one example, the electronic device performs upsampling based on the upsampling method of the first magnification to obtain a second image by downsampling based on Gaussian pyramid downsampling, and performs upsampling based on the upsampling method of Laplacian pyramid reconstruction.

[0078] In an embodiment of the present application, the target area is detected on a small-sized second image after downsampling the processed image, and in the process of obtaining the first weight map based on the second weight map corresponding to the second image, the second weight map is upsampled and smoothed. While reducing the size of the image for target detection, it is ensured that the pixels in the obtained first weight map are smooth, thereby ensuring that the fused target image is also smooth, thereby improving the image quality.

[0079] In some embodiments, the step S1021 of detecting the target area of ​​the second image and obtaining the second weight map includes:

[0080] Based on the first image feature, a first area in the second image is detected, the first area includes a second area of ​​the target area and a third area outside the target area; based on the second image feature, a position of the third area in the first area is detected, the first image feature and the second image feature are different image features; based on the position of the third area, a position of the second area in the second image is determined, and based on the position of the second area in the second image, the second weight map is determined.

[0081] In an embodiment of the present application, when the target area is identified by using the first image feature, the first area identified in the second image may also include an area outside the target area in addition to the second area of ​​the target area. Here, the area outside the target area in the identified first area is referred to as a third area. The electronic device further identifies the third area from the identified first area by using the second image feature, and excludes the third area from the first area based on the position of the third area to obtain the position of the second area in the second image, so that the identified area only includes the second area of ​​the target area.

[0082] The first image feature is an image feature shared by the second region and the third region, and the second image feature is an image feature that can only identify the third region but cannot identify the second region.

[0083] In one example, for the problem of green or purple edges, the first image feature is color information, and the second image feature is gradient. At this time, the first area identified by the electronic device from the second image through color information includes not only green or purple edges, but also normal green areas or purple areas that do not need to be eliminated. The electronic device identifies the normal green or purple areas that do not need to be eliminated through the gradient of each pixel, thereby excluding the normal green or purple areas that do not need to be eliminated from the first area, leaving only the green or purple edges that need to be eliminated. Among them, a first gradient threshold can be set to determine that the pixels in the first area whose gradient is greater than the first gradient threshold are normal pixels in the green or purple areas that do not need to be eliminated.

[0084] In one example, for the problem of highlight edge artifacts, the first image feature is grayscale, and the second image feature is brightness. At this time, the first area identified by the electronic device from the second image through grayscale information includes not only the highlight edge, but also the normal highlight area that does not need to be eliminated. The electronic device identifies the highlight area through the brightness of each pixel, thereby excluding the highlight area from the identified first area, leaving only the highlight edge area that needs to be eliminated. Among them, a brightness threshold can be set to determine that the pixels in the first area whose aggregate brightness is greater than the brightness threshold are pixels in the highlight area.

[0085] In the embodiment of the present application, if the target area is directly identified in the image to be processed, the electronic device further performs the following processing:

[0086] Based on the first image feature, a fourth area in the image to be processed is detected, and the fourth area includes the target area and a fifth area outside the target area; based on the second image feature, the fifth area in the fourth area is detected, and the first image feature and the second image feature are different image features; the area in the fourth area except the fifth area is determined as the target area, and the first weight map is determined based on the target area.

[0087] Here, the selection of the first image feature and the second image feature may refer to the identification of the first region and the identification of the third region, which will not be described in detail here.

[0088] In some embodiments, S103 performs noise reduction processing on the image to be processed to obtain a first image, including:

[0089] S1031, filtering the second image to obtain a third image, where the second image is a downsampling result of the image to be processed;

[0090] S1032: Reconstruct the third image to obtain the first image, where the size of the first image is the same as the size of the image to be processed.

[0091] Here, the electronic device downsamples the image to be processed to obtain a small-sized second image, and performs noise reduction processing based on the second image. It can be understood that the small size of the small-sized second image means that the size of the second image is small relative to the image to be processed.

[0092] The downsampling method adopted by the electronic device for downsampling may include first-rate downsampling or Gaussian pyramid downsampling, and the first rate may be 3, 4, 5, etc., determined according to the size of the image to be processed.

[0093] After obtaining the second image, the electronic device filters the second image to obtain a third image of the same size as the second image, wherein the third image can be understood as an image obtained after filtering the second image, that is, an image after eliminating image noise.

[0094] After the electronic device determines the third image, it reconstructs the third image to obtain a first image of the same size as the image to be processed. The way in which the electronic device reconstructs the first image may include: upsampling at a first magnification, Laplacian pyramid reconstruction, etc. In the embodiment of the present application, the reconstruction method adopted by the electronic device can be determined based on the downsampling method. In one example, the electronic device downsamples the second image obtained by the downsampling method at the first magnification, and then reconstructs the third image based on the upsampling method at the first magnification. In one example, the electronic device downsamples the second image obtained by the Gaussian pyramid downsampling, and then reconstructs the third image based on the upsampling method of the Laplacian pyramid reconstruction.

[0095] In the embodiment of the present application, noise reduction processing is performed on the small-sized second image after downsampling the processed image, which reduces the size of the noise-reduced image, that is, the resolution of the second image is low, so the filtering processing speed is fast, thereby improving the speed of image processing.

[0096] In some embodiments, before S1031, the electronic device may further perform the following processing:

[0097] Detecting a target area on the second image to obtain a second weight map, where the second image is a downsampling result of the image to be processed, and the second weight map includes a second probability of whether each pixel in the second image belongs to the target area;

[0098] The second image is weighted based on the second weight map to obtain an updated second image.

[0099] Here, before obtaining the second image and filtering the second image, the electronic device may first weight the second image through the second weight map, that is, multiply the second weight map and the second image, and at this time, the updated second image obtained after the multiplication is used as the filtering object for filtering. In the embodiment of the present application, the weights in the second weight map may be binarized, and the binarized second weight map may be multiplied with the second image, so as to filter out the area in the second image that needs to be processed, that is, the area belonging to the target area.

[0100] In an embodiment of the present application, before filtering the second image, the second image is multiplied by the second weight map, and the multiplication result is filtered. This is equivalent to filtering the area in the second image that needs to be processed, that is, the area belonging to the target area, through the second weight map, while other areas are not processed. Compared with the method of processing the entire second image, the filtering speed is accelerated, thereby further improving the image processing speed.

[0101] In some embodiments, if the image to be processed is an RGB image, the electronic device further performs the following processing:

[0102] Before filtering the second image to obtain the third image, performing format conversion on the second image to convert the second image into a YUV image;

[0103] After filtering the second image to obtain the third image, format conversion is performed on the third image to convert the third image into an RGB image.

[0104] In an embodiment of the present application, when the problem to be solved is unrelated to texture information and thus texture information needs to be retained during filtering, the second image can be converted from an RGB image to a YUV image, and the second image in the YUV format can be filtered to obtain a third image. At this time, the third image obtained is a YUV image, and the electronic device can perform format summons on the third image and convert the third image from a YUV image to an RGB image.

[0105] In one example, for the problem of green edge or purple edge, since the purple edge and green edge of the image are both chromatic noise, it is most suitable to perform noise reduction processing in the YUV domain. In order to obtain the best effect to maintain the edge characteristics of the image and use more brightness information, a guided filter is used to filter the second image in YUV format, and the information of the Y channel is used to guide the UV channel to perform a small-size guided filter to obtain a third image after filtering, and the third image is converted into an RGB image.

[0106] The image processing method provided in the embodiment of the present application performs filtering processing after converting the RGB image into a YUV image, so that when performing image filtering, texture information, brightness information and other information in the image that are not related to chromaticity information are retained to the greatest extent, thereby improving the image quality of the target image after image processing.

[0107] The image processing method provided in the embodiment of the present application, in the process of determining the first weight map and the first image based on the image to be processed, can be implemented in at least one of the following scenarios:

[0108] Scenario 1: directly identify the target area of ​​the image to be processed, and directly perform filtering on the image to be processed;

[0109] Scenario 2: down-sampling the image to be processed to obtain a second image, identifying the target area of ​​the second image to obtain a second weight map, up-sampling the second weight map, and directly performing filtering processing on the image to be processed;

[0110] Scenario 3: directly identifying the target area of ​​the image to be processed, downsampling the image to be processed to obtain a second image, filtering the second image to obtain a third image, and reconstructing the third image;

[0111] Scenario 4: down-sample the image to be processed to obtain a second image, identify the target area of ​​the second image to obtain a second weight map, up-sample the second weight map, filter the second image to obtain a third image, and reconstruct the third image.

[0112] In some embodiments, the electronic device further performs the following processing:

[0113] Acquire area type indication information, where the area type indication information is used to indicate the type of the target area;

[0114] A detection strategy and a filtering strategy are determined based on the region type indication information, wherein the detection strategy is used to indicate a detection method for the target region, and the filtering strategy is used to indicate a noise reduction method for the first image.

[0115] Here, the region type indication information indicates the region to be identified or the problem to be solved, wherein the region type information can be selected by the user according to the region type indication information provided by the electronic device or can be obtained by the electronic device through identification of the image to be processed.

[0116] The type of target area indicated by the area type indication information or the problem to be solved may include one or more of: green edge area, purple edge area, blue edge area, highlight edge artifact area, highlight edge jagged area, highlight edge false color area, low-frequency color noise area, polar night scene color noise area with extremely low signal-to-noise ratio, etc.

[0117] The electronic device determines the processing strategy corresponding to the currently obtained area type indication information based on the mapping relationship between the area type indication information and the processing strategy, wherein the processing strategy includes one or more of a downsampling strategy, a detection strategy, a filtering strategy, and an upsampling strategy. The downsampling strategy indicates the downsampling method used for downsampling to the second image. The detection strategy indicates the detection method used by the electronic device when identifying the target area. The filtering strategy indicates the noise reduction method used by the electronic device when performing noise reduction processing on the first image. The upsampling strategy indicates the upsampling method used when reconstructing the third image into the first image.

[0118] In one example, for the green edge or purple edge problem, the detection strategy is to detect through color information, and the filtering strategy is Gaussian filtering.

[0119] In one example, for the green edge or purple edge problem, the downsampling method is four times downsampling, the detection strategy is detection through color information, the filtering strategy is Gaussian filtering, and the upsampling method is four times upsampling.

[0120] In one example, to address the problem of highlight edge artifacts, the detection strategy is to detect through grayscale information, and the filtering strategy is Gaussian filtering.

[0121] In one example, to address the low-frequency color noise problem, the downsampling method is Gaussian pyramid sampling, the detection strategy is to determine the flat area as the target area from the low-frequency image layer of the Gaussian pyramid based on a set second gradient threshold, the filtering strategy is to select the minimum-sized layer according to the number of layers of the Gaussian pyramid, and the upsampling strategy is Laplacian pyramid reconstruction.

[0122] It should be noted that the Gaussian pyramid is used for downsampling of the second image. First, the image to be processed is taken as the bottom image level 0 (the 0th layer of the Gaussian pyramid), and it is convolved with a Gaussian kernel. Then, the convolved image is downsampled (even rows and columns are removed) to obtain the previous layer image, that is, the previous layer G1. This image is used as input, and the convolution and downsampling operations are repeated to obtain a higher layer. This is repeated many times to form a pyramid-shaped image data structure, namely the Gaussian pyramid. The Laplacian pyramid is used to reconstruct the first image.

[0123] The image processing method provided in the embodiment of the present application is applicable to the problems of image purple and green edges, highlight edge artifacts and low-frequency color noise. Similar problems such as image blue edges, highlight edge jagged edges, highlight edge false colors or color noise in polar night scenes with extremely low signal-to-noise ratio can be solved using the method in the same process. For different problems, corresponding specific processing methods can be provided based on the problems to be solved under the same process architecture.

[0124] In some embodiments, if the type of the target area is a chromatic noise area, the denoising process is performed on the image to be processed to obtain a first image, including: converting the image to be processed from the RGB domain to the YUV domain to obtain a YUV image to be processed; performing guided filtering on the YUV image to be processed through information of the Y channel to obtain a filtered YUV image; converting the filtered YUV image from the YUV domain to the RGB domain to obtain the first image.

[0125] For scenes where the target area is a green-edge area, a purple-edge area, a blue-edge area, or other chromatic noise area, the electronic device can convert the image to be processed in RGB format into a YUV format, thereby filtering the image to be processed in the YUV domain. When filtering the image to be processed in the YUV domain, the electronic device uses guided filtering to filter the image to be processed in the YUV format, uses the information of the Y channel to guide the UV channel to perform small-size guided filtering, obtains the filtered image, and converts the filtered image into an RGB image.

[0126] In practical applications, the electronic device can downsample the image to be processed to obtain a second image, perform guided filtering on the second image in the YUV domain to obtain a third image in the YUV domain, upsample the third image in the YUV domain to obtain a first image in the YUV domain, and perform format conversion on the first image in the YUV domain to obtain a first image in RDB format.

[0127] The image processing method provided in the embodiment of the present application can maintain the edge characteristics of the image when reducing chromatic noise and obtain a high-quality image.

[0128] In some embodiments, if the type of the target area is a luminance noise area, performing noise reduction processing on the image to be processed to obtain the first image includes: filtering the image to be processed using a Gaussian filter to obtain the first image.

[0129] In some embodiments, if the type of the target area is a frequency noise area, the denoising process is performed on the image to be processed to obtain a first image, including: obtaining a low-frequency feature image in the first image, filtering the low-frequency feature image using a Gaussian filter, and reconstructing the filtered image to obtain the first image.

[0130] The electronic device can obtain an image tower of a multi-layer image by downsampling the Gaussian pyramid, and use the smallest image in the image tower as a low-frequency feature image, filter the image using a Gaussian filter, filter out noise in the low-frequency image, obtain a low-frequency image that does not include noise, and reconstruct the low-frequency image that does not include noise to obtain a first image.

[0131] In some embodiments, after S101 acquires the image to be processed, the electronic device further performs the following processing: receiving a restoration request; and performing artificial intelligence AI restoration on the image to be processed based on the restoration request.

[0132] The restoration request in the electronic device can be automatically triggered by the electronic device or manually triggered by the user. For example, after the electronic device obtains the image to be processed, the electronic device automatically triggers a restoration request to trigger AI restoration of the image to be processed. For another example, after the electronic device obtains the image to be processed, a control for triggering a modification request is provided on the display interface. When the control receives the user's operation, it considers that a restoration request operation has been received. The electronic device triggers the restoration request based on the restoration request operation to trigger AI restoration of the image to be processed.

[0133] The AI ​​restoration performed by the electronic device on the device to be processed can be understood as front-end image processing such as format conversion and compression of the image, thereby obtaining the image to be processed in RGB format.

[0134] The image processing method provided in the embodiment of the present application performs image processing on the RGB image restored by the front-end AI, thereby achieving image post-processing of the image to be processed and improving the image quality.

[0135] The following is a further description of the image processing method provided in the embodiments of the present application.

[0136] The image processing method provided in the embodiment of the present application proposes a post-processing method that takes an RGB image as input and fuses the original image and the processed image according to the weights obtained by detection. The detection is also performed on a small-scale image, and the detection results are smoothed to obtain the weights corresponding to each pixel, so that the weights corresponding to each pixel can ensure the detection accuracy that meets the needs of the mobile terminal, and can solve the problems of purple fringing, green fringing, highlight edge artifacts and low-frequency color noise of the corresponding pixel points. The transition between the processing area and the adjacent area is also very natural, and there will be no false detection or missed detection that seriously affects the image quality, and the overall image quality is further improved.

[0137] The processing flow of the image processing method provided in the embodiment of the present application is as follows: Figure 5As shown, the RGB image obtained after the front-end AI image restoration is directly used as the input original image 501, i.e., the image to be processed. The original image 501 is downsampled to obtain a small-size image 502, i.e., the second image. The small-size image 502 is respectively detected and filtered to obtain an image mask 503, i.e., the second weight map, of the pixel area to be processed and a filtered image 504, i.e., the third image after filtering. The image mask 503 is sequentially upsampled and smoothed to obtain a weight map 505, i.e., the first weight map. The filtered image 504 is upsampled to obtain a processed image 506, i.e., the first image, which has the same resolution as the original image 501. The processed image 506 and the original image 501 are fused in the RGB domain according to the weight map 505 to obtain a result image 507, i.e., the target image. The electronic device can give the result image to a subsequent image processing engine for the next step of processing.

[0138] In the entire Image Signal Processing (ISP) process, for the RGB image restored by the front-end AI image, it will first be downsampled 4 times or more to obtain a small-size image, and detection and filtering will be performed on the small-scale image. Among them, for the image obtained by downsampling, corresponding detection and filtering strategies will be used for detection and filtering according to the different problems to be solved. For detection, the grayscale image obtained after detection is the mask of the pixel area that needs to be processed in the downsampled image, and in order to make the result of the subsequent fusion look smooth, the mask obtained by detection is smoothed by Gaussian filtering or other filtering methods; for filtering, filtering is performed on the downsampled image, and for some specific problems, it may be converted from the RGB domain to the YUV domain for processing. The advantage of filtering the downsampled image is that the downsampled image has a lower resolution, so the filtering process is faster, which will not bring challenges to the fast, stable and economical operation of the mobile terminal. In the embodiment of the present application, if faster processing is required, the detected mask can also be binarized and multiplied with the downsampled image, and the filtering process is performed on the result image obtained by multiplication.

[0139] The smoothed mask will use image upsampling to keep the resolution consistent with the original image, and the filtered image will also use the same upsampling method to obtain an image with the same resolution.

[0140] Finally, the processed image and the original image are fused using the obtained weight map of the same resolution as the weight to obtain an RGB image that can be output to the downstream.

[0141] The image processing method provided in the embodiments of the present application can be applied to one or more of the following scenarios: Scenario 1, post-processing part for purple and green edges of the image; Scenario 2, post-processing for highlight edge artifacts; Scenario 3, post-processing for low-frequency color noise of the image.

[0142] The following describes the above three scenarios respectively.

[0143] Scenario 1: Post-processing of purple and green edges of images

[0144] like Figure 6 As shown, it includes: performing four-fold downsampling of the RGB image 601 after AI image restoration to obtain a downsampled image 602, performing purple edge and green edge area detection on the downsampled image 602 to obtain a mask image 603, and performing format conversion and filtering on the downsampled image 602 to obtain a filtered image 604, wherein the downsampled image is converted from an RGB image to a YUV image, and the YUV image is filtered, and the filtered filtered image is format converted from a YUV image to an RGB image; upsampling and smoothing the mask image 603 to obtain a weight map 605, performing four-fold upsampling on the filtered image 604 to obtain a processed image 606, and fusing the RGB image 601 after AI image restoration and the processed image based on the weight map 604 to obtain a result image 606. Among them,

[0145] Due to the four-fold downsampling and the power consumption limitation of mobile phones, detection and filtering need to be performed on the downsampled image. Taking into account the detection accuracy and the quality of the final result image, four-fold downsampling is adopted, and the downsampling method adopted is bilinear interpolation downsampling.

[0146] For the detection of purple and green edge areas, for the purple and green edge problems caused by optical hardware limitations, the sobel edge detection operator is first used to detect the edge of the image, and the purple or green edge of the image is screened out according to the color information in the detected edge. Since some purple and green edges do not normally need to be processed, these purple and green edges need to be excluded. It can be found that the purple and green edge areas that need to be processed have relatively high gradients. Therefore, the Gaussian difference operator (Difference of Gaussian, DoG) of each area can be combined to exclude the purple and green edges that do not need to be processed, thereby obtaining the mask map of the purple or green edge areas that need to be processed.

[0147] For format conversion and filtering, since the purple and green edges of the image are both chromatic noise, it is best to process them in the YUV domain. In order to obtain the best effect and maintain the edge characteristics of the image while using more brightness information, a guided filter is used to filter the image, using the information of the Y channel to guide the UV channel to perform a small-size guided filter, and the filtered image is converted to an RGB image.

[0148] For smoothing and four-fold upsampling, in order to ensure the smoothness of the area during fusion, the mask image obtained by detection needs to be smoothed. The smoothing is done by Gaussian filtering. The smoothed mask image and the filtered image are both four-fold upsampled using bilinear interpolation to obtain a weight map and a processed image with the same resolution as the original image.

[0149] For fusion, weights are assigned according to the obtained weight map, and the processed image and the original image are fused to finally obtain the result image after removing the purple and green edges of the image. The comparison of the results before and after removing the purple and green edges is shown in the figure below. Figure 7 , Figure 8 As shown. Figure 7 In the figure, image 701 is the image to be processed. The intersection of white and leaves in image 701 is purple (grayscale processed). After the purple edge is removed, image 702 no longer has purple edge. Figure 8 In the figure, image 801 is the image to be processed. The edge of the text in image 801 has a green border (grayscale processing has been performed). After the green border is removed, the green border no longer exists in image 802.

[0150] Scene 2: Post-processing of highlight edge artifacts

[0151] like Fig. 9 As shown, it includes: performing four-fold downsampling on the RGB image 901 after AI image restoration to obtain a downsampled image 902, performing highlight edge area detection on the downsampled image 902 to obtain a mask image 903, and filtering the downsampled image 902 to obtain a filtered image 904; upsampling and smoothing the mask image to obtain a weight map 905, upsampling the filtered image 904 to obtain a processed image 906, and fusing the RGB image 901 after AI image restoration and the processed image 906 based on the weight map 905 to obtain a result image 907.

[0152] For four times downsampling, the RGB image after four times downsampling AI image restoration.

[0153] For highlight edge area detection, the image grayscale value is used for detection. Since the red, green, and blue channels all have highlights and highlight edge artifacts, it is necessary to perform grayscale value detection on these three channels separately. Grayscale values ​​higher than the set threshold are determined as highlight areas and highlight edge areas. After detecting the highlight and highlight edge areas, the highlight area needs to be detected and the highlight edge area is retained. The detection is performed based on the image brightness adaptive threshold.

[0154] For filtering, in order to smooth out the artifacts in the highlight edge area, the downsampled image is processed by Gaussian filtering to obtain a filtered image with the highlight edge artifacts removed.

[0155] For smoothing and upsampling, smoothing, upsampling weight maps and upsampling processed images.

[0156] For fusion, weights are assigned according to the weight map obtained at the same scale, and the processed image is fused with the input RGB image to finally obtain the result image with the highlight edge artifacts removed. The comparison of the results before and after processing is shown in the figure below. Fig.10 As shown, image 1001 is the image to be processed. There are obvious horizontal and vertical stripes next to the highlighted text in image 1001. After removing the highlight edge artifacts of the image, the highlight edge artifacts in image 1002 have been smoothed out.

[0157] Scenario 3: Post-processing of low-frequency color noise in images

[0158] like Fig.11 As shown, it includes: performing Gaussian pyramid down-sampling on the RGB image 1101 after AI image restoration to obtain a down-sampled image 1102, performing low-frequency color noise area detection on the down-sampled image 1102 to obtain a mask image 1103, and filtering the down-sampled image 1102 to obtain a filtered image 1104; up-sampling and smoothing the mask image 1103 to obtain a weight map 1105, performing Laplacian pyramid reconstruction on the filtered image 1104 to obtain a processed image 1106, and fusing the RGB image 1101 after AI image restoration and the processed image 1106 based on the weight map 1105 to obtain a result image 1107.

[0159] For downsampling, due to the frequency characteristics of low-frequency color noise, Gaussian pyramid is used to downsample to a low-frequency image.

[0160] For low-frequency color noise area detection, after the Gaussian pyramid is downsampled to the low-frequency image, the high frequency is concentrated in the area with higher gradient, and the low frequency is concentrated in the area with lower gradient. Therefore, according to the set second gradient threshold, a low-frequency flat area mask map can be obtained.

[0161] For filtering, Gaussian filtering is performed on the small-scale image at the bottom layer of the Gaussian pyramid.

[0162] For smoothing and upsampling, the mask image of the flat area is smoothed and upsampled to obtain the weight map, and the Laplacian pyramid is used to reconstruct the filtered result image.

[0163] For fusion, weights are assigned according to the weight map obtained at the same scale, and the processed image and the RGB image restored by the AI ​​image are fused to finally obtain the result image with the low-frequency color noise removed. The comparison of the results before and after processing is shown in the figure below. Fig.12 As shown, there is obvious red low-frequency color noise (grayscale processed) in the middle of the image 1201 to be processed, and the image 1202 after removing the low-frequency color noise does not include the red low-frequency color noise.

[0164] The image processing method provided in the embodiment of the present application:

[0165] First, we need to solve the problems that have long plagued the industry, such as image purple and green fringing, highlight edge artifacts, and low-frequency color noise that affect the objective image quality. We use a relatively consistent process to address these three problems. Only in downsampling, small-scale detection, and small-scale image processing are there some differences due to different problems. The process makes it easier to deploy post-processing algorithms.

[0166] In addition, under the condition of extreme downsampling to ensure detection accuracy, small-scale image detection and processing will not cause great pressure on mobile terminal deployment, and can be used to deal with multiple types of similar problems, not limited to solving the above three problems.

[0167] Finally, by fusion processing the image and the original image based on the smoothing weights, it can be ensured that the transition between the processed area and the unprocessed area in the final result image is smooth and natural, thereby improving the basic image quality and supporting customization according to different application scenarios for different problem scenarios.

[0168] Therefore, the image processing method provided in the embodiment of the present application utilizes small-scale image detection and filtering processing, as well as image fusion methods, to effectively solve the problems of image purple and green edges, highlight edge artifacts and low-frequency color noise, and will not bring challenges to the speed, stability and economy of the deployment level. Ultimately, it outputs an RGB image with completely solved image quality problems, improved human eye perception and realism, while meeting the requirements of mobile phone processing speed, power consumption, computing power and resolution.

[0169] In addition, on the one hand, the image processing method provided in the embodiment of the present application not only supports solving the problems of purple and green edges of images, highlight edge artifacts and low-frequency color noise, but also similar problems such as blue edges of images, highlight edge jaggedness, highlight edge false colors or color noise in polar night scenes with extremely low signal-to-noise ratio can be solved by the method in the process. The problems that can be solved are relatively comprehensive and have many application scenarios in the current field of mobile image processing. On the other hand, the image processing method provided in the embodiment of the present application supports solving only individual or multiple problems, can arbitrarily select and combine solution strategies, supports parallel processing, and is not limited to RGB image processing after AI image restoration. For YUV images, or images processed by high dynamic range, tone mapping, etc., this method can also be used for processing, thereby solving objective problems in image quality and improving the human eye perception of the image.

[0170] An electronic device according to an embodiment of the present application, such as Fig.13 As shown, the electronic device 1300 includes:

[0171] An acquisition unit 1301 is configured to acquire an image to be processed;

[0172] The identification unit 1302 is configured to identify the target area of ​​the image to be processed and obtain a first weight map, wherein the first weight map includes a first weight corresponding to each pixel in the image to be processed, and the first weight represents a probability of whether the corresponding pixel belongs to the target area;

[0173] A noise reduction unit 1303 is configured to perform noise reduction processing on the image to be processed to obtain a first image;

[0174] The fusion unit 1304 is configured to fuse the image to be processed, the first weight map and the first image to obtain a target image.

[0175] In some embodiments, the identification unit 1302 is further configured to:

[0176] Detecting a target area of ​​a second image to obtain a second weight map, wherein the second image is obtained by downsampling the image to be processed, and the second weight map includes a second weight corresponding to each pixel in the second image, and the second weight represents a probability of whether the corresponding pixel belongs to the target area;

[0177] The second weight map is upsampled and smoothed to obtain the first weight map, and the size of the first weight map is the same as the size of the image to be processed.

[0178] In some embodiments, the identification unit 1302 is further configured to:

[0179] Based on the first image feature, detecting a first area in the second image, the first area including a second area of ​​the target area and a third area outside the target area;

[0180] Based on a second image feature, detecting a position of the third area in the first area, the first image feature and the second image feature being different image features;

[0181] The position of the second region in the second image is determined based on the position of the third region, and the second weight map is determined based on the position of the second region in the second image.

[0182] In some embodiments, the noise reduction unit 1303 is further configured to:

[0183] filtering the second image to obtain a third image, wherein the second image is a down-sampling result of the image to be processed;

[0184] The third image is reconstructed to obtain the first image, where the size of the first image is the same as the size of the image to be processed.

[0185] In some embodiments, the noise reduction unit 1303 is further configured to:

[0186] Detecting a target area on the second image to obtain a second weight map, where the second image is a downsampling result of the image to be processed, and the second weight map includes a second probability of whether each pixel in the second image belongs to the target area;

[0187] The second image is weighted based on the second weight map to obtain an updated second image.

[0188] In some embodiments, the noise reduction unit 1303 is further configured to:

[0189] If the image to be processed is an RGB image, before filtering the second image to obtain the third image, performing format conversion on the second image to convert the second image into a YUV image;

[0190] After filtering the second image to obtain the third image, format conversion is performed on the third image to convert the third image into an RGB image.

[0191] In some embodiments, the electronic device further includes: an indication unit configured to:

[0192] Acquire area type indication information, where the area type indication information is used to indicate the type of the target area;

[0193] A detection strategy and a filtering strategy are determined based on the region type indication information, wherein the detection strategy is used to indicate a detection method for the target region, and the filtering strategy is used to indicate a noise reduction method for the first image.

[0194] In some embodiments, the noise reduction unit 1303 is further configured to:

[0195] If the type of the target area is chromatic noise, converting the image to be processed from the RGB domain to the YUV domain to obtain a YUV image to be processed;

[0196] Performing guided filtering on the YUV image to be processed according to the information of the Y channel to obtain a filtered YUV image;

[0197] The filtered YUV image is converted from the YUV domain to the RGB domain to obtain the first image.

[0198] In some embodiments, the electronic device further includes: a recovery unit configured to:

[0199] receiving a restoration request;

[0200] Based on the restoration request, artificial intelligence AI restoration is performed on the image to be processed.

[0201] In practical applications, the above-mentioned acquisition unit, recognition unit, noise reduction unit, fusion unit, indication unit, restoration unit and other modules can be implemented by a processor located on the electronic device, specifically a combination of one or more of a central processing unit (CPU), a microprocessor (MPU), an application processor (AP), a digital signal processor (DSP), and a field programmable gate array (FPGA).

[0202] Those skilled in the art should understand that the relevant description of the above-mentioned electronic device in the embodiment of the present application can be understood by referring to the relevant description of the image processing method in the embodiment of the present application.

[0203] Fig.14 An optional implementation of the present application is provided as a structural diagram of an electronic device, such as Fig.14 As shown, the embodiment of the present application provides an electronic device 1400, including an electronic chip 1401, which can be implemented as an image processing method as described in one or more embodiments above. The electronic chip can be implemented as a processor of an electronic device or as a microprocessor of a fast charging module.

[0204] An embodiment of the present application provides an electronic device, Fig.15 Another optional implementation provided in the embodiment of the present application is a structural diagram of an electronic device, such as Fig.15 As shown, an embodiment of the present application provides an electronic device 1500, including:

[0205] A processor 1501 and a storage medium 1502 storing executable instructions of the processor 1501, wherein the storage medium 1502 relies on the processor 1501 to perform operations through a communication bus 1503, and when the instructions are executed by the processor 1501, the image processing method performed in one or more of the above embodiments is executed.

[0206] It should be noted that in actual application, the various components in the terminal are coupled together through the communication bus 1503. It is understandable that the communication bus 1503 is used to realize the connection and communication between these components. In addition to the data bus, the communication bus 1503 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Fig.15 Various buses are labeled as communication buses 1503.

[0207] An embodiment of the present application provides a computer storage medium, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program enables a computer to execute the steps of the image processing method described in one or more of the above embodiments.

[0208] An embodiment of the present application provides a schematic structural diagram of an electronic device 1600. Fig.16 The electronic device 1600 shown includes a processor 1610. The processor 1610 is configured to:

[0209] Get the image to be processed;

[0210] Identify the target area of ​​the image to be processed, and obtain a first weight map, wherein the first weight map includes a first weight corresponding to each pixel in the image to be processed, and the first weight represents the probability of whether the corresponding pixel belongs to the target area;

[0211] Performing noise reduction processing on the image to be processed to obtain a first image;

[0212] The image to be processed is fused with the first weight map and the first image to obtain a target image.

[0213] In the embodiment of the present application, the processor 1610 can call and run a computer program from the memory to implement the image processing method in the embodiment of the present application.

[0214] Alternatively, if Fig.16As shown, the electronic device 1600 may further include a memory 1620. The processor 1610 may call and run a computer program from the memory 1620 to implement the image processing method in the embodiment of the present application.

[0215] The memory 1620 may be a separate device independent of the processor 1610 , or may be integrated into the processor 1610 .

[0216] Alternatively, if Fig.16 As shown, the electronic device 1600 may further include a transceiver 1630, and the processor 1610 may control the transceiver 1630 to communicate with other devices, specifically, to receive signals sent by other devices. Here, the transceiver may include at least two antennas.

[0217] It is understandable that the transceiver includes multiple physical paths for receiving or transmitting signals, and the physical elements on one or more physical paths for transmitting signals constitute a transmitter, wherein the MPR values ​​of each physical path in the transmitter are independent.

[0218] Optionally, the electronic device 1600 can implement the corresponding processes implemented by the electronic device in each method of the embodiment of the present application, and for the sake of brevity, it will not be repeated here. It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Arra y, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined to execute. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0219] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0220] It should be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application may also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct RAM bus random access memory (Direct RAM, DR RAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0221] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0222] Optionally, the computer-readable storage medium can be applied to the electronic device in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0223] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0224] Optionally, the computer program can be applied to the electronic device in the embodiments of the present application. When the computer program runs on a computer, the computer executes the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they are not described here.

[0225] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0226] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0227] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0228] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0229] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0230] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0231] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: The method comprises: Get the image to be processed; Identify the target area of ​​the image to be processed, and obtain a first weight map, wherein the first weight map includes a first weight corresponding to each pixel in the image to be processed, and the first weight represents the probability of whether the corresponding pixel belongs to the target area; Performing noise reduction processing on the image to be processed to obtain a first image; The image to be processed is fused with the first weight map and the first image to obtain a target image.

2. The method according to claim 1, characterized in that The step of identifying the target area of ​​the image to be processed and obtaining a first weight map includes: Detecting a target area of ​​a second image to obtain a second weight map, wherein the second image is obtained by downsampling the image to be processed, and the second weight map includes a second weight corresponding to each pixel in the second image, and the second weight represents a probability of whether the corresponding pixel belongs to the target area; The second weight map is upsampled and smoothed to obtain the first weight map, and the size of the first weight map is the same as the size of the image to be processed.

3. The method according to claim 2, characterized in that The detecting the target area of ​​the second image to obtain a second weight map includes: Based on the first image feature, detecting a first area in the second image, the first area including a second area of ​​the target area and a third area outside the target area; Based on a second image feature, detecting a position of the third area in the first area, the first image feature and the second image feature being different image features; The position of the second region in the second image is determined based on the position of the third region, and the second weight map is determined based on the position of the second region in the second image.

4. The method according to claim 1, characterized in that: The step of performing noise reduction processing on the image to be processed to obtain a first image includes: filtering the second image to obtain a third image, wherein the second image is a down-sampling result of the image to be processed; The third image is reconstructed to obtain the first image, where the size of the first image is the same as the size of the image to be processed.

5. The method according to claim 4, characterized in that Before filtering the second image to obtain the third image, the method further comprises: Detecting a target area on the second image to obtain a second weight map, where the second image is a downsampling result of the image to be processed, and the second weight map includes a second probability of whether each pixel in the second image belongs to the target area; The second image is weighted based on the second weight map to obtain an updated second image.

6. The method according to claim 4 or 5, characterized in that: The image to be processed is an RGB image, and the method further comprises: Before filtering the second image to obtain the third image, performing format conversion on the second image to convert the second image into a YUV image; After filtering the second image to obtain the third image, format conversion is performed on the third image to convert the third image into an RGB image.

7. The method according to any one of claims 1, 2, 4 and 5, characterized in that: The method further comprises: Acquire area type indication information, where the area type indication information is used to indicate the type of the target area; A detection strategy and a filtering strategy are determined based on the region type indication information, wherein the detection strategy is used to indicate a detection method for the target region, and the filtering strategy is used to indicate a noise reduction method for the first image.

8. The method according to any one of claims 7, characterized in that If the type of the target area is chromatic noise, performing noise reduction processing on the image to be processed to obtain a first image includes: Convert the image to be processed from the RGB domain to the YUV domain to obtain a YUV image to be processed; Performing guided filtering on the YUV image to be processed according to the information of the Y channel to obtain a filtered YUV image; The filtered YUV image is converted from the YUV domain to the RGB domain to obtain the first image.

9. The method according to any one of claims 1, 2, 4 and 5, characterized in that: After acquiring the image to be processed, the method further comprises: receiving a restoration request; Based on the restoration request, artificial intelligence AI restoration is performed on the image to be processed.

10. An electronic device, characterized in that: The electronic device comprises: An acquisition unit, configured to acquire an image to be processed; an identification unit configured to identify a target area of ​​the image to be processed and obtain a first weight map, wherein the first weight map includes a first weight corresponding to each pixel in the image to be processed, and the first weight represents a probability of whether the corresponding pixel belongs to the target area; A noise reduction unit, configured to perform noise reduction processing on the image to be processed to obtain a first image; The fusion unit is configured to fuse the image to be processed with the first weight map and the first image to obtain a target image.

11. An electronic device, comprising a processor, characterized in that: The processor is configured to: Get the image to be processed; Identify the target area of ​​the image to be processed, and obtain a first weight map, wherein the first weight map includes a first weight corresponding to each pixel in the image to be processed, and the first weight represents the probability of whether the corresponding pixel belongs to the target area; Performing noise reduction processing on the image to be processed to obtain a first image; The image to be processed is fused with the first weight map and the first image to obtain a target image.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps in the image processing method according to any one of claims 1 to 9 are implemented.

13. A storage medium storing an executable program, characterized in that: When the executable program is executed by a processor, the image processing method according to any one of claims 1 to 9 is implemented.

Images