Image processing method and device, electronic equipment and storage medium

Abstract

The application discloses an image processing method and device, electronic equipment and storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring a to-be-processed image; determining a pixel attribution value of a current pixel point in the to-be-processed image; determining a pixel smoothing value of the current pixel point in the to-be-processed image; calculating a local variance value of the current pixel point according to the pixel attribution value and the pixel smoothing value of the current pixel point and a preset condition; and performing segmented statistics on the local variance value corresponding to each pixel point based on the brightness value of each pixel point in the to-be-processed image to obtain noise level estimation of an HDR image in different brightness intervals. The pixel attribution value is calculated to determine the pixel source, the pixel smoothing value is calculated to avoid the interference of a texture region, and then the local variance value is calculated to reflect the noise level of the pixel point, so that the estimation accuracy of the noise is improved. Finally, the noise level estimation in different brightness intervals is output, and high-quality input is provided for subsequent image processing.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Image denoising is a crucial topic in Image Signal Processing (ISP) technology. A common denoising method involves obtaining all pixels in the neighborhood of the current pixel and applying a low-pass filter to these pixels to obtain the denoised pixel. However, this denoising technique typically reduces the high-frequency components of the image, making it blurry.

[0003] In conventional image denoising, the noise level (NStd) and high-frequency components (HF) of the current image are first measured. Then, by comparing their magnitudes, it is determined whether the current image region is image texture. For example, when HF is significantly higher than NStd, the current image region is considered image texture, thus avoiding blurring of the image during denoising. When HF is approximately equal to or less than NStd, the current image region is considered a noise region, and a stronger blurring process is applied to the image.

[0004] However, current solutions can estimate noise for some regular images, but HDR images are usually formed by fusing two or more RAW images. The noise levels of the RAW images before fusion are different, and because the image signal is amplified during the HDR fusion process, the noise level after fusion is often discontinuous, with discontinuities appearing at the boundaries of image fusion. As a result, there is currently no good solution for noise estimation of HDR images. Summary of the Invention

[0005] In view of this, this application provides an image processing method, apparatus, electronic device, and storage medium, the main purpose of which is to solve the problem that noise estimation of HDR images is currently impossible.

[0006] In a first aspect, this application provides an image processing method, comprising:

[0007] Obtain the image to be processed, which is an HDR image obtained by fusing at least two original images;

[0008] Determine the pixel attribution value of the current pixel in the image to be processed, and the pixel attribution value is used to determine the original image to which the current pixel belongs;

[0009] Determine the pixel smoothing value of the current pixel in the image to be processed, and the pixel smoothing value is used to determine the image smoothness of the region where the current pixel is located;

[0010] Based on the pixel attribution value and pixel smoothness value of the current pixel, calculate the local variance value of the current pixel according to preset conditions;

[0011] Based on the brightness value of each pixel in the image to be processed, the local variance value corresponding to each pixel is segmented and statistically analyzed to obtain the noise level estimate of the HDR image in different brightness ranges.

[0012] Optionally, determining the pixel attribution value of the current pixel in the image to be processed includes: obtaining the brightness value corresponding to each original image of the image to be processed obtained by fusion; obtaining the brightness value of the current pixel; and determining the pixel attribution value of the current pixel based on the brightness value of the current pixel and the brightness value corresponding to each original image.

[0013] Optionally, determining the pixel attribution value of the current pixel based on the brightness value of the current pixel and the brightness value corresponding to each original image includes: if the image to be processed is obtained by fusing a first original image and a second original image, then obtaining the brightness value of the current pixel; comparing the brightness value of the current pixel with a preset brightness threshold, and assigning a first attribution value to the current pixel if the brightness value of the current pixel is less than or equal to the preset brightness threshold; or, assigning a second attribution value to the current pixel if the brightness value of the current pixel is greater than the preset brightness threshold; wherein the first attribution value indicates that the current pixel originates from the first original image, the second attribution value indicates that the current pixel originates from the second original image, and the preset brightness threshold is set according to the brightness value corresponding to the first original image during fusion and the brightness value corresponding to the second original image during fusion.

[0014] Optionally, determining the pixel smoothing value of the current pixel in the image to be processed includes: obtaining the pixel value of each pixel within a first preset region centered on the current pixel; performing filtering calculations on the pixel value of each pixel within the first preset region using a high-pass filter operator to obtain the high-frequency component corresponding to the first preset region; comparing the high-frequency component corresponding to the first preset region with a preset component threshold; assigning a first smoothing value to the current pixel if the high-frequency component corresponding to the first preset region is less than or equal to the preset component threshold; or assigning a second smoothing value to the current pixel if the high-frequency component corresponding to the first preset region is greater than the preset component threshold; wherein the first smoothing value indicates that the region where the current pixel is located is a flat region; the second smoothing value indicates that the region where the current pixel is located is a textured region, and the high-pass filter operator includes a Sobel operator or a Laplacian operator.

[0015] Optionally, the step of calculating the local variance of the current pixel based on the pixel attribution value and the pixel smoothing value according to preset conditions includes: filtering out a first set of pixels within a second preset region whose pixel smoothing value is the second smoothing value, centered on the current pixel; determining the pixel attribution value of the current pixel, filtering out pixels in the first set of pixels whose pixel attribution value is different from that of the current pixel to obtain a second set of pixels; and calculating the local variance of the current pixel based on the current pixel and the second set of pixels.

[0016] Optionally, the local variance value of the current pixel is calculated based on the current pixel and the second set of pixels using the following formula:

[0017]

[0018] in, This represents the local variance value of the current pixel. The pixel value corresponding to the i-th pixel within the second preset area; For the attribution determination, if the pixel attribution value of the i-th pixel is the same as that of the current pixel, It is 1 if it is true, otherwise it is 0. For smoothness assessment, given that the current pixel's smoothness value is the first smoothness value, It is 1 if it is true, otherwise it is 0.

[0019] Optionally, the step of performing segmented statistical analysis on the local variance value corresponding to each pixel based on the brightness value of each pixel in the image to be processed to obtain the noise level estimate of the HDR image in different brightness intervals includes: dividing the brightness of the image to be processed into N consecutive brightness intervals, where N is an integer greater than 1; classifying each pixel and its corresponding local variance value into a corresponding brightness interval according to the brightness value of each pixel in the image to be processed; for each brightness interval, calculating the statistical average of the local variance values ​​corresponding to all pixels falling into that interval; and using the statistical average value corresponding to each brightness interval as the noise level estimate of the HDR image in that brightness interval.

[0020] Secondly, this application provides an image processing apparatus, comprising:

[0021] The acquisition unit is configured to acquire an image to be processed, wherein the image to be processed is an HDR image obtained by fusing at least two original images;

[0022] The first calculation unit is configured to determine the pixel attribution value of the current pixel in the image to be processed, and the pixel attribution value is used to determine the original image to which the current pixel belongs;

[0023] The second calculation unit is configured to determine the pixel smoothing value of the current pixel in the image to be processed, and the pixel smoothing value is used to determine the image smoothness of the region where the current pixel is located.

[0024] The processing unit is configured to calculate the local variance value of the current pixel based on the pixel attribution value and the pixel smoothing value according to preset conditions.

[0025] The statistical unit is configured to perform segmented statistical analysis on the local variance value corresponding to each pixel based on the brightness value of each pixel in the image to be processed, so as to obtain the noise level estimate of the HDR image in different brightness ranges.

[0026] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method described in the first aspect.

[0027] Fourthly, this application provides an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement the image processing method described in the first aspect.

[0028] By employing the above technical solutions, this application provides an image processing method, apparatus, electronic device, and storage medium. First, it acquires an image to be processed, which is an HDR image obtained by fusing at least two original images. Next, it determines the pixel attribution value of the current pixel in the image to be processed, which is used to determine the original image to which the current pixel belongs. Then, it determines the pixel smoothing value of the current pixel in the image to be processed, which is used to determine the smoothness of the image in the region where the current pixel is located. Based on the pixel attribution value and the pixel smoothing value, it calculates the local variance value of the current pixel according to preset conditions. Based on the brightness value of each pixel in the image to be processed, it performs segmented statistical analysis on the local variance value corresponding to each pixel to obtain noise level estimates of the HDR image in different brightness ranges. This application proposes a method for noise estimation of HDR images. By calculating the pixel attribution value of each pixel in the HDR image to determine the pixel source, calculating the pixel smoothing value to avoid interference from texture regions, and then calculating the local variance value based on the pixel attribution value and the pixel smoothing value to reflect the noise level of the pixel, it improves the noise estimation accuracy. Finally, it outputs noise level estimates in different brightness ranges, providing high-quality input for subsequent image processing.

[0029] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0030] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0031] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 A schematic flowchart of an image processing method provided in an embodiment of this application is shown;

[0033] Figure 2 This illustration shows a schematic diagram of image fusion based on changes in scene brightness, according to an embodiment of this application.

[0034] Figure 3 A schematic diagram of a noise level estimation curve provided in an embodiment of this application is shown;

[0035] Figure 4 A flowchart illustrating another image processing method provided in an embodiment of this application is shown;

[0036] Figure 5 A schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application is shown. Detailed Implementation

[0037] To better understand the above-mentioned objectives, features, and advantages of this application, the solution of this application will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0038] The image processing method proposed in this embodiment is applied to an image processing device or electronic device. The device or electronic device can be installed or integrated into some image noise analysis system or image noise reduction processing system, and can execute any of the image processing methods mentioned below during operation.

[0039] To address the current issue of poor noise estimation accuracy in fused HDR images, this embodiment proposes an image processing method, such as... Figure 1 As shown, the method includes:

[0040] S101, Obtain the image to be processed.

[0041] The image to be processed is an HDR image obtained by fusing at least two original images. For example, two or more RAW images with different initial brightness are scaled up by a certain factor to enlarge the initial image with lower initial brightness, and then fused to obtain an HDR image.

[0042] This section uses the fusion of two RAW images as an example to briefly introduce the fusion process. An image sensor captures a scene, converting light signals into electrical signals to obtain an image. Each image has a cutoff signal, which is the maximum brightness the image can display. Because different RAW images have different exposure times, their maximum brightness and noise levels differ. During fusion, the RAW image with lower brightness is enlarged and then fused with the other image. This process often results in discontinuities in the brightness and noise levels of the fused image, creating a break at the fusion boundary.

[0043] S102, determine the pixel affiliation value of the current pixel in the image to be processed.

[0044] The pixel attribution value is used to determine the original image to which the current pixel belongs, that is, to distinguish which of the two or more original images the current pixel belongs to. The purpose of this distinction is to use the same original image as one of the preset conditions in the subsequent local variance value calculation, thereby avoiding interference from pixels of other images after fusion, and thus improving the accuracy of the local variance value in reflecting the noise level of the current pixel.

[0045] S103, Determine the pixel smoothing value of the current pixel in the image to be processed.

[0046] The pixel smoothness value is used to determine the smoothness of the image in the area where the current pixel is located. Pixel smoothness refers to the smoothness of the area where the current pixel is located. That is, with the current pixel as the center, the smoothness of all pixels within a preset range or window is calculated to determine whether this area is smooth or textured.

[0047] S104, calculate the local variance value of the current pixel according to the pixel affiliation value and pixel smoothing value of the current pixel under preset conditions.

[0048] The local variance of the current pixel reflects the noise level of the current pixel. Since the pixel attribution value and pixel smoothing value of the current pixel are introduced, they can be adjusted according to preset conditions to avoid interference from different images and texture regions, thereby improving the evaluation accuracy of the local variance of the current pixel.

[0049] S105, based on the brightness value of each pixel in the image to be processed, the local variance value corresponding to each pixel is segmented and statistically analyzed to obtain the noise level estimate of the HDR image in different brightness ranges.

[0050] Since the noise level depends on the signal strength, i.e. the pixel brightness, statistical analysis can be performed to estimate the noise level of the HDR image in different brightness ranges, such as specifying the noise at a brightness of 100 and the noise at a brightness of 800, thus facilitating the application and processing of other noise reduction algorithms.

[0051] In this embodiment, the image to be processed is first acquired; this image is an HDR image obtained by fusing at least two original images. The pixel attribution value of the current pixel in the image to be processed is determined, and this value is used to determine the original image to which the current pixel belongs. The pixel smoothing value of the current pixel in the image to be processed is also determined, and this value is used to determine the smoothness of the image in the region where the current pixel is located. Based on the pixel attribution value and the pixel smoothing value, the local variance value of the current pixel is calculated according to preset conditions. Based on the brightness value of each pixel in the image to be processed, the local variance value corresponding to each pixel is statistically analyzed in segments to obtain noise level estimates of the HDR image in different brightness ranges. This application proposes a method for noise estimation of HDR images. By calculating the pixel attribution value of each pixel in the HDR image to determine the pixel source, and by calculating the pixel smoothing value to avoid interference from texture regions, the local variance value is calculated based on the pixel attribution value and the pixel smoothing value to reflect the noise level of the pixel, thereby improving the noise estimation accuracy. Finally, noise level estimates in different brightness ranges are output, providing high-quality input for subsequent image processing.

[0052] The specific steps of the solution will be explained below. In S102 above, the pixel assignment value of the current pixel in the image to be processed is determined, including: obtaining the brightness value corresponding to each original image of the image to be processed obtained by fusion; obtaining the brightness value of the current pixel; and determining the pixel assignment value of the current pixel based on the brightness value of the current pixel and the brightness value corresponding to each original image.

[0053] In this embodiment, the pixel affiliation value is determined based on the brightness value corresponding to each original image of the fused image to be processed. As mentioned above, when fusing two or more original images, the RAW image with lower brightness is magnified image by image before being fused with another image. The magnification factor is usually determined based on the relationship between the two or more images, such as the exposure ratio or duration ratio. By using the brightness value as the criterion, it is easier to determine the pixel affiliation value of the current pixel, that is, to determine which original image it belongs to.

[0054] Furthermore, the following example, using the fusion of two original images into an image to be processed, illustrates how to determine the pixel attribution value of the current pixel. The pixel attribution value is determined based on the brightness value of the current pixel and the corresponding brightness value of each original image. This includes: if the image to be processed is obtained by fusing the first and second original images, then the brightness value of the current pixel is obtained; the brightness value of the current pixel is compared with a preset brightness threshold; if the brightness value of the current pixel is less than or equal to the preset brightness threshold, a first attribution value is assigned to the current pixel; or, if the brightness value of the current pixel is greater than the preset brightness threshold, a second attribution value is assigned to the current pixel; wherein, the first attribution value indicates that the current pixel originates from the first original image, the second attribution value indicates that the current pixel originates from the second original image, and the preset brightness threshold is set based on the brightness values ​​of the first and second original images during fusion.

[0055] In this embodiment, combined with Figure 2 As shown, the HDR fused image is composed of two original images: Image 1 (orange portion) and Image 2 (blue portion). Image 1 has a longer exposure time, while Image 2 has a shorter exposure time. This means Image 1 is inherently brighter; as scene brightness increases, the image signal increases, but it reaches the cutoff point earlier. Once Image 1 reaches its cutoff point, further increases in scene brightness will have no effect. For example, if Image 1 depicts the sun, the sun will gradually appear white as scene brightness increases, but after reaching the cutoff point, further increases in brightness will not change the white portion of the sun. Image 2, on the other hand, is darker. Although it also depicts the sun, it can handle a higher scene brightness compared to Image 1. Therefore, the cutoff point for Image 2 is later than that for Image 1, allowing it to handle more scene brightness. During fusion, Image 2 needs to be enlarged by a certain proportion (e.g., the dashed line portion of Image 2 is enlarged to a solid line portion) to be merged with Image 1. The enlargement ratio can be determined based on the relationship between the brightness thresholds or exposure ratios of the two images.

[0056] The above is merely an introduction to the fusion process to facilitate understanding of the cutoff position, and is not intended to limit how to obtain the HDR image to be processed. Making some changes to the fusion method can also yield a satisfactory HDR image. For the fused HDR image, the brightness value of the current pixel needs to be obtained first, and then compared with a preset brightness threshold to determine whether the pixel belongs to the first or second attribute value, i.e., which original image it belongs to. The preset brightness threshold can be determined based on the brightness values ​​of the first and second original images during fusion. The brightness values ​​of the first and second original images during fusion include the different signal cutoff positions during the fusion of the two original images, such as... Figure 2 The image shows signal cutoff position 1 corresponding to image 1 and signal cutoff position 2 corresponding to the magnified image 2. For example, if the preset brightness threshold is set to the brightness corresponding to signal cutoff position 1, then it can be determined that if the brightness value of the current pixel in the fused HDR image is greater than the preset brightness threshold, it belongs to image 2; otherwise, it belongs to image 1. Of course, the preset brightness threshold can also be slightly larger than signal cutoff position 1, because the boundary between the two brightness values ​​at signal cutoff position 1 is not entirely clear. For example, if the brightness corresponding to the current pixel value is the same as the brightness corresponding to signal cutoff position 1, it is not possible to completely distinguish whether the image originates from image 1 or image 2. Therefore, the preset brightness threshold can be set slightly larger than the brightness corresponding to signal cutoff position 1, making it easier to distinguish the source of the image.

[0057] In one feasible implementation, the pixel attribution value can be set to S_MASK, with a first attribution value of 0 and a second attribution value of 1. In this embodiment, if the brightness value of the current pixel is less than the brightness value corresponding to signal cutoff position 1, it indicates that it comes from image 1, and S_MASK is assigned to 0; if the brightness value of the current pixel is greater than the brightness value corresponding to signal cutoff position 1, it indicates that it comes from image 2, and S_MASK is assigned to 1.

[0058] The above method can also be used when fusing multiple original images into a single image. For example, consider an HDR image obtained by fusing three images A, B, and C, with brightness values ​​a, b, and c respectively before fusion. The signal cutoff position of image A is X, and the signal cutoff position of image B is Y, where a < X < b < Y < c. If the brightness of a pixel in the fused HDR image is less than X, its source is determined to be image A; if the brightness of a pixel is between X and Y, its source is determined to be image B.

[0059] Optionally, determining the pixel smoothing value of the current pixel in the image to be processed includes: obtaining the pixel value of each pixel within a first preset region centered on the current pixel; performing filtering calculations on the pixel value of each pixel within the first preset region using a high-pass filter operator to obtain the high-frequency component corresponding to the first preset region; comparing the high-frequency component corresponding to the first preset region with a preset component threshold; assigning a first smoothing value to the current pixel if the high-frequency component corresponding to the first preset region is less than or equal to the preset component threshold; or assigning a second smoothing value to the current pixel if the high-frequency component corresponding to the first preset region is greater than the preset component threshold; wherein the first smoothing value indicates that the region where the current pixel is located is a flat region; the second smoothing value indicates that the region where the current pixel is located is a textured region, and the high-pass filter operator includes the Sobel operator or the Laplacian operator.

[0060] In this embodiment, the pixel smoothing value is used to determine whether the region defined for each pixel in the HDR fused image is a textured region or a flat region. Specifically, this can be achieved by performing a high-pass filter on the image and determining whether it is a textured region based on the magnitude of the high-frequency components of the pixels. Commonly used high-pass filter operators include the Sobel operator and the Laplacian operator. Here, the first preset region refers to a region or window of a preset size centered on the current pixel, such as a 3×3 window centered on the current pixel, which serves as the preset region corresponding to the current pixel. The preset component threshold can be set according to the actual situation. In one feasible calculation method, the high-frequency components of each pixel in this region are first calculated, and the difference between each component and the preset component threshold is averaged. The first smoothing value or the second smoothing value is then assigned based on the magnitude of the average value. The first smoothing value represents a flat region, and the second smoothing value represents a textured region.

[0061] For a specific example, taking a certain pixel as the center, the high-frequency components corresponding to the nine pixels in a 3×3 window are calculated. Then, the difference between these nine high-frequency components and a preset threshold is calculated to obtain the mean. A threshold value is then set, which can be based on practical experience. If the value is greater than the threshold value, the region is determined to be a textured region; alternatively, a model analysis method can be used, where the model analyzes historical data to determine if the mean value reflects the flatness or texture characteristics of the region. The first smoothing value can be represented as D_MASK=0, and the second smoothing value can be represented as D_MASK=1. It should be noted that although the determination is made on whether a region is flat or textured, the flatness or texture characteristics of that region are reflected in the pixel smoothing value of the center point. That is, in the above embodiment, although the pixel smoothing value D_MASK corresponding to the center pixel is ultimately calculated for the nine pixels, macroscopically, each pixel is the center point of its first preset region, and these nine pixels themselves each have a corresponding pixel smoothing value D_MASK.

[0062] Optionally, based on the pixel attribution value and pixel smoothing value of the current pixel, the local variance value of the current pixel is calculated according to preset conditions, including: taking the current pixel as the center, filtering out a first set of pixels in a second preset region whose pixel smoothing value is the second smoothing value; determining the pixel attribution value of the current pixel, filtering out pixels in the first set of pixels whose pixel attribution value is different from that of the current pixel to obtain a second set of pixels; and calculating the local variance value of the current pixel based on the current pixel and the second set of pixels.

[0063] In this embodiment, the setting of preset conditions is explained. First, the second preset region is also a region or window centered on the current pixel, and can be the same as the first preset region. Similarly, centered on the current pixel, the 9 pixels in the 3×3 window are calculated. As mentioned above, each of the 9 pixels has a corresponding D_MASK. First, it is necessary to determine the pixels with D_MASK=0 within the second preset region, that is, to filter out the pixels with D_MASK=1, resulting in the first pixel set. For example, if 2 out of the original 9 pixels have D_MASK=1, then the first pixel set consists of the remaining 7 pixels with D_MASK=0. Then, pixels with different pixel values ​​from the current pixel are filtered out from the first pixel set, resulting in the second pixel set. That is, among the 7 pixels with D_MASK=0, if the pixel value S_MASK of the center point is 0, then only the pixels with S_MASK=0 among the 7 points are considered. For example, if 3 out of 7 pixels have S_MASK=1, then the second set of pixels consists of the remaining 4 pixels with S_MASK=0. Then, based on the current pixel and the remaining 4 pixels, the local variance is calculated.

[0064] Specifically, based on the current pixel and the set of second pixels, the local variance of the current pixel is calculated using the following formula:

[0065] (Formula 1)

[0066] in, This represents the local variance value of the current pixel. The pixel value corresponding to the i-th pixel within the second preset area; For the attribution determination, if the pixel attribution value of the i-th pixel is the same as that of the current pixel, It is 1 if it is true, otherwise it is 0. For smoothness assessment, given that the current pixel's smoothness value is the first smoothness value, It is 1 if it is true, otherwise it is 0.

[0067] In this embodiment, setting D_MASK=0 is to eliminate the influence of texture regions within the second preset area on noise estimation; selecting points with S_MASK identical to the current pixel is to determine the same image source, thereby avoiding noise errors introduced by other images. By setting the preset conditions, the accuracy of noise estimation is improved.

[0068] Optionally, based on the brightness value of each pixel in the image to be processed, the local variance value corresponding to each pixel is statistically analyzed in segments to obtain the noise level estimate of the HDR image in different brightness intervals. This includes: dividing the brightness of the image to be processed into N consecutive brightness intervals, where N is an integer greater than 1; classifying each pixel and its corresponding local variance value into the corresponding brightness interval according to the brightness value of each pixel in the image to be processed; for each brightness interval, calculating the statistical average of the local variance values ​​corresponding to all pixels falling into that interval; and using the statistical average value corresponding to each brightness interval as the noise level estimate of the HDR image in that brightness interval.

[0069] In this embodiment, noise level estimates for HDR images in different brightness ranges are obtained statistically based on brightness, facilitating the application and processing of other noise reduction algorithms. For example... Figure 3 The diagram shows a schematic of a noise level estimation curve. The vertical axis represents the noise level, which is reflected by the local variance; the horizontal axis represents the image signal intensity, which is the magnitude of the image brightness signal; the orange curve represents the noise level of image 1, and the blue curve represents the noise level of image 2. Figure 3 As can be seen, by dividing the brightness of the image to be processed into N consecutive brightness intervals and then quantizing the local variance values, the noise level of the HDR image within the brightness interval is estimated, which facilitates subsequent denoising processing.

[0070] In conjunction with the image processing methods mentioned in any of the above embodiments, this embodiment also provides a system architecture for a specific operation module for image processing, which consists of an S_MASK generation module, a D_MASK generation module, a local variance calculation module, and a statistics module.

[0071] Combination Figure 4As shown, the process of image processing by each module is explained. First, a fused HDR image is input. Then, the S_MASK generation module generates the S_MASK for each pixel in the input HDR image, i.e., the pixel assignment value. The D_MASK generation module generates the D_MASK pixel smoothing value for each pixel in the input HDR image. Then, the local variance calculation module calculates the local variance value for each pixel according to preset conditions. Finally, the statistics module performs pixel brightness statistics and noise assessment. Specific details can be referred to the image processing method mentioned in the above embodiment, and will not be repeated here. This embodiment determines the pixel source by calculating the pixel assignment value for each pixel in the HDR image, avoids interference from texture regions by calculating the pixel smoothing value, and then calculates the local variance value based on the pixel assignment value and the pixel smoothing value to reflect the noise level of the pixel, thereby improving the noise estimation accuracy. Finally, it outputs noise level estimates under different brightness ranges, providing high-quality input for subsequent image processing.

[0072] Furthermore, as Figures 1 to 4 The specific implementation of the method shown in this embodiment provides an image processing device, such as... Figure 5 As shown, the device includes: an acquisition unit 501, a first calculation unit 502, a second calculation unit 503, a processing unit 504, and a statistics unit 505.

[0073] The acquisition unit 501 is configured to acquire an image to be processed, wherein the image to be processed is an HDR image obtained by fusing at least two original images;

[0074] The first calculation unit 502 is configured to determine the pixel attribution value of the current pixel in the image to be processed, and the pixel attribution value is used to determine the original image to which the current pixel belongs.

[0075] The second calculation unit 503 is configured to determine the pixel smoothing value of the current pixel in the image to be processed, wherein the pixel smoothing value is used to determine the image smoothness of the region where the current pixel is located.

[0076] Processing unit 504 is configured to calculate the local variance value of the current pixel based on the pixel attribution value and the pixel smoothing value of the current pixel according to preset conditions;

[0077] The statistical unit 505 is configured to perform segmented statistics on the local variance value corresponding to each pixel based on the brightness value of each pixel in the image to be processed, so as to obtain the noise level estimate of the HDR image in different brightness ranges.

[0078] In a specific application scenario, the first computing unit 502 is further configured to obtain the brightness value corresponding to each original image of the image to be processed obtained by fusion; obtain the brightness value of the current pixel; and determine the pixel affiliation value of the current pixel based on the brightness value of the current pixel and the brightness value corresponding to each original image.

[0079] In a specific application scenario, the first computing unit 502 is further configured to: if the image to be processed is obtained by fusing a first original image and a second original image, then obtain the brightness value of the current pixel; compare the brightness value of the current pixel with a preset brightness threshold; if the brightness value of the current pixel is less than or equal to the preset brightness threshold, assign a first attribution value to the current pixel; or, if the brightness value of the current pixel is greater than the preset brightness threshold, assign a second attribution value to the current pixel; wherein, the first attribution value indicates that the current pixel originates from the first original image, the second attribution value indicates that the current pixel originates from the second original image, and the preset brightness threshold is set according to the brightness value of the first original image during fusion and the brightness value of the second original image during fusion.

[0080] In a specific application scenario, the second calculation unit 503 is further configured to: obtain the pixel value of each pixel within a first preset region, centered on the current pixel; perform filtering calculations on the pixel value of each pixel within the first preset region using a high-pass filter operator to obtain the high-frequency component corresponding to the first preset region; compare the high-frequency component corresponding to the first preset region with a preset component threshold; if the high-frequency component corresponding to the first preset region is less than or equal to the preset component threshold, assign a first smoothing value to the current pixel; or, if the high-frequency component corresponding to the first preset region is greater than the preset component threshold, assign a second smoothing value to the current pixel; wherein, the first smoothing value indicates that the region where the current pixel is located is a flat region; the second smoothing value indicates that the region where the current pixel is located is a textured region; and the high-pass filter operator includes a Sobel operator or a Laplacian operator.

[0081] In a specific application scenario, the processing unit 504 is further configured to, with the current pixel as the center, filter out a first set of pixels within a second preset region whose pixel smoothing value is the second smoothing value; determine the pixel affiliation value of the current pixel; filter out pixels in the first set of pixels whose pixel affiliation value is different from that of the current pixel to obtain a second set of pixels; and calculate the local variance value of the current pixel based on the current pixel and the second set of pixels.

[0082] In specific application scenarios, processing unit 504 is further configured to calculate using the following formula:

[0083]

[0084] in, This represents the local variance value of the current pixel. The pixel value corresponding to the i-th pixel within the second preset area; For the attribution determination, if the pixel attribution value of the i-th pixel is the same as that of the current pixel, It is 1 if it is true, otherwise it is 0. For smoothness assessment, given that the current pixel's smoothness value is the first smoothness value, It is 1 if it is true, otherwise it is 0.

[0085] In a specific application scenario, the statistical unit 505 is further configured to divide the brightness of the image to be processed into N consecutive brightness intervals, where N is an integer greater than 1; based on the brightness value of each pixel in the image to be processed, divide each pixel and its corresponding local variance value into the corresponding brightness interval; for each brightness interval, calculate the statistical average of the local variance values ​​corresponding to all pixels falling into that interval; and use the statistical average value corresponding to each brightness interval as an estimate of the noise level of the HDR image within that brightness interval.

[0086] It should be noted that other corresponding descriptions of the functional units involved in the image processing apparatus provided in this embodiment can be found in [reference needed]. Figures 1 to 4 The corresponding descriptions in [the document] will not be repeated here.

[0087] Based on the above, Figures 1 to 4 Accordingly, this embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. Figures 1 to 4 The method shown.

[0088] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.

[0089] Based on the above, Figures 1 to 4 The method shown, and Figure 5To achieve the above objectives, this application also provides an electronic device, which can be configured on a computer side, etc. The device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-described objectives. Figures 1 to 4 The method shown.

[0090] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0091] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0092] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0093] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented using software plus necessary general-purpose hardware platforms, or it can be implemented in hardware. By applying the scheme of this embodiment, compared with related technologies, the pixel source is determined by calculating the pixel attribution value of each pixel in the HDR image, interference from texture regions is avoided by calculating pixel smoothing values, and then a local variance value is calculated based on the pixel attribution value and pixel smoothing value to reflect the noise level of the pixel, thereby improving the noise estimation accuracy. Finally, noise level estimates under different brightness ranges are output, providing high-quality input for subsequent image processing.

[0094] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0095] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An image processing method, characterized in that, include: Obtain the image to be processed, which is an HDR image obtained by fusing at least two original images; Determine the pixel attribution value of the current pixel in the image to be processed, and the pixel attribution value is used to determine the original image to which the current pixel belongs; Determine the pixel smoothing value of the current pixel in the image to be processed, and the pixel smoothing value is used to determine the image smoothness of the region where the current pixel is located; Based on the pixel attribution value and pixel smoothness value of the current pixel, calculate the local variance value of the current pixel according to preset conditions; Based on the brightness value of each pixel in the image to be processed, the local variance value corresponding to each pixel is segmented and statistically analyzed to obtain the noise level estimate of the HDR image in different brightness ranges. The step of calculating the local variance of the current pixel based on the pixel attribution value and the pixel smoothing value according to preset conditions includes: Centered on the current pixel, filter out a set of first pixels within a second preset region whose pixel smoothing value is a second smoothing value; the second smoothing value indicates that the region where the current pixel is located is a texture region; the second preset region is a region within a preset range centered on the current pixel; Determine the pixel affiliation value of the current pixel, filter out pixels with different pixel affiliation values ​​from the first set of pixels, and obtain a second set of pixels. Calculate the local variance value of the current pixel based on the current pixel and the second set of pixels.

2. The method according to claim 1, characterized in that, Determining the pixel affiliation value of the current pixel in the image to be processed includes: Obtain the brightness value corresponding to each original image of the fused image to be processed; Obtain the brightness value of the current pixel; The pixel affiliation value of the current pixel is determined based on the brightness value of the current pixel and the brightness value corresponding to each original image.

3. The method according to claim 2, characterized in that, The step of determining the pixel affiliation value of the current pixel based on the brightness value of the current pixel and the brightness value corresponding to each original image includes: If the image to be processed is obtained by fusing the first original image and the second original image, then the brightness value of the current pixel is obtained; The brightness value of the current pixel is compared with a preset brightness threshold. If the brightness value of the current pixel is less than or equal to the preset brightness threshold, a first assignment value is assigned to the current pixel; or, If the brightness value of the current pixel is greater than the preset brightness threshold, a second assignment value is assigned to the current pixel. Wherein, the first attribution value indicates that the current pixel originates from the first original image, the second attribution value indicates that the current pixel originates from the second original image, and the preset brightness threshold is set according to the brightness value corresponding to the first original image during fusion and the brightness value corresponding to the second original image during fusion.

4. The method according to claim 3, characterized in that, Determining the pixel smoothing value of the current pixel in the image to be processed includes: Using the current pixel as the center, obtain the pixel value of each pixel within the first preset area; By using a high-pass filter operator, the pixel value of each pixel in the first preset region is filtered and calculated to obtain the high-frequency component corresponding to the first preset region. The high-frequency component corresponding to the first preset region is compared with a preset component threshold. If the high-frequency component corresponding to the first preset region is less than or equal to the preset component threshold, a first smoothing value is assigned to the current pixel; or, If the high-frequency component corresponding to the first preset region is greater than the preset component threshold, a second smoothing value is assigned to the current pixel. Wherein, the first smoothing value indicates that the area where the current pixel is located is a flat area; the second smoothing value indicates that the area where the current pixel is located is a textured area; and the high-pass filter operator includes the Sobel operator or the Laplacian operator.

5. The method according to claim 1, characterized in that, The local variance value of the current pixel is calculated based on the current pixel and the second set of pixels using the following formula: ； in, This represents the local variance value of the current pixel. The pixel value corresponding to the i-th pixel within the second preset area; For the attribution determination, if the pixel attribution value of the i-th pixel is the same as that of the current pixel, It is 1 if it is true, otherwise it is 0; For smoothness assessment, given that the current pixel's smoothness value is the first smoothness value, It is 1 if it is true, otherwise it is 0.

6. The method according to claim 2, characterized in that, The step of performing segmented statistical analysis on the local variance value corresponding to each pixel based on the brightness value of each pixel in the image to be processed, to obtain the noise level estimate of the HDR image in different brightness ranges, includes: The brightness of the image to be processed is divided into N consecutive brightness intervals, where N is an integer greater than 1; Based on the brightness value of each pixel in the image to be processed, each pixel and its corresponding local variance value are divided into corresponding brightness intervals; For each brightness range, calculate the statistical average of the local variance values ​​corresponding to all pixels falling within that range; The statistical average value corresponding to each brightness range is used as the noise level estimate of the HDR image within that brightness range.

7. An image processing apparatus, characterized in that, include: The acquisition unit is configured to acquire an image to be processed, wherein the image to be processed is an HDR image obtained by fusing at least two original images; The first calculation unit is configured to determine the pixel attribution value of the current pixel in the image to be processed, and the pixel attribution value is used to determine the original image to which the current pixel belongs; The second calculation unit is configured to determine the pixel smoothing value of the current pixel in the image to be processed, and the pixel smoothing value is used to determine the image smoothness of the region where the current pixel is located. The processing unit is configured to calculate the local variance value of the current pixel based on the pixel attribution value and the pixel smoothing value according to preset conditions. The statistical unit is configured to perform segmented statistics on the local variance value corresponding to each pixel based on the brightness value of each pixel in the image to be processed, so as to obtain the noise level estimate of the HDR image in different brightness ranges. Specifically, the processing unit performs the calculation of the local variance value of the current pixel based on the pixel attribution value and the pixel smoothing value according to preset conditions, and is configured as follows: Centered on the current pixel, filter out a set of first pixels within a second preset region whose pixel smoothing value is a second smoothing value; the second smoothing value indicates that the region where the current pixel is located is a texture region; the second preset region is a region within a preset range centered on the current pixel; Determine the pixel affiliation value of the current pixel, filter out pixels with different pixel affiliation values ​​from the first set of pixels, and obtain a second set of pixels. Calculate the local variance value of the current pixel based on the current pixel and the second set of pixels.

8. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images