Image Denoising Methods and Devices in Intelligent Image Processing

Abstract

This application relates to an image denoising method and apparatus in intelligent image processing. The method includes: determining the black level bias value of a target image and subtracting the black level bias value to obtain a de-denoised image; determining the image denoising filtering method to be used based on image characteristics and processing requirements, and determining the size of the adjacency matrix of the filtering window according to the image denoising filtering method; calculating the filtering value of each pixel in the image based on the image denoising filtering and the size of the adjacency matrix of the filtering window; and updating the original pixel value of each pixel according to the filtering value to obtain the filtered image. This solves the problems in related technologies where image details cannot be preserved while denoising, resulting in reduced visual performance, decreased peak signal-to-noise ratio and structural similarity, which is detrimental to subsequent image recognition, detection, and segmentation tasks.

Description

Technical Field

[0001] This application relates to the field of image denoising technology, and in particular to an image denoising method and apparatus in intelligent image processing. Background Technology

[0002] Image denoising is a crucial research area in intelligent image processing. Color distortion, noise interference, and edge blurring in images can all have unpredictable impacts on subsequent processing. During image acquisition and transmission, images often contain noise contamination due to the influence of image processing equipment or external factors. Therefore, image denoising technology is of great research significance.

[0003] Related technologies are mainly divided into traditional image denoising techniques and neural network-based image denoising techniques. Traditional image denoising techniques can be further divided into spatial domain methods and transform domain methods based on their processing methods. Spatial domain methods directly process the image pixel values ​​to achieve denoising, while transform domain methods transform the image to a new feature space for denoising. Neural network-based image denoising techniques utilize the powerful modeling and fitting capabilities of neural networks to directly learn the mapping relationship from noisy images to noise-free images from large-scale data.

[0004] However, the relevant technologies cannot preserve image details while denoising the image, which reduces the image's visual performance, peak signal-to-noise ratio, and structural similarity, and is not conducive to subsequent image recognition, detection, and segmentation tasks, and urgently needs improvement. Summary of the Invention

[0005] This application provides an image denoising method and apparatus in intelligent image processing to solve the problems in related technologies, such as the inability to preserve image details while denoising, which reduces the visual performance of the image, decreases the peak signal-to-noise ratio and structural similarity of the image, and is not conducive to subsequent image recognition, detection and segmentation tasks.

[0006] The first aspect of this application provides an image denoising method in intelligent image processing, comprising the following steps: determining the black level bias value of a target image and subtracting the black level bias value to obtain a denominated image; determining the image denoising filtering method to be used based on image characteristics and processing requirements, and determining the size of the adjacency matrix of the filtering window according to the image denoising filtering method; calculating the filtering value of each pixel of the image based on the image denoising filtering and the size of the adjacency matrix of the filtering window; and updating the original pixel value of each pixel according to the filtering value to obtain the filtered image.

[0007] Optionally, in one embodiment of this application, determining the bias value of the black level of the target image and subtracting the bias value of the black level to obtain the subtracted image includes: capturing the target image under the condition of ensuring a preset zero light intake.

[0008] Optionally, in one embodiment of this application, determining the black level bias value of the target image and subtracting the black level bias value to obtain the subtracted image further includes: capturing the target image as a RAW format data image while ensuring that no light enters the lens; reading the data output of each pixel of the RAW format data image and calculating the black level bias value based on the data output; and subtracting the black level reference value from the global image.

[0009] Optionally, in one embodiment of this application, the step of reading the data output of each pixel of the RAW format image and calculating the bias value of the black level based on the data output includes: subtracting the black level by subtracting the mean, maximum value, median or local mean based on the data output of the R, G and B channels in the image; or, subtracting different values ​​of the black level based on different gains, and selecting whether linear stretching is needed for the G channel as required.

[0010] Optionally, in one embodiment of this application, determining the image denoising filtering method to be used based on image characteristics and processing requirements, and determining the size of the adjacency matrix of the filtering window according to the image denoising filtering method, includes: selecting the image denoising filtering method based on the image processing requirements and evaluation indicators, combined with the processor's computing power and the computational complexity of each filtering method; and selecting the size of the adjacency matrix of the filtering window according to the image denoising filtering method, combined with the image processing requirements and the processor's computing power.

[0011] Optionally, in one embodiment of this application, the image denoising filtering method includes at least one of median filtering, nonlocal mean filtering, median rational hybrid filtering, and denoising methods based on the human visual system.

[0012] Optionally, in one embodiment of this application, the step of calculating the filtered value of each pixel of the image based on the image denoising filter and the size of the filter window adjacency matrix includes: obtaining the filtered value of each pixel by calculating the median value or the weighted mean value based on the determined filter window matrix size and the filtering method.

[0013] A second aspect of this application provides an image denoising device for intelligent image processing, comprising: a subtraction module, configured to determine a black level bias value of a target image and subtract the black level bias value to obtain a subtracted image; a determination module, configured to determine the image denoising filtering method to be used based on image characteristics and processing requirements, and determine the size of the adjacency matrix of the filtering window according to the image denoising filtering method; a calculation module, configured to calculate the filtering value of each pixel of the image based on the image denoising filtering and the size of the adjacency matrix of the filtering window; and an update module, configured to update the original pixel value of each pixel according to the filtering value to obtain the filtered image.

[0014] Optionally, in one embodiment of this application, the subtraction module includes: a first shooting unit, used to capture the target image while ensuring a preset zero light intake.

[0015] Optionally, in one embodiment of this application, the subtraction module further includes: a second shooting unit, configured to capture the target image as a RAW format data image while ensuring that no light enters the lens; a first calculation unit, configured to read the data output of each pixel of the RAW format data image and calculate the bias value of the black level based on the data output; and a subtraction unit, configured to subtract the black level reference value from the global data.

[0016] Optionally, in one embodiment of this application, the first calculation unit includes: a first subtraction component, used to subtract the black level by subtracting the mean, maximum value, median or local mean based on the data output of the R, G and B channels in the image; or, a second subtraction component, used to subtract different values ​​of the black level according to different gains, and select whether linear stretching is required for the G channel as needed.

[0017] Optionally, in one embodiment of this application, the determining module includes: a first selection unit, configured to select the image denoising filtering method based on the image processing requirements and evaluation metrics, combined with the processor's computing power and the computational complexity of each filtering method; and a second selection unit, configured to select the size of the filtering window adjacency matrix according to the image denoising filtering method, combined with the image processing requirements and the processor's computing power.

[0018] Optionally, in one embodiment of this application, the image denoising filtering method includes at least one of median filtering, nonlocal mean filtering, median rational hybrid filtering, and denoising methods based on the human visual system.

[0019] Optionally, in one embodiment of this application, the calculation module includes: a second calculation unit, used to obtain the filtered value of each pixel by calculating the median value or the weighted mean value based on a determined filter window matrix size and filtering method.

[0020] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image denoising method in intelligent image processing as described in the above embodiments.

[0021] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image denoising method in the intelligent image processing described above.

[0022] This application's embodiments can perform image denoising processing by selecting an appropriate image denoising filtering method based on black level reference values, image processing requirements, and evaluation metrics. This reduces the impact of noise in the image while preserving more image details, improving visual performance, increasing peak signal-to-noise ratio (PSNR) and structural similarity, and facilitating subsequent image recognition, detection, and segmentation tasks. Therefore, it solves the problems in related technologies where image details cannot be preserved during denoising, resulting in reduced visual performance, decreased PSNR and structural similarity, and hindering subsequent image recognition, detection, and segmentation tasks.

[0023] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0024] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0025] Figure 1 This is a flowchart of an image denoising method in intelligent image processing according to an embodiment of this application;

[0026] Figure 2 This is a schematic diagram illustrating the principle of a median rational mixture filtering method according to an embodiment of this application;

[0027] Figure 3 This is a schematic diagram illustrating the principle of a nonlocal mean filtering method according to an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of the structure of an image denoising device in intelligent image processing according to an embodiment of this application;

[0029] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0030] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0031] The image denoising method and apparatus in intelligent image processing according to embodiments of this application are described below with reference to the accompanying drawings. Addressing the problem mentioned in the background section that related technologies cannot preserve image details while denoising, thus reducing image visual performance, peak signal-to-noise ratio (PSNR), and structural similarity, which is detrimental to subsequent image recognition, detection, and segmentation tasks, this application provides an image denoising method in intelligent image processing. In this method, based on black level reference values, image processing requirements, and evaluation metrics, an appropriate image denoising filtering method is selected for image denoising processing. This reduces the impact of noise in the image while preserving more image details, improving image visual performance, increasing PSNR and structural similarity, and facilitating subsequent image recognition, detection, and segmentation tasks. Therefore, this solves the problems in related technologies where image details cannot be preserved while denoising, reducing image visual performance, PSNR, and structural similarity, which is detrimental to subsequent image recognition, detection, and segmentation tasks.

[0032] Specifically, Figure 1 This is a flowchart illustrating an image denoising method in intelligent image processing provided in an embodiment of this application.

[0033] like Figure 1 As shown, the image denoising method in this intelligent image processing includes the following steps:

[0034] In step S101, the bias value of the black level of the target image is determined, and the bias value of the black level is subtracted to obtain the image after subtraction.

[0035] Optionally, in one embodiment of this application, determining the black level bias value of the target image and subtracting the black level bias value to obtain the subtracted image includes: capturing the target image while ensuring a preset zero light intake.

[0036] For example, embodiments of this application can capture images under conditions of zero light intake, determine the bias value of the black level, and subtract the black level value using a fixed value subtraction method.

[0037] Optionally, in one embodiment of this application, determining the black level bias value of the target image and subtracting the black level bias value to obtain the subtracted image further includes: ensuring that no light enters the lens, capturing the target image as a RAW format data image; reading the data output of each pixel of the RAW format data image, calculating the black level bias value based on the data output; and subtracting the black level reference value from the global image.

[0038] In actual operation, since the dark current generated by CMOS sensors is usually related to ambient temperature, it is necessary to ensure that the ambient temperature is stable during each calibration. When the ambient temperature changes, the black level needs to be recalibrated.

[0039] Optionally, in one embodiment of this application, reading the data output of each pixel of the RAW format image and calculating the black level bias value based on the data output includes: subtracting the black level by subtracting the mean, maximum value, median or local mean based on the data output of the R, G and B channels in the image; or, subtracting different values ​​of black level based on different gains, and selecting whether to perform linear stretching on the G channel as needed.

[0040] For example, embodiments of this application can use a fixed-value subtraction method to correct the black level. First, the black frame RAW data format image acquired by the sensor is divided into four channels: Gr, Gb, R, and B. Then, one of the average, median, maximum, and local mean values ​​of the four channels is calculated as a reference value for the black level. Then, the black level reference value obtained during correction is subtracted from each channel in subsequent images. Those skilled in the art can choose the method for calculating the black level reference value according to the actual situation, and no specific limitations are imposed here.

[0041] After obtaining the black level reference value, the output range may become smaller due to subtracting the black level reference value. Therefore, it is necessary to normalize the final output data, that is, to correct the output data. In the embodiments of this application, linear normalization is used to correct the output data. Assuming that the black level reference value measured for a certain pixel is blc, the gain is gain, and the brightness value obtained after a certain imaging is ln, the output value Out after subtracting the black level reference value and correcting can be expressed as follows:

[0042] Out = gain * (ln - blc)

[0043] Therefore, the range of the corrected output value can be normalized to [0, 255] for easier subsequent processing.

[0044] Furthermore, since the magnitude of the black level may be related to the gain, different black level reference values ​​can be subtracted according to different gains during actual execution. However, this will result in greater computational and storage complexity. Those skilled in the art can make the choice based on the actual situation, and no specific restrictions are imposed here.

[0045] In step S102, the image denoising filtering method to be used is determined based on the image characteristics and processing requirements, and the size of the adjacency matrix of the filtering window is determined according to the image denoising filtering method.

[0046] Optionally, in one embodiment of this application, determining the image denoising filtering method to be used based on image characteristics and processing requirements, and determining the size of the adjacency matrix of the filtering window according to the image denoising filtering method, includes: selecting an image denoising filtering method based on the image processing requirements and evaluation metrics, combined with the processor's computing power and the computational complexity of each filtering method; and selecting the size of the adjacency matrix of the filtering window according to the image denoising filtering method, combined with the image processing requirements and the processor's computing power.

[0047] For example, embodiments of this application can use peak signal-to-noise ratio (PSNR) or image structure similarity index as metrics for evaluating image processing. PSNR is a commonly used metric for assessing image quality; it quantifies image quality by comparing the difference between the original signal (reference image) and the processed signal (target image). PSNR is calculated based on the mean squared error (MSE) of the peak signal and noise, typically expressed logarithmically in decibels (dB). A higher PSNR value indicates a smaller difference between the two images, thus indicating better image quality. The formula for calculating PSNR is as follows:

[0048]

[0049]

[0050] Where R represents the maximum possible value of a pixel, typically 255 for RGB images, M and N represent the width and height of the image, and X(i,j) and Y(i,j) represent the pixel values ​​at the same position in the reference and target images, respectively.

[0051] Furthermore, SSIM (Structural Similarity Index) is also a commonly used metric for evaluating image quality. Compared to Peak Signal-to-Noise Ratio (PSNR), SSIM places greater emphasis on the subjective perception of an image by the human eye, and can more accurately assess the structural similarity and quality of an image. SSIM quantifies image quality by comparing differences in structure, brightness, and contrast between a reference image and a target image. The calculation of SSIM is based on three key factors: luminance similarity, contrast similarity, and structural similarity. Its range is between 0 and 1, with values ​​closer to 1 indicating better image quality. The formula for calculating SSIM is as follows:

[0052] SSIM = l(x,y) α *c(x,y) β *s(x,y) γ

[0053]

[0054] Where x and y represent the reference image and the target image, l(x,y), c(x,y), and s(x,y) represent the similarity of brightness, contrast, and structure, respectively, and α, β, and γ represent adjustable parameters. μ and σ represent the mean and standard deviation, respectively. In particular, σ xy Let c1, c2, and c3 represent the covariance of the two variables, and c1, c2, and c3 represent constants to avoid the case where the denominator is 0. Generally, we can take c3 = c2 / 2, α = β = γ = 1, then we can obtain:

[0055]

[0056] It should be noted that those skilled in the art can select appropriate image denoising evaluation metrics based on the actual situation, and no specific restrictions are imposed here.

[0057] Furthermore, embodiments of this application can employ median rational hybrid filtering to denoise the image. In this method, a 5x5 square matrix surrounding each pixel is taken as the filtering adjacency matrix, and the filtered value is calculated in subsequent steps based on the pixel values ​​within this adjacency matrix.

[0058] It should be noted that, in actual implementation, those skilled in the art need to select an appropriate adjacency matrix based on the chosen image denoising method and the arrangement characteristics of different primary color pixels, without making specific restrictions here.

[0059] Optionally, in one embodiment of this application, the image denoising filtering method includes at least one of median filtering, nonlocal mean filtering, median rational hybrid filtering, and denoising method based on the human visual system.

[0060] In step S103, the filter value of each pixel in the image is calculated based on the image denoising filter and the size of the adjacency matrix of the filter window.

[0061] Optionally, in one embodiment of this application, calculating the filtered value of each pixel in the image based on the image denoising filter and the size of the filter window adjacency matrix includes: obtaining the filtered value of each pixel by calculating the median value or the weighted mean value based on the determined filter window matrix size and the filtering method.

[0062] Specifically, the median rational hybrid filtering method is based on median filtering. Its essence is to apply multiple median or weighted median filtering methods to the neighborhood of a pixel, and then perform specific calculations on the multiple filtering results. For any pixel, a 3x3 neighborhood is typically selected, and three filtering methods are applied to this neighborhood: The first method uses a cross-shaped window with five pixels (horizontally and vertically) for median filtering, using the median of these five pixels as the filter value; the second method uses the same window but assigns a weight of 3 to the center pixel and a weight of 1 to the surrounding pixels, performing center-weighted median filtering; the third method uses an X-shaped window with five pixels at the four corners of the two diagonal lines for median filtering. After obtaining the three filtering results, the final filtering result is calculated using a formula.

[0063]

[0064] Where x, y, and z represent the results of the three methods, h and k represent specific parameters, and r is the final result.

[0065] However, for RAW domain images, the R, G, and B components are located in different positions. Therefore, for any pixel using any of the three filtering methods, it is desirable to obtain the value of its corresponding component for processing. Thus, for RAW domain images, the neighborhood values ​​of a pixel need to be 5x5 to obtain adjacent pixels in the same channel. Since the G component has twice as many pixels as the R and B components, there are slight differences in the location of the pixel being extracted. A diagram illustrating the pixel extraction process is shown below. Figure 2 As shown, the final result can be obtained by using the same formula after point filtering.

[0066] In step S104, the original pixel value of each pixel is updated according to the filtered value to obtain the filtered image.

[0067] In actual implementation, the embodiments of this application can calculate the filter value of each pixel and then update the original pixel value of the pixel with the filter value, thus completing the image denoising filtering. Afterwards, those skilled in the art can convert the processed RAW image into the required image format according to actual needs, without making specific limitations here.

[0068] The following is combined Figures 2 to 3 As shown, the working principle of the image denoising method in intelligent image processing of this application is explained in detail with multiple specific embodiments.

[0069] like Figure 2 As shown, the image denoising method in intelligent image processing of this application, when using median rational mixture filtering to filter an image, may specifically include the following steps:

[0070] Step S1: Take a picture under the condition of zero light intake, determine the black level bias value and use the fixed value subtraction method to subtract the black level value.

[0071] In actual operation, since the dark current generated by CMOS sensors is usually related to ambient temperature, it is necessary to ensure that the ambient temperature is stable during each calibration. When the ambient temperature changes, the black level needs to be recalibrated.

[0072] For example, embodiments of this application can use a fixed-value subtraction method to correct the black level. First, the black frame RAW data format image acquired by the sensor is divided into four channels: Gr, Gb, R, and B. Then, the average, median, maximum, and local mean values ​​of the four channels are calculated as a reference value for the black level. Finally, the black level reference value obtained during correction is subtracted from each channel in subsequent images. Those skilled in the art can choose the method for calculating the black level reference value according to the actual situation, and no specific limitations are imposed here.

[0073] After obtaining the black level reference value, the output range may become smaller due to subtracting the black level reference value. Therefore, it is necessary to normalize the final output data, that is, to correct the output data. In the embodiments of this application, linear normalization is used to correct the output data. Assuming that the black level reference value measured for a certain pixel is blc, the gain is gain, and the brightness value obtained after a certain imaging is ln, the output value Out after subtracting the black level reference value and correcting can be expressed as follows:

[0074] Out = gain * (ln - blc)

[0075] Therefore, the range of the corrected output value can be normalized to [0, 255] for easier subsequent processing.

[0076] Furthermore, since the magnitude of the black level may be related to the gain, different black level reference values ​​can be subtracted according to different gains during actual execution. However, this will bring greater computational and storage complexity. Those skilled in the art can make the choice according to the actual situation, and no specific restrictions are imposed here.

[0077] Step S2: Determine the image denoising filtering method to be used based on the image characteristics and processing requirements, and determine the size of the adjacency matrix of the filtering window according to the selected filtering method.

[0078] For example, embodiments of this application may use the peak signal-to-noise ratio (PSNR) or the image structure similarity index (SSI) as metrics for evaluating image processing. PSNR is calculated based on the mean square error between the peak signal and noise, typically expressed logarithmically in decibels (dB). A higher PSNR value indicates a smaller difference between two images, thus representing better image quality. The formula for calculating PSNR is as follows:

[0079]

[0080]

[0081] Where R represents the maximum possible value of a pixel, typically 255 for RGB images, M and N represent the width and height of the image, and X(i,j) and Y(i,j) represent the pixel values ​​at the same position in the reference and target images, respectively.

[0082] Compared to Peak Signal-to-Noise Ratio (SNR), SSIM places greater emphasis on the subjective perception of images by the human eye, enabling a more accurate assessment of structural similarity and quality. SSIM quantifies image quality by comparing differences in structure, brightness, and contrast between a reference image and a target image. SSIM calculation is based on three key factors: brightness similarity, contrast similarity, and structural similarity. These values ​​range from 0 to 1, with values ​​closer to 1 indicating better image quality. The SSIM calculation formula is as follows:

[0083] SSIM = l(x,y) α *c(x,y) β *s(x,y) γ

[0084]

[0085] Where x and y represent the reference image and the target image, l(x,y), c(x,y), and s(x,y) represent the similarity of brightness, contrast, and structure, respectively, and α, β, and γ are adjustable parameters. μ and σ are the mean and standard deviation, respectively. In particular, σ xyLet c1, c2, and c3 be the covariances of the two variables, and c1, c2, and c3 be constants to avoid the case where the denominator is 0. Generally, we can take c3 = c2 / 2, α = β = γ = 1, then we can obtain:

[0086]

[0087] For example, embodiments of this application may use peak signal-to-noise ratio (PSNR) or image structural similarity index as metrics for evaluating image processing. Those skilled in the art can select appropriate image denoising evaluation metrics based on actual circumstances, and no specific limitations are imposed here.

[0088] Step S3: Based on the determined filtering method and the size of the adjacency matrix of the filtering window, calculate the filtered value of each pixel.

[0089] A schematic diagram of the median rational mixture filtering method can be shown as follows: Figure 2 As shown, this filtering method is based on median filtering. Essentially, it applies multiple median or weighted median filtering methods to the neighborhood of a pixel, and then performs specific calculations on the multiple filtering results. For any given pixel, a 3x3 neighborhood is typically selected, and three filtering methods are applied to this neighborhood: The first method uses a cross-shaped window with five pixels (horizontally and vertically) for median filtering, using the median of these five pixels as the filter value; the second method uses the same window but assigns a weight of 3 to the center pixel and a weight of 1 to the surrounding pixels, performing center-weighted median filtering; the third method uses an X-shaped window with five pixels at the four corners of the two diagonal lines for median filtering. After obtaining the three filtering results, the final filtering result is calculated using a formula:

[0090]

[0091] In the formula, x, y, and z represent the results of the three methods, h and k represent specific parameters, and r represents the final result.

[0092] However, for RAW domain images, the R, G, and B components are located in different positions. Therefore, for any pixel using any of the three filtering methods, it is desirable to obtain the value of its corresponding component for processing. Thus, for RAW domain images, the neighborhood values ​​of a pixel need to be 5x5 to obtain adjacent pixels in the same channel. Since the G component has twice as many pixels as the R and B components, there are slight differences in the location of the pixel being extracted. A diagram illustrating the pixel extraction process is shown below. Figure 2 As shown, the final result can be obtained by using the same formula after point filtering.

[0093] Step S4: Update the original pixel value of each pixel according to the calculated filter value to obtain the filtered image.

[0094] In actual implementation, after calculating the filter value of each pixel in step S3, the original pixel value of the pixel is updated to the filter value, thus completing the image denoising filtering. Afterwards, those skilled in the art can convert the processed RAW image into the required image format according to actual needs, without any specific limitations.

[0095] Furthermore, such as Figure 3 As shown, the image denoising method in intelligent image processing of this application embodiment, when using non-local mean filtering to filter the image, may specifically include the following steps:

[0096] Step S1: Take a picture under the condition of zero light intake, determine the black level bias value and use the fixed value subtraction method to subtract the black level value.

[0097] In actual operation, since the dark current generated by CMOS sensors is usually related to ambient temperature, it is necessary to ensure that the ambient temperature is stable during each calibration. When the ambient temperature changes, the black level needs to be recalibrated.

[0098] For example, embodiments of this application can use a fixed-value subtraction method to correct the black level. First, the black frame RAW data format image acquired by the sensor is divided into four channels: Gr, Gb, R, and B. Then, the average, median, maximum, and local mean values ​​of the four channels are calculated as a reference value for the black level. Finally, the black level reference value obtained during correction is subtracted from each channel in subsequent images. Those skilled in the art can choose the method for calculating the black level reference value according to the actual situation, and no specific limitations are imposed here.

[0099] After obtaining the black level reference value, the output range may become smaller due to subtracting the black level reference value. Therefore, it is necessary to normalize the final output data, that is, to correct the output data. In the embodiments of this application, linear normalization is used to correct the output data. Assuming that the black level reference value measured for a certain pixel is blc, the gain is gain, and the brightness value obtained after a certain imaging is ln, the output value Out after subtracting the black level reference value and correcting can be expressed as follows:

[0100] Out = gain * (ln - blc)

[0101] Therefore, the range of the corrected output value can be normalized to [0, 255] for easier subsequent processing.

[0102] Furthermore, since the magnitude of the black level may be related to the gain, different black level reference values ​​can be subtracted according to different gains during actual execution. However, this would lead to greater computational and storage complexity. Those skilled in the art can make the choice based on the actual situation, and no specific restrictions are imposed here.

[0103] Step S2: Determine the image denoising filtering method to be used based on the image characteristics and processing requirements, and determine the size of the adjacency matrix of the filtering window according to the selected filtering method.

[0104] For example, embodiments of this application may use peak signal-to-noise ratio (PSNR) or image structural similarity index as indicators for evaluating image processing. Those skilled in the art can select appropriate image denoising evaluation indicators according to actual circumstances, and no specific limitations are imposed here.

[0105] Step S3: Based on the determined filtering method and the size of the adjacency matrix of the filtering window, calculate the filtered value of each pixel.

[0106] Specifically, a schematic diagram of the nonlocal mean filtering method can be shown as follows: Figure 3 As shown, this filtering method differs from the ordinary mean filtering method. It considers the similarity between the pixels' surrounding areas and the pixels themselves, and applies a weighted average. The specific implementation process is as follows:

[0107] (1) For any pixel in the image, take a larger search box around it, and the final filtered value will be calculated based on the pixels in this search box;

[0108] (2) For each pixel in the search box, there is a smaller neighborhood box. Subtract this neighborhood box from the neighborhood box of the origin pixel and sum the squares. The calculated value represents the similarity between the neighborhood of this pixel and the neighborhood of the origin pixel.

[0109] (3) Substitute the sum of squares into the Gaussian kernel so that the smaller the sum of squares, the larger the value, which means that the two regions are more similar and the weight is greater. Then normalize the final value of each pixel in the search box. In this way, the weight obtained from the similarity is obtained, that is, the higher the similarity, the larger the weight.

[0110] (4) After each pixel in the search box has a weight, the weighted average of the pixels in the search box is calculated based on this weight to obtain the filtered value of the original pixel.

[0111] The embodiments of this application can use a non-local mean filtering method to denoise the image, which can better preserve the image details while denoising.

[0112] Step S4: Update the original pixel value of each pixel according to the calculated filter value to obtain the filtered image.

[0113] In actual implementation, the embodiments of this application can update the original pixel value of each pixel to the filtered value after calculating the filtered value of each pixel through S3, thus completing the image denoising filtering. Afterwards, those skilled in the art can convert the processed RAW image to the required image format according to actual needs, without making specific limitations here.

[0114] Furthermore, the image denoising method in the intelligent image processing of this application embodiment, when using a denoising method based on the human visual system to filter the image, may specifically include the following steps:

[0115] Step S1: Take a picture under the condition of zero light intake, determine the black level bias value and use the fixed value subtraction method to subtract the black level value.

[0116] In actual operation, since the dark current generated by CMOS sensors is usually related to ambient temperature, it is necessary to ensure that the ambient temperature is stable during each calibration. When the ambient temperature changes, the black level needs to be recalibrated.

[0117] For example, embodiments of this application can use a fixed-value subtraction method to correct the black level. First, the black frame RAW data format image acquired by the sensor is divided into four channels: Gr, Gb, R, and B. Then, the average, median, maximum, and local mean values ​​of the four channels are calculated as a reference value for the black level. Finally, the black level reference value obtained during correction is subtracted from each channel in subsequent images. Those skilled in the art can choose the method for calculating the black level reference value according to the actual situation, and no specific limitations are imposed here.

[0118] Step S2: Determine the image denoising filtering method to be used based on the image characteristics and processing requirements, and determine the size of the adjacency matrix of the filtering window according to the selected filtering method.

[0119] For example, embodiments of this application may use peak signal-to-noise ratio (PSNR) or image structural similarity index as indicators for evaluating image processing. Those skilled in the art can select appropriate image denoising evaluation indicators according to actual circumstances, and no specific limitations are imposed here.

[0120] Step S3: Based on the determined filtering method and the size of the adjacency matrix of the filtering window, calculate the filtered value of each pixel.

[0121] In noise reduction methods based on the human visual system, noise in the RAW domain increases with brightness, resulting in higher noise levels in brighter areas, thus allowing for the use of stronger noise reduction techniques. However, because the human visual system has varying sensitivities to different brightness levels, changes in brightness are less noticeable in darker areas. Therefore, applying stronger noise reduction techniques to low-brightness areas, even if it results in the loss of image details, will not be significantly different to the human eye. Thus, a curve representing the change in noise reduction intensity with pixel brightness can be defined as a broken line that is high at both ends and low in the middle.

[0122] This denoising method can determine local flatness using HVS characteristics and self-iterative noise estimation, thereby determining how to allocate neighborhood weights.

[0123] From the 5x5 neighborhood of each pixel, select 8 pixels of the same channel located on the edge of the matrix to form the neighborhood information of that pixel. Calculate the absolute difference between each neighboring pixel and the center pixel to obtain the maximum value of the difference, D. max and minimum value D min This is used for subsequent processing. The specific steps are as follows:

[0124] (1) Signal Analyzer:

[0125] In this embodiment of the application, an HVS value can be determined for each pixel using the HVS characteristic curve.

[0126] (2) Texture analyzer:

[0127] In this embodiment, a texture threshold can be determined using the HVS value and a previously estimated noise level. This threshold is used to determine the flatness T of the current pixel region. d When the maximum difference D max When the difference exceeds a threshold, the flatness is 0, representing the most textured region; when the maximum difference is 0, the flatness is 1, representing the flattest region. Between the threshold and 0, as the maximum difference gradually decreases, the flatness linearly increases from 0 to 1. However, since the human eye is sensitive to the green (G) component but relatively insensitive to the red and blue (R and B) components, the flatness of the red and blue components can be linearly increased to 1 at a certain value, and remains at 1 between 0 and that value, thus ignoring some small red and blue differences.

[0128] (3) Similarity threshold and weight calculation:

[0129] After obtaining the region flatness, the embodiments of this application can determine the weights assigned to neighboring pixels. Based on the flatness, two similarity thresholds Th can be derived using a formula. low and Th high For the difference between each neighboring pixel, if it is less than Th... lowIf the similarity is the highest, then it is assigned the largest weight; if it is greater than Th, then it is assigned the largest weight. high If the similarity is zero, it indicates that there is virtually no similarity, and a weight of 0 is assigned; however, if the similarity is zero in Th... low and Th high The values ​​between these values ​​indicate a moderate similarity, with weights that decrease linearly (i.e., after linear interpolation).

[0130] (4) Calculation of weighted mean:

[0131] In this embodiment, after obtaining the weights of the neighborhood, the weighted average of all 8 neighborhood pixels is calculated and then normalized to obtain the final new pixel value.

[0132] (5) Noise level estimation:

[0133] After obtaining the flatness, embodiments of this application can update the noise level for the next step using the flatness. If it is T... d For a flat region where the noise level is 1, the noise level is set to the maximum difference D. max If it is T d For textured regions where the noise level is 0, the noise level remains the same as the previous pixel. For regions with intermediate flatness, the noise level is calculated using interpolation. Initially, there is an initial noise value because there is no previous noise level. In subsequent pixel calculations, the previous noise level is used first, and then the noise level is updated for the next calculation.

[0134] Step S4: Update the original pixel value of each pixel according to the calculated filter value to obtain the filtered image.

[0135] In actual implementation, the embodiments of this application can update the original pixel value of each pixel to the filtered value after calculating the filtered value of each pixel through S3, thus completing the image denoising filtering. Afterwards, those skilled in the art can convert the processed RAW image to the required image format according to actual needs, without making specific limitations here.

[0136] The image denoising method in intelligent image processing proposed in this application can perform image denoising processing by selecting an appropriate image denoising filtering method based on black level reference values, image processing requirements, and evaluation metrics. This reduces the impact of noise in the image while preserving more image details, improving the image's visual performance, increasing peak signal-to-noise ratio (PSNR) and structural similarity, and facilitating subsequent image recognition, detection, and segmentation tasks. Therefore, it solves the problems in related technologies where image details cannot be preserved during denoising, resulting in reduced visual performance, decreased PSNR and structural similarity, and hindering subsequent image recognition, detection, and segmentation tasks.

[0137] Next, with reference to the accompanying drawings, an image denoising apparatus in intelligent image processing according to an embodiment of this application is described.

[0138] Figure 4 This is a block diagram of an image denoising device in intelligent image processing according to an embodiment of this application.

[0139] like Figure 4 As shown, the image denoising device 10 in the intelligent image processing includes: a subtraction module 100, a determination module 200, a calculation module 300, and an update module 400.

[0140] The subtraction module 100 is used to determine the bias value of the black level of the target image and subtract the bias value of the black level to obtain the subtracted image.

[0141] The determination module 200 is used to determine the image denoising filtering method to be used based on the image characteristics and processing requirements, and to determine the size of the adjacency matrix of the filtering window according to the image denoising filtering method.

[0142] The calculation module 300 is used to calculate the filter value of each pixel in the image based on the image denoising filter and the size of the adjacency matrix of the filter window.

[0143] The update module 400 is used to update the original pixel value of each pixel according to the filter value to obtain the filtered image.

[0144] Optionally, in one embodiment of this application, the deduction module 100 includes: a first shooting unit.

[0145] The first imaging unit is used to capture a target image while ensuring a preset zero light intake.

[0146] Optionally, in one embodiment of this application, the subtraction module 100 further includes: a second shooting unit, a first calculation unit, and a subtraction unit.

[0147] The second shooting unit is used to capture target images in RAW format data images while ensuring that no light enters the lens.

[0148] The first calculation unit is used to read the data output of each pixel of the RAW format image and calculate the black level bias value based on the data output.

[0149] The subtraction unit is used to subtract the black level reference value from the global value.

[0150] Optionally, in one embodiment of this application, the first calculation unit includes: a first subtraction component or a second subtraction component.

[0151] The first subtraction component is used to subtract the black level based on the data output of the R, G, and B channels in the image, by subtracting the mean, maximum, median, or local mean.

[0152] Alternatively, a second subtraction component can be used to subtract different values ​​of black level based on different gains, and to select whether linear stretching is required for the G channel as needed.

[0153] Optionally, in one embodiment of this application, the determining module 200 includes: a first selection unit and a second selection unit.

[0154] The first selection unit is used to select an image denoising filtering method based on the image processing requirements and evaluation metrics, combined with the processor's computing power and the computational complexity of each filtering method.

[0155] The second selection unit is used to select the size of the adjacency matrix of the filtering window based on the image denoising filtering method, combined with the image processing requirements and the processor's computing power.

[0156] Optionally, in one embodiment of this application, the image denoising filtering method includes at least one of median filtering, nonlocal mean filtering, median rational hybrid filtering, and denoising method based on the human visual system.

[0157] Optionally, in one embodiment of this application, the calculation module 300 includes a second calculation unit for obtaining the filtered value of each pixel by calculating the median value or the weighted mean value based on a determined filter window matrix size and filtering method.

[0158] It should be noted that the foregoing explanation of the image denoising method embodiment in intelligent image processing also applies to the image denoising device in intelligent image processing of this embodiment, and will not be repeated here.

[0159] The image denoising device in intelligent image processing proposed in this application can perform image denoising processing by selecting an appropriate image denoising filtering method based on the black level reference value, image processing requirements, and evaluation indicators. This reduces the impact of noise in the image while preserving more image details, improving the image's visual performance, increasing the peak signal-to-noise ratio and structural similarity, and facilitating subsequent image recognition, detection, and segmentation tasks. Therefore, it solves the problems in related technologies where image details cannot be preserved while denoising, resulting in reduced visual performance, decreased peak signal-to-noise ratio and structural similarity, and hindering subsequent image recognition, detection, and segmentation tasks.

[0160] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0161] The memory 501, the processor 502, and the computer program stored on the memory 501 and capable of running on the processor 502.

[0162] When the processor 502 executes the program, it implements the image denoising method in the intelligent image processing provided in the above embodiments.

[0163] Furthermore, electronic devices also include:

[0164] Communication interface 503 is used for communication between memory 501 and processor 502.

[0165] The memory 501 is used to store computer programs that can run on the processor 502.

[0166] The memory 501 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0167] If the memory 501, processor 502, and communication interface 503 are implemented independently, then the communication interface 503, memory 501, and processor 502 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0168] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0169] Processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0170] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the image denoising method in the intelligent image processing described above.

[0171] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0172] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0173] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0174] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0175] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0176] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0177] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0178] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for image denoising in intelligent image processing, characterized in that, Includes the following steps: Determine the black level offset value of the target image and subtract the black level offset value to obtain the subtracted image. The process of determining the black level offset value of the target image and subtracting the black level offset value to obtain the subtracted image includes: the target image is a RAW format data image captured under the condition that no light enters the lens. Read the data output of each pixel of the RAW format image, calculate the black level bias value based on the data output, and subtract the black level bias value from the global value. The step of reading the data output of each pixel of the RAW format image and calculating the black level bias value based on the data output includes: Based on the data output of the R, G, and B channels in the image, the black level is subtracted by subtracting the mean, maximum value, median, or local mean. Alternatively, different values ​​of the black level can be subtracted based on different gains, and whether linear stretching of the G channel is required can be selected as needed. The process involves determining the required image denoising filtering method based on image characteristics and processing requirements, and then determining the size of the adjacency matrix of the filtering window according to the image denoising filtering method. This process includes: Based on the image processing requirements and evaluation metrics, and considering the processor's computing power and the computational complexity of each filtering method, the image denoising filtering method is selected. Based on the image denoising filtering method, and considering the image processing requirements and the processor's computing power, the size of the adjacency matrix of the filtering window is selected. Based on the image denoising filtering method and the size of the adjacency matrix of the filtering window, the filtered value of each pixel in the image is calculated; and The original pixel value of each pixel is updated according to the filtered value to obtain the filtered image.

2. The method of claim 1, wherein, The process of determining the black level bias value of the target image and subtracting the black level bias value to obtain the subtracted image includes: Under the condition of ensuring zero light intake, the target image is captured.

3. The method of claim 1, wherein, The image denoising filtering method includes at least one of the following: median filtering, nonlocal mean filtering, median rational hybrid filtering, and denoising method based on the human visual system.

4. The method of claim 1, wherein, The step of calculating the filtered value of each pixel in the image based on the image denoising filter and the size of the adjacency matrix of the filter window includes: Based on the determined size of the filtering window matrix and the filtering method, the filtered value of each pixel is obtained by calculating the median value or the weighted mean.

5. An image denoising device in intelligent image processing, characterized in that, include: The subtraction module is used to determine the black level offset value of the target image and subtract the black level offset value to obtain the subtracted image. The process of determining the black level offset value of the target image and subtracting the black level offset value to obtain the subtracted image includes: the target image is a RAW format data image captured under the condition that no light enters the lens. Read the data output of each pixel of the RAW format image, calculate the black level bias value based on the data output, and subtract the black level bias value from the global value. The step of reading the data output of each pixel of the RAW format image and calculating the black level bias value based on the data output includes: Based on the data output of the R, G, and B channels in the image, the black level is subtracted by subtracting the mean, maximum value, median, or local mean. Alternatively, different values ​​of the black level can be subtracted based on different gains, and whether linear stretching of the G channel is required can be selected as needed. The determination module is used to determine the image denoising filtering method to be used based on image characteristics and processing requirements, and to determine the size of the adjacency matrix of the filtering window according to the image denoising filtering method. The determination of the image denoising filtering method based on image characteristics and processing requirements, and the determination of the size of the adjacency matrix of the filtering window according to the image denoising filtering method, includes: Based on the image processing requirements and evaluation metrics, and considering the processor's computing power and the computational complexity of each filtering method, the image denoising filtering method is selected. Based on the image denoising filtering method, and considering the image processing requirements and the processor's computing power, the size of the adjacency matrix of the filtering window is selected. The calculation module is used to calculate the filtered value of each pixel in the image based on the image denoising filtering method and the size of the adjacency matrix of the filtering window; and The update module is used to update the original pixel value of each pixel according to the filter value to obtain the filtered image.

6. An electronic device, comprising: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the image denoising method in intelligent image processing as described in any one of claims 1-4.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the image denoising method in intelligent image processing as described in any one of claims 1-4.

Images