Low-illumination scene image intelligent noise reduction method, device, equipment and storage medium based on AI ISP

Abstract

The present application discloses a method, device, equipment and storage medium for intelligent noise reduction of low-light scene images based on AI ISP, the method comprising: separating the brightness channel and chromaticity channel of the low-light image, obtaining brightness information and chromaticity information based on the brightness channel and chromaticity channel respectively; determining the brightness distribution and chromaticity distribution of the brightness information and chromaticity information through histograms respectively, and obtaining brightness features and chromaticity features according to the brightness distribution and chromaticity distribution; performing local variance calculations on the brightness features and the chromaticity features respectively, determining the image noise distribution, partitioning the low-light image according to the image noise distribution, obtaining multiple image partitions, and the image noise distribution in different image partitions is different noise intervals; determining the noise extremes of the image partitions respectively, determining the noise reduction strategy based on the noise extremes, and performing noise reduction on the image partitions according to the noise reduction strategy to obtain a noise-reduced image. In the above manner, the image details of the low-light image are improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for intelligent noise reduction of low-light scene images based on AI ISP. Background Art

[0002] In low-light scenes, due to insufficient light, images captured by image sensors are usually affected by a lot of noise, which leads to a decrease in image quality. To improve this situation, traditional image signal processors (ISPs) use fixed algorithms and parameters for image processing, such as noise reduction, white balance, and exposure adjustment. However, in complex scenes, the lighting conditions in different areas vary greatly, resulting in inconsistent noise levels. Therefore, these traditional methods will lose image detail information when performing image noise reduction processing, resulting in a decrease in image quality, and thus fail to achieve the desired image quality effect.

[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a method, device, equipment and storage medium for intelligent noise reduction of low-light scene images based on AI ISP, aiming to solve the technical problem in the prior art that it is impossible to retain the details in the image during the noise reduction process.

[0005] To achieve the above objectives, the present application provides a low-illumination scene image intelligent noise reduction method based on AI ISP, the method comprising:

[0006] Separating a luminance channel and a chrominance channel of the low-light image, and obtaining luminance information and chrominance information based on the luminance channel and the chrominance channel respectively;

[0007] Determine brightness distribution and chromaticity distribution of the brightness information and the chromaticity information through histograms respectively, and obtain brightness features and chromaticity features according to the brightness distribution and chromaticity distribution;

[0008] Performing local variance calculation on the brightness feature and the chromaticity feature respectively to determine image noise distribution, partitioning the low-illuminance image according to the image noise distribution to obtain a plurality of image partitions, wherein the image noise distribution in different image partitions is in different noise intervals;

[0009] Determine the noise extremes of the image partitions respectively, determine an AI ISP noise reduction strategy based on the noise extremes, and perform noise reduction on the image partitions according to the AI ​​ISP noise reduction strategy to obtain a noise-reduced image.

[0010] In one embodiment, the step of separating the brightness channel and the chromaticity channel of the low-light image and obtaining brightness information and chromaticity information based on the brightness channel and the chromaticity channel respectively comprises:

[0011] Convert the color space of the low-light image into a YUV color space, and determine a brightness channel and a chromaticity channel in the low-light image, wherein the brightness channel is the Y channel in the YUV color space, and the chromaticity channel is the U channel and the V channel in the YUV color space;

[0012] Extracting a brightness component of the brightness channel, performing a histogram analysis on the brightness component to obtain a brightness distribution, and extracting brightness information according to the brightness distribution;

[0013] The chromaticity component of the chromaticity channel is extracted, a color histogram analysis is performed on the chromaticity component to obtain a color distribution in the low-light image, and chromaticity information is extracted according to the color distribution.

[0014] In one embodiment, the step of determining the brightness distribution and the chromaticity distribution of the brightness information and the chromaticity information through histograms respectively, and obtaining the brightness feature and the chromaticity feature according to the brightness distribution and the chromaticity distribution comprises:

[0015] Determining brightness distribution and chromaticity distribution of the brightness information and the chromaticity information through histograms respectively;

[0016] determining an abnormal exposure area in the low illumination image according to the brightness distribution;

[0017] Filtering the abnormal exposure area based on a bilateral filter to obtain texture information, and obtaining brightness features according to the contrast of the texture information;

[0018] The color mean of each pixel in the low-illuminance image is determined according to the chromaticity distribution, and the chromaticity feature is determined according to the color mean.

[0019] In one embodiment, the step of filtering the abnormal exposure area based on a bilateral filter to obtain texture information, and obtaining brightness features according to the contrast of the texture information includes:

[0020] Performing threshold segmentation on the abnormal exposure area to determine a pixel interval of the abnormal exposure area;

[0021] Expanding the pixel interval to an adjacent area to obtain a neighborhood interval;

[0022] Determine a brightness value of each pixel in the pixel interval, and aggregate the brightness values ​​into a brightness set;

[0023] The brightness set, the neighborhood interval, the normalization factor of the bilateral filter and the Gaussian kernel are combined to obtain texture information;

[0024] Calculating a gradient value of the texture information, and obtaining a texture direction and a texture intensity based on the gradient value;

[0025] Obtaining a structure tensor according to the texture direction and the texture strength;

[0026] A texture energy map is generated based on the structure tensor, and the texture energy map is analyzed to obtain a brightness feature.

[0027] In one embodiment, the step of determining the color mean of each pixel in the low-light image according to the chromaticity distribution, and determining the chromaticity feature according to the color mean comprises:

[0028] Determine a neighborhood local color mean of each pixel according to the chromaticity distribution, and determine a color vector field according to the neighborhood local color mean;

[0029] Obtaining a color difference according to the chromaticity component of the pixel point and the average value of the local color of the neighborhood;

[0030] A local color change amount is determined based on the color difference and the color vector field, a gradient change amplitude is calculated according to the local color change amount, and the gradient change amplitude is determined as a chromaticity feature.

[0031] In one embodiment, the steps of performing local variance calculation on the brightness feature and the chromaticity feature respectively to determine image noise distribution, and partitioning the low-illuminance image according to the image noise distribution to obtain a plurality of image partitions include:

[0032] Determine the size of the sliding window, and set the sliding window to the size of the sliding window;

[0033] Determining the brightness feature and the chromaticity feature of the pixel point selected by the sliding window;

[0034] Calculating local variances for the brightness feature and the chromaticity feature respectively to obtain a brightness mean and a chromaticity mean;

[0035] Obtaining a noise value of the pixel based on the brightness mean value and the chromaticity mean value;

[0036] Aggregating the noise values ​​of the plurality of pixel points to obtain image noise distribution;

[0037] The low illumination image is partitioned based on the image noise distribution to obtain a plurality of image partitions.

[0038] In one embodiment, the step of partitioning the low-illuminance image based on the image noise distribution to obtain a plurality of image partitions comprises:

[0039] Determine the noise partition threshold, compare the noise value of the pixel point with the noise partition threshold, add a first mark to the pixel point whose value is greater than the noise partition threshold, and add a second mark to the pixel point whose value is less than the noise partition threshold;

[0040] A noise partition schematic diagram is generated based on the image noise distribution, the first mark, and the second mark, and boundaries of each partition are determined to divide the low-illuminance image into a plurality of image partitions.

[0041] In one embodiment, the steps of respectively determining the noise extremes of the image partitions, determining an AI ISP noise reduction strategy based on the noise extremes, and performing noise reduction on the image partitions according to the AI ​​ISP noise reduction strategy to obtain a noise-reduced image include:

[0042] Determine the maximum noise value and the minimum noise value in the image partition respectively, and determine the noise range of the current image partition according to the maximum noise value and the minimum noise value;

[0043] Determine an initial noise reduction parameter according to the partition type of the image partition, and modify the initial noise reduction parameter based on the noise range to obtain a target noise reduction parameter for the current image partition;

[0044] Performing noise reduction processing on each pixel point in the current image partition based on the target noise reduction parameter to obtain a noise-reduced image partition;

[0045] A plurality of the denoised image partitions are aggregated to obtain a denoised image.

[0046] In one embodiment, the step of determining the initial noise reduction parameter according to the partition type of the image partition, and correcting the initial noise reduction parameter based on the noise range to obtain the target noise reduction parameter of the current image partition includes:

[0047] Determine a partition type of the image partition, and search a parameter table according to the partition type to obtain an initial noise reduction parameter of the partition type;

[0048] Determine a correction factor according to the noise range, a first difference between the maximum noise value and the average noise value, and a second difference between the minimum noise value and the average noise value;

[0049] The initial noise reduction parameter is modified according to the correction factor to obtain the target noise reduction parameter of the current image partition.

[0050] In one embodiment, the step of performing noise reduction processing on each pixel point in the current image partition based on the target noise reduction parameter to obtain the noise-reduced image partition includes:

[0051] Determine the noise value of each pixel in the current partition and generate a noise value set;

[0052] Determine a target noise reduction parameter for the current partition, calculate the noise value set and the target noise reduction parameter, and obtain a target noise value of the pixel after noise reduction;

[0053] The noise value of each pixel point of the current image partition is updated to the target noise value to obtain a denoised image partition.

[0054] In addition, to achieve the above purpose, the present application also proposes an intelligent noise reduction device for low-illumination scene images based on AI ISP. The intelligent noise reduction device for low-illumination scene images based on AI ISP includes:

[0055] A channel separation module, used to separate the brightness channel and the chromaticity channel of the low-light image, and obtain brightness information and chromaticity information based on the brightness channel and the chromaticity channel respectively;

[0056] A feature extraction module, used to determine the brightness distribution and the chromaticity distribution of the brightness information and the chromaticity information through histograms, and obtain brightness features and chromaticity features according to the brightness distribution and the chromaticity distribution;

[0057] An image partitioning module, used to perform local variance calculation on the brightness feature and the chromaticity feature respectively, determine image noise distribution, and partition the low-illuminance image according to the image noise distribution to obtain a plurality of image partitions, wherein the image noise distribution in different image partitions is in different noise intervals;

[0058] The image denoising module is used to respectively determine the noise extremes of the image partitions, determine the AI ​​ISP denoising strategy based on the noise extremes, and denoise the image partitions according to the AI ​​ISP denoising strategy to obtain a denoised image.

[0059] In addition, to achieve the above-mentioned objectives, the present application also proposes a low-illumination scene image intelligent noise reduction device based on AI ISP, and the low-illumination scene image intelligent noise reduction device based on AI ISP includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the low-illumination scene image intelligent noise reduction method based on AI ISP as described above.

[0060] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the low-illumination scene image intelligent denoising method based on AIISP are implemented as described above.

[0061] In addition, to achieve the above-mentioned objectives, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the low-illumination scene image intelligent noise reduction method based on AI ISP as described above.

[0062] The present application provides an intelligent denoising method for low-illuminance scene images based on AI ISP, by separating the brightness channel and chromaticity channel of the low-illuminance image, and obtaining brightness information and chromaticity information based on the brightness channel and the chromaticity channel respectively; determining the brightness distribution and chromaticity distribution of the brightness information and the chromaticity information through histograms respectively, and obtaining brightness features and chromaticity features according to the brightness distribution and chromaticity distribution; performing local variance calculation on the brightness features and the chromaticity features respectively to determine the image noise distribution, and partitioning the low-illuminance image according to the image noise distribution to obtain multiple image partitions, and the image noise distribution in different image partitions is in different noise intervals; determining the noise extremes of the image partitions respectively, determining the AI ​​ISP denoising strategy based on the noise extremes, and denoising the image partitions according to the AI ​​ISP denoising strategy to obtain a denoised image. The present application processes low-light images from two dimensions, brightness and chromaticity, and can determine the noise value of the image from different aspects, thereby improving the optimization angle of the noise value, solving the technical problem of poor noise reduction effect in the image noise reduction process in the prior art, improving the image details of the low-light image, and reducing the impact of noise on the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0064] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0065] Figure 1 This is a flow chart of Embodiment 1 of the method for intelligent noise reduction of low-light scene images based on AI ISP of the present application;

[0066] Figure 2The luminance distribution and chromaticity distribution histograms of an embodiment of an intelligent noise reduction method for low-light scene images based on AI ISP of this application;

[0067] Figure 3 This is a comparison diagram before and after noise reduction of an embodiment of an intelligent noise reduction method for low-illumination scene images based on AI ISP in this application;

[0068] Figure 4 This is a schematic diagram of the module structure of an intelligent noise reduction device for low-illumination scene images based on AI ISP according to an embodiment of the present application;

[0069] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the low-illumination scene image intelligent denoising method based on AI ISP in an embodiment of the present application.

[0070] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0071] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0072] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0073] The main solution of the embodiment of the present application is: separate the brightness channel and chromaticity channel of the low-light image, and obtain brightness information and chromaticity information based on the brightness channel and the chromaticity channel respectively; determine the brightness distribution and chromaticity distribution of the brightness information and the chromaticity information through histograms respectively, and obtain brightness features and chromaticity features according to the brightness distribution and chromaticity distribution; perform local variance calculation on the brightness features and the chromaticity features respectively to determine the image noise distribution, and partition the low-light image according to the image noise distribution to obtain multiple image partitions, and the image noise distribution in different image partitions is different noise intervals; determine the noise extremes of the image partitions respectively, determine the AI ​​ISP noise reduction strategy based on the noise extremes, and denoise the image partitions according to the AI ​​ISP noise reduction strategy to obtain a denoised image.

[0074] At present, traditional image signal processors (ISPs) use fixed algorithms and parameters for image processing, such as noise reduction, white balance, and exposure adjustment. However, in complex scenes, the lighting conditions in different areas vary greatly, resulting in inconsistent noise levels. Therefore, these traditional methods will lose image detail information when performing image noise reduction, resulting in a decrease in image quality, and thus fail to achieve the desired image quality effect.

[0075] The present application provides a solution, by separating the brightness channel and chromaticity channel of the low-light image, obtaining brightness information and chromaticity information based on the brightness channel and chromaticity channel respectively; determining the brightness distribution and chromaticity distribution of the brightness information and chromaticity information through histograms respectively, and obtaining brightness features and chromaticity features according to the brightness distribution and chromaticity distribution; performing local variance calculations on the brightness features and the chromaticity features respectively, determining the image noise distribution, partitioning the low-light image according to the image noise distribution, obtaining multiple image partitions, and the image noise distribution in different image partitions is different noise intervals; determining the noise extremes of the image partitions respectively, determining the AI ​​ISP noise reduction strategy based on the noise extremes, and performing noise reduction on the image partitions according to the AI ​​ISP noise reduction strategy to obtain a noise-reduced image. In the above manner, the image details of the low-light image are improved.

[0076] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device that can realize the above functions, an AI ISP-based low-light scene image intelligent noise reduction device, etc., and this embodiment does not specifically limit this. The following takes the AI ​​ISP-based low-light scene image intelligent noise reduction device as an example to illustrate this embodiment and the following embodiments.

[0077] The present application embodiment provides a low-light scene image intelligent noise reduction method based on AI ISP, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the low-illumination scene image intelligent noise reduction method based on AI ISP in this application.

[0078] In this embodiment, the low-illumination scene image intelligent noise reduction method based on AI ISP includes steps S10 to S40:

[0079] Step S10, separating the brightness channel and the chromaticity channel of the low-light image, and obtaining brightness information and chromaticity information based on the brightness channel and the chromaticity channel respectively.

[0080] It should be noted that low-light images refer to photos or video frames taken in low-light environments, which usually have the characteristics of high noise, color distortion, low contrast, underexposure, etc. The luminance channel and chrominance channel can represent the brightness information and color information of the image respectively.

[0081] In a specific implementation, when an image signal processor (ISP) captures a low-illumination image, it is capable of performing image processing on the current low-illumination image, converting the color space of the low-illumination image into a YUV color space, and determining a brightness channel and a chromaticity channel in the low-illumination image, wherein the brightness channel is the Y channel in the YUV color space, and the chromaticity channel is the U channel and the V channel in the YUV color space; extracting a brightness component of the brightness channel, performing a histogram analysis on the brightness component, obtaining a brightness distribution, and extracting brightness information based on the brightness distribution; extracting a chromaticity component of the chromaticity channel, performing a color histogram analysis on the chromaticity component, obtaining a color distribution in the low-illumination image, and extracting chromaticity information based on the color distribution.

[0082] Reference Figure 2 , Figure 2 is the brightness distribution and chromaticity distribution histogram. When performing color space conversion, the low-light image originally in RGB color space can be converted into YUV color space, where the image information in YUV color space consists of Y channel, U channel and V channel, where the Y channel is the brightness channel, indicating the brightness information of the pixel value, the U channel is the chromaticity channel of the red component, and the V channel is the chromaticity channel of the blue component. The conversion formula from RGB to YUV is:

[0083]

[0084]

[0085]

[0086] Among them, Y is the brightness channel, and U and V are the chrominance channels.

[0087] In the YUV color space, the Y channel directly represents the brightness component. Therefore, the Y channel can be used directly as the brightness component. In order to obtain the brightness distribution, a histogram analysis can be performed on the Y channel. The histogram is a discrete function that represents the frequency of occurrence of each brightness value. Assuming that the pixel value range of the Y channel is 0 to 255, the histogram It can be defined as:

[0088]

[0089] Where M and N are the height and width of the image respectively. It's location The brightness value at It's Dirac Function, when 1 if the value is 0, otherwise it is 0.

[0090] According to the brightness distribution, various brightness information can be extracted, such as average brightness, cumulative distribution function of brightness histogram, and brightness contrast.

[0091] Average brightness, which can calculate the average value of all brightness.

[0092]

[0093] Cumulative distribution function (CDF) of the brightness histogram: used to evaluate the overall distribution of brightness.

[0094]

[0095] Brightness contrast: can be measured by calculating the standard deviation of brightness values.

[0096]

[0097] In the YUV color space, the U and V channels represent the chrominance components. Therefore, the U and V channels can be used directly as chrominance components. In order to obtain the color distribution, the color histogram analysis can be performed on the U and V channels. Assuming that the pixel value range of the U and V channels is 0 to 255, the color histogram and It can be defined as:

[0098]

[0099]

[0100] According to the color distribution, various chromaticity information can be extracted, such as average chromaticity, cumulative distribution function of chromaticity histogram, and chromaticity saturation.

[0101] Average chromaticity, which can calculate the average value of all chromaticities.

[0102]

[0103]

[0104] Cumulative distribution function (CDF) of the chromaticity histogram: used to evaluate the overall distribution of chromaticity.

[0105]

[0106]

[0107] Chroma saturation: can be measured by calculating the standard deviation of the chroma values.

[0108]

[0109]

[0110] Through the above steps, the brightness channel and chromaticity channel of the low-light image can be effectively separated, and the brightness information and chromaticity information can be extracted respectively. This information can be used for subsequent image enhancement, noise reduction, contrast adjustment and other processing, thereby improving the quality of the low-light image.

[0111] Step S20, determining the brightness distribution and the chromaticity distribution of the brightness information and the chromaticity information through histograms respectively, and obtaining brightness characteristics and chromaticity characteristics according to the brightness distribution and the chromaticity distribution.

[0112] It should be noted that brightness distribution refers to the frequency of occurrence of different brightness values ​​in an image. It is usually represented by a brightness histogram, where the horizontal axis represents the brightness value (from 0 to 255) and the vertical axis represents the number of times or proportion of the brightness value in the image. Similarly, chromaticity distribution is similar to brightness distribution, with the horizontal axis representing the chromaticity value and the vertical axis representing the number of times or proportion of the chromaticity value in the image.

[0113] In a feasible implementation manner, the step of determining the brightness distribution and the chromaticity distribution of the brightness information and the chromaticity information through histograms respectively, and obtaining the brightness feature and the chromaticity feature according to the brightness distribution and the chromaticity distribution comprises:

[0114] Determining brightness distribution and chromaticity distribution of the brightness information and the chromaticity information through histograms respectively;

[0115] determining an abnormal exposure area in the low illumination image according to the brightness distribution;

[0116] Filtering the abnormal exposure area based on a bilateral filter to obtain texture information, and obtaining brightness features according to the contrast of the texture information;

[0117] The color mean of each pixel in the low-illuminance image is determined according to the chromaticity distribution, and the chromaticity feature is determined according to the color mean.

[0118] It should be noted that the abnormal exposure area refers to the area formed by pixels whose exposure values ​​are not between normal brightness values. The abnormal exposure can be overexposure or underexposure, and is usually underexposure in low-light images.

[0119] In a specific implementation, after determining the brightness distribution and chromaticity distribution through the histogram of the brightness information and the chromaticity information, the abnormal exposure area in the low-light image can be determined based on the brightness distribution. Then the abnormal exposure area is filtered by a bilateral filter. After bilateral filtering, texture information can be extracted from the filtered brightness image. Texture information can be obtained by calculating the local contrast. The local contrast can be calculated by the following formula:

[0120]

[0121] in, is the number of pixels in the neighborhood window, is the average brightness value in the neighborhood window.

[0122] When determining the brightness feature, texture information can be extracted to obtain the brightness feature, including the average contrast and the contrast standard deviation.

[0123] The average contrast is the average value of the local contrast of the pixels, calculated as:

[0124]

[0125] The contrast standard deviation is a measure of the range of local contrast variation and can be expressed as:

[0126]

[0127] Then, the color mean of each pixel in the low-light image is determined according to the chromaticity distribution, and the chromaticity feature is determined according to the color mean.

[0128] In a feasible implementation manner, the step of filtering the abnormal exposure area based on a bilateral filter to obtain texture information, and obtaining brightness features according to the contrast of the texture information includes:

[0129] Performing threshold segmentation on the abnormal exposure area to determine a pixel interval of the abnormal exposure area;

[0130] Expanding the pixel interval to an adjacent area to obtain a neighborhood interval;

[0131] Determine a brightness value of each pixel in the pixel interval, and aggregate the brightness values ​​into a brightness set;

[0132] The brightness set, the neighborhood interval, the normalization factor of the bilateral filter and the Gaussian kernel are combined to obtain texture information;

[0133] Calculating a gradient value of the texture information, and obtaining a texture direction and a texture intensity based on the gradient value;

[0134] Obtaining a structure tensor according to the texture direction and the texture strength;

[0135] A texture energy map is generated based on the structure tensor, and the texture energy map is analyzed to obtain a brightness feature.

[0136] It should be noted that the bilateral filter is a nonlinear filter that can retain important edge and texture information while smoothing the image. The bilateral filter can be expressed as:

[0137]

[0138] in, is the intensity value of pixel p, is the neighborhood window centered on p, is the spatial Gaussian kernel, controlling the influence of spatial distance, is the range Gaussian kernel, controlling the effect of pixel intensity differences, It is a normalization factor to ensure that the pixel value after filtering is within a reasonable range.

[0139] In the specific implementation, the abnormal exposure area is threshold segmented to divide the image into normal exposure area and abnormal exposure area. This can be achieved through adaptive thresholds such as the Otsu algorithm. The same processing method is performed on each pixel in the pixel interval to expand to the adjacent area to obtain the neighborhood interval. At the same time, the brightness value of each pixel in the pixel interval is aggregated to obtain a brightness set. Then, the texture information is obtained according to the brightness set, the neighborhood interval, the normalization factor of the bilateral filter and the Gaussian kernel. Then, the gradient value of the texture information is calculated, the speed-up operator is calculated, and the gradient amplitude and direction of the filtered image are calculated. The gradient amplitude reflects the intensity of the brightness change in the image, and the gradient direction indicates the direction of the change. Then, a structure tensor is constructed. The structure tensor is a 2×2 matrix used to describe the texture features of the local area. The structure tensor can quantify the direction and intensity of the texture. And by calculating the eigenvalue of the structure tensor, a texture energy map is generated. The larger the eigenvalue, the richer the texture of the area, and the smaller the eigenvalue, the smoother the area. After that, wavelet analysis is performed on the texture energy map to decompose the filtered image into representations at multiple scales. The low-frequency components contain the structural information of the image, while the high-frequency components reflect the texture details. Then the local contrast of each pixel is calculated, and the obtained local contrast is used as the extracted brightness feature. The calculation method is:

[0140]

[0141] Among them, q is the pixel in the neighborhood window centered on p.

[0142] In a feasible implementation manner, the step of determining the color mean of each pixel in the low-light image according to the chromaticity distribution, and determining the chromaticity feature according to the color mean includes:

[0143] Determine a neighborhood local color mean of each pixel according to the chromaticity distribution, and determine a color vector field according to the neighborhood local color mean;

[0144] Obtaining a color difference according to the chromaticity component of the pixel point and the average value of the local color of the neighborhood;

[0145] A local color change amount is determined based on the color difference and the color vector field, a gradient change amplitude is calculated according to the local color change amount, and the gradient change amplitude is determined as a chromaticity feature.

[0146] In the specific implementation, the local color mean of the neighborhood of each pixel is determined according to the chromaticity distribution, and the color vector field is determined according to the local color mean of the neighborhood; for each pixel, the local mean of the chromaticity components U and V in the neighborhood is calculated, and the calculation formula is:

[0147]

[0148]

[0149] in, is the neighborhood window centered on p, M is the number of pixels in the window, and are the U and V values ​​of the pixels in the window respectively.

[0150] Represent the local mean of each pixel as a color vector field , which is used to describe the color distribution of different areas in the image. Then the difference between the chromaticity component of each pixel and the mean of its neighborhood is calculated using the following formula:

[0151]

[0152]

[0153] And calculate the variance between the chromaticity component of each pixel and its neighborhood mean, the calculation formula is:

[0154]

[0155]

[0156] Step S30, performing local variance calculation on the brightness feature and the chromaticity feature respectively to determine image noise distribution, partitioning the low-illuminance image according to the image noise distribution to obtain multiple image partitions, wherein the image noise distribution in different image partitions is different noise intervals.

[0157] It should be noted that the image partitioning is to divide the low-illumination image based on the noise value to obtain a plurality of image partitions, and each image partition corresponds to a noise interval.

[0158] It can be understood that after calculating the local variance of the brightness feature and the chromaticity feature respectively, the noise distribution of the image can be determined by constructing a noise histogram based on the local variance values ​​of all pixels. According to the determined noise interval, the low-light image is divided into multiple image partitions, each partition corresponds to a different noise level, and a noise label is assigned to each pixel to indicate the noise interval to which it belongs. For example, 0 can be used to represent a low noise area, 1 to represent a medium noise area, and 2 to represent a high noise area. The connected region analysis algorithm (such as morphologically based connected region labeling) is used to merge adjacent pixels with the same noise label into one partition. This ensures that the noise level in each partition is relatively consistent.

[0159] In a feasible implementation manner, the steps of respectively performing local variance calculation on the brightness feature and the chromaticity feature to determine image noise distribution, partitioning the low-illuminance image according to the image noise distribution, and obtaining a plurality of image partitions include:

[0160] Determine the size of the sliding window, and set the sliding window to the size of the sliding window;

[0161] Determining the brightness feature and the chromaticity feature of the pixel point selected by the sliding window;

[0162] Calculating local variances for the brightness feature and the chromaticity feature respectively to obtain a brightness mean and a chromaticity mean;

[0163] Obtaining a noise value of the pixel based on the brightness mean value and the chromaticity mean value;

[0164] Aggregating the noise values ​​of the plurality of pixel points to obtain image noise distribution;

[0165] The low illumination image is partitioned based on the image noise distribution to obtain a plurality of image partitions.

[0166] It should be noted that the selection of the sliding window size depends on the resolution of the image and the required processing accuracy. Common sliding window sizes include 3x3, 5x5, 7x7, etc. A larger sliding window can capture more context information, but may smooth out details; a smaller sliding window can retain more details, but may be sensitive to noise, so the sliding window size is not limited in this embodiment.

[0167] In a specific implementation, for low-light images, this embodiment uses a 5x5 sliding window as an example to effectively capture local features while maintaining details. The sliding window is set to a selected size, and it is ensured that the sliding window can be moved pixel by pixel on the image, and the center position of the sliding window corresponds to the currently processed pixel. For each sliding window position, the brightness value Y(i, j) of all pixels in the sliding window is extracted, and the brightness mean and brightness standard value are calculated, and the chromaticity features in the sliding window are extracted, and the chromaticity mean and chromaticity standard deviation are calculated. Then the local variance is calculated based on the brightness feature and the chromaticity feature to obtain the brightness mean and the chromaticity mean. The local variance of the brightness feature can be expressed as:

[0168]

[0169] The local variance of the chromaticity feature can be expressed as:

[0170]

[0171]

[0172] In order to obtain the noise value of each pixel, the local variance of the brightness feature and the chromaticity feature can be weighted and combined, and the weights are , , , comprehensive noise value It is expressed as:

[0173]

[0174] The noise value of all pixels Collect them and construct a noise histogram. The noise histogram shows the frequency of occurrence of different noise levels. Then, the image noise distribution is obtained, the noise partition threshold is determined, the noise value of the pixel point is compared with the noise partition threshold, the pixel point greater than the noise partition threshold is added with a first mark, and the pixel point less than the noise partition threshold is added with a second mark; based on the image noise distribution, the first mark and the second mark, a noise partition map is generated, and the boundaries of each partition are determined, and the low-light image is divided into multiple image partitions.

[0175] Step S40: respectively determine the noise range of the image partitions, determine the AI ​​ISP noise reduction strategy based on the noise range, and perform noise reduction on the image partitions according to the AI ​​ISP noise reduction strategy to obtain a noise-reduced image.

[0176] It should be noted that the noise range refers to the difference between the maximum noise value and the minimum noise value in the image partition, and the noise range can reflect the uniformity of the noise value in the image partition. There is a mapping relationship between the AI ​​ISP noise reduction strategy and the noise range. When the noise range is consistent with the target noise range interval corresponding to the AI ​​ISP noise reduction strategy, the current AI ISP noise reduction strategy is determined.

[0177] In a feasible implementation manner, the steps of respectively determining the noise extremes of the image partitions, determining the AI ​​ISP noise reduction strategy based on the noise extremes, and performing noise reduction on the image partitions according to the AI ​​ISP noise reduction strategy to obtain the noise-reduced image include:

[0178] Determine the maximum noise value and the minimum noise value in the image partition respectively, and determine the noise range of the current image partition according to the maximum noise value and the minimum noise value;

[0179] Determine an initial noise reduction parameter according to the partition type of the image partition, and modify the initial noise reduction parameter based on the noise range to obtain a target noise reduction parameter for the current image partition;

[0180] Performing noise reduction processing on each pixel point in the current image partition based on the target noise reduction parameter to obtain a noise-reduced image partition;

[0181] A plurality of the denoised image partitions are aggregated to obtain a denoised image.

[0182] In the specific implementation, for each image partition, we need to calculate the maximum and minimum noise values ​​in the area. Assume that the current image partition is p, and the noise value of the pixel point in the period is , then the maximum noise value and the minimum noise value are:

[0183]

[0184]

[0185] Calculate the noise range of the current image partition based on the maximum noise value and the minimum noise value .

[0186]

[0187] According to the noise partition threshold determined previously, the image partitions are divided into different types, such as low noise area, medium noise area, and high noise area. A set of initial noise reduction parameters can be preset for each type of partition. For example, the initial bilateral filter parameters are set as:

[0188] Filter window size: 5×5;

[0189] Spatial standard deviation: 1.0;

[0190] Intensity standard deviation: 10 (low noise area), 20 (medium noise area), 30 (high noise area);

[0191] Adjust the initial noise reduction parameters according to the noise range of the current image partition. The larger the noise range, the more drastic the noise fluctuation in the area, and a stronger noise reduction process is required; conversely, the smaller the noise range, the relatively uniform the noise, and a weaker noise reduction process can be used. Perform noise reduction on each pixel in the current image partition according to the corrected target noise reduction parameters. Ensure that the boundary consistency between the partitions is maintained during the processing to avoid obvious splicing marks. In order to ensure the effect, the processing method is as follows:

[0192]

[0193] Reference Figure 3 , Figure 3 This is a comparison chart before and after noise reduction. After noise reduction processing, the noise reduction result of the current image partition is obtained. Ensure that the noise reduction effect of each partition meets expectations, and record the image data after noise reduction to obtain a noise-reduced image.

[0194] In a feasible implementation manner, the step of determining the initial noise reduction parameter according to the partition type of the image partition, and correcting the initial noise reduction parameter based on the noise range to obtain the target noise reduction parameter of the current image partition includes:

[0195] Determine a partition type of the image partition, and search a parameter table according to the partition type to obtain an initial noise reduction parameter of the partition type;

[0196] Determine a correction factor according to the noise range, a first difference between the maximum noise value and the average noise value, and a second difference between the minimum noise value and the average noise value;

[0197] The initial noise reduction parameter is modified according to the correction factor to obtain the target noise reduction parameter of the current image partition.

[0198] In a specific implementation, after determining the image partition type, the initial noise reduction parameters corresponding to the partition type can be determined, and then the noise range, the maximum noise value and the first difference between the average noise value are used to calculate the noise reduction parameters. , the second difference between the minimum noise value and the average noise value Get the correction factor The initial noise reduction parameters are modified according to the correction factor to obtain the target noise reduction parameters of the current image partition.

[0199] In a feasible implementation manner, the step of performing noise reduction processing on each pixel point in the current image partition based on the target noise reduction parameter to obtain the noise-reduced image partition includes:

[0200] Determine the noise value of each pixel in the current partition and generate a noise value set;

[0201] Determine a target noise reduction parameter for the current partition, calculate the noise value set and the target noise reduction parameter, and obtain a target noise value of the pixel after noise reduction;

[0202] The noise value of each pixel point of the current image partition is updated to the target noise value to obtain a denoised image partition.

[0203] In the specific implementation, a noise value set can be generated according to the noise value of each pixel point in the current partition, and then the target noise reduction parameter of the current partition is determined, and the target noise reduction parameter is calculated with each noise value in the noise reduction value set to obtain the target noise value after pixel noise reduction. The target noise value will be much lower than the original noise value. Finally, the noise value of each pixel point in the image partition can be updated to the target noise value, thereby achieving the purpose of noise reduction of the current image partition and obtaining the denoised image partition.

[0204] This embodiment provides an intelligent noise reduction method for low-light scene images based on AI ISP, by separating the brightness channel and chromaticity channel of the low-light image, respectively obtaining brightness information and chromaticity information based on the brightness channel and chromaticity channel; respectively determining the brightness distribution and chromaticity distribution of the brightness information and chromaticity information through histograms, and obtaining brightness features and chromaticity features based on the brightness distribution and chromaticity distribution; respectively performing local variance calculations on the brightness features and the chromaticity features to determine the image noise distribution, partitioning the low-light image according to the image noise distribution to obtain multiple image partitions, and the image noise distribution in different image partitions is different noise intervals; respectively determining the noise extremes of the image partitions, determining the AI ​​ISP noise reduction strategy based on the noise extremes, and performing noise reduction on the image partitions according to the AI ​​ISP noise reduction strategy to obtain a noise-reduced image. In the above manner, the image details of the low-light image are improved.

[0205] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the low-light scene image intelligent noise reduction method based on AI ISP of the present application. More forms of simple transformations based on this technical concept are all within the scope of protection of the present application.

[0206] This application also provides a low-light scene image intelligent noise reduction device based on AI ISP, please refer to Figure 4 , the low-light scene image intelligent noise reduction device based on AI ISP includes:

[0207] A channel separation module 10 is used to separate a brightness channel and a chromaticity channel of a low-light image, and obtain brightness information and chromaticity information based on the brightness channel and the chromaticity channel respectively;

[0208] A feature extraction module 20, configured to determine the brightness distribution and the chromaticity distribution of the brightness information and the chromaticity information through histograms, respectively, and obtain brightness features and chromaticity features according to the brightness distribution and the chromaticity distribution;

[0209] An image partitioning module 30 is used to perform local variance calculation on the brightness feature and the chromaticity feature respectively to determine image noise distribution, and partition the low-illuminance image according to the image noise distribution to obtain a plurality of image partitions, wherein the image noise distribution in different image partitions is in different noise intervals;

[0210] The image denoising module 40 is used to respectively determine the noise extremes of the image partitions, determine the AI ​​ISP denoising strategy based on the noise extremes, and denoise the image partitions according to the AI ​​ISP denoising strategy to obtain a denoised image.

[0211] In a feasible implementation manner, the channel separation module 10 is further used to convert the color space of the low-light image into a YUV color space, and determine a brightness channel and a chromaticity channel in the low-light image, wherein the brightness channel is the Y channel in the YUV color space, and the chromaticity channel is the U channel and the V channel in the YUV color space;

[0212] Extracting a brightness component of the brightness channel, performing a histogram analysis on the brightness component to obtain a brightness distribution, and extracting brightness information according to the brightness distribution;

[0213] The chromaticity component of the chromaticity channel is extracted, a color histogram analysis is performed on the chromaticity component to obtain a color distribution in the low-light image, and chromaticity information is extracted according to the color distribution.

[0214] In a feasible implementation manner, the feature extraction module 20 is further used to determine the brightness distribution and the chromaticity distribution of the brightness information and the chromaticity information through histograms respectively;

[0215] determining an abnormal exposure area in the low illumination image according to the brightness distribution;

[0216] Filtering the abnormal exposure area based on a bilateral filter to obtain texture information, and obtaining brightness features according to the contrast of the texture information;

[0217] The color mean of each pixel in the low-illuminance image is determined according to the chromaticity distribution, and the chromaticity feature is determined according to the color mean.

[0218] In a feasible implementation manner, the feature extraction module 20 is further used to perform threshold segmentation on the abnormal exposure area to determine the pixel interval of the abnormal exposure area;

[0219] Expanding the pixel interval to an adjacent area to obtain a neighborhood interval;

[0220] Determine a brightness value of each pixel in the pixel interval, and aggregate the brightness values ​​into a brightness set;

[0221] The brightness set, the neighborhood interval, the normalization factor of the bilateral filter and the Gaussian kernel are combined to obtain texture information;

[0222] Calculating a gradient value of the texture information, and obtaining a texture direction and a texture intensity based on the gradient value;

[0223] Obtaining a structure tensor according to the texture direction and the texture strength;

[0224] A texture energy map is generated based on the structure tensor, and the texture energy map is analyzed to obtain a brightness feature.

[0225] In a feasible implementation manner, the feature extraction module 20 is further used to determine the neighborhood local color mean of each pixel point according to the chromaticity distribution, and determine the color vector field according to the neighborhood local color mean;

[0226] Obtaining a color difference according to the chromaticity component of the pixel point and the average value of the local color of the neighborhood;

[0227] A local color change amount is determined based on the color difference and the color vector field, a gradient change amplitude is calculated according to the local color change amount, and the gradient change amplitude is determined as a chromaticity feature.

[0228] In a feasible implementation manner, the image partitioning module 30 is further used to determine a sliding window size, and set the sliding window to the sliding window size;

[0229] Determining the brightness feature and the chromaticity feature of the pixel point selected by the sliding window;

[0230] Calculating local variances for the brightness feature and the chromaticity feature respectively to obtain a brightness mean and a chromaticity mean;

[0231] Obtaining a noise value of the pixel based on the brightness mean value and the chromaticity mean value;

[0232] Aggregating the noise values ​​of the plurality of pixel points to obtain image noise distribution;

[0233] The low illumination image is partitioned based on the image noise distribution to obtain a plurality of image partitions.

[0234] In a feasible implementation manner, the image partitioning module 30 is further used to determine the noise partitioning threshold, compare the noise value of the pixel point with the noise partitioning threshold, add a first mark to the pixel point whose value is greater than the noise partitioning threshold, and add a second mark to the pixel point whose value is less than the noise partitioning threshold;

[0235] A noise partition schematic diagram is generated based on the image noise distribution, the first mark, and the second mark, and boundaries of each partition are determined to divide the low-illuminance image into a plurality of image partitions.

[0236] In a feasible implementation manner, the image noise reduction module 40 is further used to respectively determine the maximum noise value and the minimum noise value in the image partition, and determine the noise range of the current image partition according to the maximum noise value and the minimum noise value;

[0237] Determine an initial noise reduction parameter according to the partition type of the image partition, and modify the initial noise reduction parameter based on the noise range to obtain a target noise reduction parameter for the current image partition;

[0238] Performing noise reduction processing on each pixel point in the current image partition based on the target noise reduction parameter to obtain a noise-reduced image partition;

[0239] A plurality of the denoised image partitions are aggregated to obtain a denoised image.

[0240] In a feasible implementation manner, the image denoising module 40 is further used to determine a partition type of the image partition, and to obtain an initial denoising parameter of the partition type by searching a parameter table according to the partition type;

[0241] Determine a correction factor according to the noise range, a first difference between the maximum noise value and the average noise value, and a second difference between the minimum noise value and the average noise value;

[0242] The initial noise reduction parameter is modified according to the correction factor to obtain the target noise reduction parameter of the current image partition.

[0243] In a feasible implementation manner, the image noise reduction module 40 is further used to determine the noise value of each pixel in the current partition and generate a noise value set;

[0244] Determine a target noise reduction parameter for the current partition, calculate the noise value set and the target noise reduction parameter, and obtain a target noise value of the pixel after noise reduction;

[0245] The noise value of each pixel point of the current image partition is updated to the target noise value to obtain a denoised image partition.

[0246] The low-illumination scene image intelligent denoising device based on AI ISP provided in the present application adopts the low-illumination scene image intelligent denoising method based on AI ISP in the above-mentioned embodiment, which can solve the technical problem of being unable to retain the details in the image during the denoising process. Compared with the prior art, the beneficial effects of the low-illumination scene image intelligent denoising device based on AI ISP provided in the present application are the same as the beneficial effects of the low-illumination scene image intelligent denoising method based on AI ISP provided in the above-mentioned embodiment, and the other technical features of the low-illumination scene image intelligent denoising device based on AI ISP are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0247] The present application provides an AI ISP-based low-illumination scene image intelligent noise reduction device, and the AI ​​ISP-based low-illumination scene image intelligent noise reduction device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the low-illumination scene image intelligent noise reduction method based on the AI ​​ISP in the above-mentioned embodiment one.

[0248] Reference below Figure 5 , which shows a schematic diagram of the structure of an AI ISP-based low-light scene image intelligent noise reduction device suitable for implementing the embodiment of the present application. The AI ​​ISP-based low-light scene image intelligent noise reduction device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The low-light scene image intelligent noise reduction device based on AI ISP shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0249] like Figure 5As shown, the low-light scene image intelligent noise reduction device based on AI ISP may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the low-light scene image intelligent noise reduction device based on AI ISP are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the low-light scene image intelligent noise reduction device based on the AI ​​ISP to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a low-light scene image intelligent noise reduction device based on the AI ​​ISP with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or provided alternatively.

[0250] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0251] The low-illumination scene image intelligent noise reduction device based on AI ISP provided in the present application adopts the low-illumination scene image intelligent noise reduction method based on AI ISP in the above-mentioned embodiment, which can solve the technical problem of low-illumination scene image intelligent noise reduction based on AI ISP. Compared with the prior art, the beneficial effects of the low-illumination scene image intelligent noise reduction device based on AI ISP provided in the present application are the same as the beneficial effects of the low-illumination scene image intelligent noise reduction method based on AI ISP provided in the above-mentioned embodiment, and the other technical features of the low-illumination scene image intelligent noise reduction device based on AI ISP are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0252] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0253] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0254] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, and the computer-readable program instructions are used to execute the low-illumination scene image intelligent noise reduction method based on AI ISP in the above-mentioned embodiment.

[0255] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0256] The above-mentioned computer-readable storage medium may be included in the low-illumination scene image intelligent noise reduction device based on AI ISP; or it may exist independently without being assembled into the low-illumination scene image intelligent noise reduction device based on AI ISP.

[0257] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the low-illuminance scene image intelligent noise reduction device based on the AI ​​ISP, the low-illuminance scene image intelligent noise reduction device based on the AI ​​ISP: separates the brightness channel and the chromaticity channel of the low-illuminance image, and obtains brightness information and chromaticity information based on the brightness channel and the chromaticity channel respectively; determines the brightness distribution and the chromaticity distribution of the brightness information and the chromaticity information through a histogram respectively, and obtains brightness features and chromaticity features according to the brightness distribution and the chromaticity distribution; performs local variance calculation on the brightness features and the chromaticity features respectively to determine the image noise distribution, and partitions the low-illuminance image according to the image noise distribution to obtain multiple image partitions, and the image noise distribution in different image partitions is different The noise interval; determines the noise extremes of the image partitions respectively, determines the AI ​​ISP noise reduction strategy based on the noise extremes, and performs noise reduction on the image partitions according to the AI ​​ISP noise reduction strategy to obtain a denoised image.

[0258] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0259] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0260] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0261] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned low-illumination scene image intelligent denoising method based on AI ISP, and can solve the technical problem of low-illumination scene image intelligent denoising based on AI ISP. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the low-illumination scene image intelligent denoising method based on AI ISP provided in the above-mentioned embodiment, and will not be repeated here.

[0262] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned AI ISP-based low-illumination scene image intelligent noise reduction method.

[0263] The computer program product provided in this application can solve the technical problem of intelligent noise reduction of low-light scene images based on AI ISP. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the intelligent noise reduction method of low-light scene images based on AI ISP provided in the above embodiment, and will not be repeated here.

[0264] The above are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An intelligent noise reduction method for low-light scene images based on AI ISP, characterized in that: The AI ​​ISP-based low-light scene image intelligent noise reduction method includes: Separating a luminance channel and a chrominance channel of the low-light image, and obtaining luminance information and chrominance information based on the luminance channel and the chrominance channel respectively; Determine brightness distribution and chromaticity distribution of the brightness information and the chromaticity information through histograms respectively, and obtain brightness features and chromaticity features according to the brightness distribution and chromaticity distribution; Performing local variance calculation on the brightness feature and the chromaticity feature respectively to determine image noise distribution, partitioning the low-illuminance image according to the image noise distribution to obtain a plurality of image partitions, wherein the image noise distribution in different image partitions is in different noise intervals; Determine the noise extremes of the image partitions respectively, determine an AI ISP noise reduction strategy based on the noise extremes, and perform noise reduction on the image partitions according to the AI ​​ISP noise reduction strategy to obtain a noise-reduced image; The step of determining the brightness distribution and the chromaticity distribution of the brightness information and the chromaticity information through histograms respectively, and obtaining the brightness feature and the chromaticity feature according to the brightness distribution and the chromaticity distribution comprises: Determining brightness distribution and chromaticity distribution of the brightness information and the chromaticity information through histograms respectively; determining an abnormal exposure area in the low illumination image according to the brightness distribution; Filtering the abnormal exposure area based on a bilateral filter to obtain texture information, and obtaining brightness features according to the contrast of the texture information; Determine a color mean of each pixel in the low-light image according to the chromaticity distribution, and determine the chromaticity feature according to the color mean; The step of filtering the abnormal exposure area based on a bilateral filter to obtain texture information, and obtaining brightness features according to the contrast of the texture information comprises: Performing threshold segmentation on the abnormal exposure area to determine a pixel interval of the abnormal exposure area; Expanding the pixel interval to an adjacent area to obtain a neighborhood interval; Determine a brightness value of each pixel in the pixel interval, and aggregate the brightness values ​​into a brightness set; The brightness set, the neighborhood interval, the normalization factor of the bilateral filter and the Gaussian kernel are combined to obtain texture information; Calculating a gradient value of the texture information, and obtaining a texture direction and a texture intensity based on the gradient value; Obtaining a structure tensor according to the texture direction and the texture strength; A texture energy map is generated based on the structure tensor, and the texture energy map is analyzed to obtain a brightness feature.

2. The method according to claim 1, characterized in that The step of separating the brightness channel and the chromaticity channel of the low-light image and obtaining brightness information and chromaticity information based on the brightness channel and the chromaticity channel respectively comprises: The color space of the low-light image is converted into a YUV color space, and a brightness channel and a chromaticity channel in the low-light image are determined, wherein the brightness channel is the Y channel in the YUV color space, and the chromaticity channel is the U channel and the V channel in the YUV color space.

3. The method according to claim 1, characterized in that The step of determining the color mean of each pixel in the low-light image according to the chromaticity distribution, and determining the chromaticity feature according to the color mean comprises: Determine a neighborhood local color mean of each pixel according to the chromaticity distribution, and determine a color vector field according to the neighborhood local color mean; Obtaining a color difference according to the chromaticity component of the pixel point and the average value of the local color of the neighborhood; A local color change amount is determined based on the color difference and the color vector field, a gradient change amplitude is calculated according to the local color change amount, and the gradient change amplitude is determined as a chromaticity feature.

4. The method according to claim 1, characterized in that The steps of respectively performing local variance calculation on the brightness feature and the chromaticity feature to determine image noise distribution, and partitioning the low-illuminance image according to the image noise distribution to obtain a plurality of image partitions include: Determine the size of the sliding window, and set the sliding window to the size of the sliding window; Determining the brightness feature and the chromaticity feature of the pixel point selected by the sliding window; Calculating local variances for the brightness feature and the chrominance feature respectively, and obtaining a noise value of the pixel based on the local variances; Aggregating the noise values ​​of the plurality of pixel points to obtain image noise distribution; The low illumination image is partitioned based on the image noise distribution to obtain a plurality of image partitions.

5. The method according to claim 4, characterized in that The step of partitioning the low-illuminance image based on the image noise distribution to obtain a plurality of image partitions comprises: Determine a noise partition threshold, compare the noise value of the pixel with the noise partition threshold, add a first mark to the pixel whose value is greater than the noise partition threshold, and add a second mark to the pixel whose value is less than the noise partition threshold; A noise partition schematic diagram is generated based on the image noise distribution, the first mark, and the second mark, and boundaries of each partition are determined to divide the low-illuminance image into a plurality of image partitions.

6. The method according to claim 1, characterized in that The steps of respectively determining the noise extremes of the image partitions, determining the AI ​​ISP noise reduction strategy based on the noise extremes, and performing noise reduction on the image partitions according to the AI ​​ISP noise reduction strategy to obtain a noise-reduced image include: Determine the maximum noise value and the minimum noise value in the image partition respectively, and determine the noise range of the current image partition according to the maximum noise value and the minimum noise value; Determine an initial noise reduction parameter according to the partition type of the image partition, and modify the initial noise reduction parameter based on the noise range to obtain a target noise reduction parameter for the current image partition; Performing noise reduction processing on each pixel point in the current image partition based on the target noise reduction parameter to obtain a noise-reduced image partition; A plurality of the denoised image partitions are aggregated to obtain a denoised image.

7. The method according to claim 6, characterized in that The step of determining the initial noise reduction parameter according to the partition type of the image partition, and correcting the initial noise reduction parameter based on the noise range to obtain the target noise reduction parameter of the current image partition comprises: Determine a partition type of the image partition, and search a parameter table according to the partition type to obtain an initial noise reduction parameter of the partition type; Determine a correction factor according to a first difference between the maximum noise value and the average noise value, a second difference between the minimum noise value and the average noise value, and the noise range; The initial noise reduction parameter is modified according to the correction factor to obtain the target noise reduction parameter of the current image partition.

8. The method according to claim 6, characterized in that The step of performing noise reduction processing on each pixel point in the current image partition based on the target noise reduction parameter to obtain a noise-reduced image partition comprises: Determine the noise value of each pixel in the current partition and generate a noise value set; Determine a target noise reduction parameter for the current partition, calculate the noise value set and the target noise reduction parameter, and obtain a target noise value of the pixel after noise reduction; The noise value of each pixel point of the current image partition is updated to the target noise value to obtain a denoised image partition.

9. An intelligent noise reduction device for low-light scene images based on AI ISP, characterized in that: The low-illumination scene image intelligent noise reduction device based on AI ISP includes: A channel separation module, used to separate the brightness channel and the chromaticity channel of the low-light image, and obtain brightness information and chromaticity information based on the brightness channel and the chromaticity channel respectively; A feature extraction module, used to determine the brightness distribution and the chromaticity distribution of the brightness information and the chromaticity information through histograms, and obtain brightness features and chromaticity features according to the brightness distribution and the chromaticity distribution; An image partitioning module, used to perform local variance calculation on the brightness feature and the chromaticity feature respectively, determine image noise distribution, and partition the low-illuminance image according to the image noise distribution to obtain a plurality of image partitions, wherein the image noise distribution in different image partitions is a different noise interval; An image denoising module, used to respectively determine the noise extremes of the image partitions, determine an AIISP denoising strategy based on the noise extremes, and denoise the image partitions according to the AI ​​ISP denoising strategy to obtain a denoised image; The step of determining the brightness distribution and the chromaticity distribution of the brightness information and the chromaticity information through histograms respectively, and obtaining the brightness feature and the chromaticity feature according to the brightness distribution and the chromaticity distribution comprises: Determining brightness distribution and chromaticity distribution of the brightness information and the chromaticity information through histograms respectively; determining an abnormal exposure area in the low illumination image according to the brightness distribution; Filtering the abnormal exposure area based on a bilateral filter to obtain texture information, and obtaining brightness features according to the contrast of the texture information; Determine a color mean of each pixel in the low-light image according to the chromaticity distribution, and determine the chromaticity feature according to the color mean; The step of filtering the abnormal exposure area based on a bilateral filter to obtain texture information, and obtaining brightness features according to the contrast of the texture information comprises: Performing threshold segmentation on the abnormal exposure area to determine a pixel interval of the abnormal exposure area; Expanding the pixel interval to an adjacent area to obtain a neighborhood interval; Determine a brightness value of each pixel in the pixel interval, and aggregate the brightness values ​​into a brightness set; The brightness set, the neighborhood interval, the normalization factor of the bilateral filter and the Gaussian kernel are combined to obtain texture information; Calculating a gradient value of the texture information, and obtaining a texture direction and a texture intensity based on the gradient value; Obtaining a structure tensor according to the texture direction and the texture strength; A texture energy map is generated based on the structure tensor, and the texture energy map is analyzed to obtain a brightness feature.

10. An AI ISP-based low-light scene image intelligent noise reduction device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the AI ​​ISP-based low-illumination scene image intelligent noise reduction method as described in any one of claims 1 to 8.

11. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the low-illumination scene image intelligent noise reduction method based on AI ISP are implemented as described in any one of claims 1 to 8.

Images