Self-adaptive image enhancement method and device in low-illumination scene

Abstract

The invention provides a self-adaptive image enhancement method and device in a low-illumination scene. The method comprises the following steps: performing low-illumination scene judgment on an RGB image to obtain a scene category, and performing self-adaptive partition processing on the RGB image according to the scene category to obtain a plurality of area images; extracting illumination features, noise features and detail features of each regional image, and generating a feature matrix according to the features; dynamically matching an image enhancement algorithm or an algorithm combination for each area image from an enhancement strategy library through a preset matching strategy according to the scene category and the feature matrix; performing image enhancement processing on the corresponding area image by adopting an image enhancement algorithm or algorithm combination matched with each area image; and sequentially carrying out cross-partition fusion, color consistency calibration and naturalness optimization processing on each region image after enhancement processing, and outputting an RGB enhanced image. According to the method and the device, the brightness, the detail definition and the color consistency of the RGB image in the low-light environment are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to an adaptive image enhancement method and device in a low-light scene. BACKGROUND

[0002] With the wide application of artificial intelligence technology in the field of image processing, image enhancement in a low-light environment has become a key to improving visual perception. Especially in the scenes of security monitoring, intelligent transportation, etc., due to the limitations of night, overcast or indoor weak light, etc., the collected RGB images often have problems such as low overall brightness, significant noise, and serious loss of details.

[0003] The existing technology mostly adopts a globally uniform enhancement method, such as fixed-parameter gamma correction or histogram equalization, which lacks adaptability to local features of images and is difficult to balance the relationship between brightness enhancement and noise suppression, detail preservation. In addition, the traditional method does not consider the differentiated needs of different low-light intensities. For example, in the extreme low-light scene, the contradiction between noise and details is more prominent, while in the conventional low-light scene, color distortion caused by excessive enhancement needs to be avoided.

[0004] Therefore, there is an urgent need in the prior art for an adaptive image enhancement method and device in a low-light scene that can adaptively partition according to scene characteristics, dynamically match enhancement strategies, and consider naturalness optimization, so as to comprehensively improve the visual quality and usability of images in a low-light environment. SUMMARY

[0005] Therefore, there is an urgent need in the prior art for an adaptive image enhancement method and device in a low-light scene that can adaptively partition according to scene characteristics, dynamically match enhancement strategies, and consider naturalness optimization, so as to comprehensively improve the visual quality and usability of images in a low-light environment.

[0006] In a first aspect, the present application provides an adaptive image enhancement method in a low-light scene, comprising:

[0007] determining a low-light scene of the RGB image to obtain a corresponding scene category, wherein the scene category includes a conventional low-light scene and an extreme low-light scene;

[0008] According to the scene category, the RGB image is adaptively partitioned to obtain a plurality of region images, and the plurality of region images include a center region, a near-edge transition region and a far-edge low-light region;

[0009] For each of the plurality of region images, light feature, noise feature and detail feature are extracted, and a feature matrix corresponding to each of the region images is generated according to the extracted light feature, noise feature and detail feature;

[0010] According to the scene category and the feature matrix, a corresponding image enhancement algorithm or algorithm combination is dynamically matched for each region image from a preset enhancement strategy library through a preset matching strategy;

[0011] The image enhancement algorithm or algorithm combination matched for each region image is used to perform image enhancement processing on the corresponding region image, so that each region image after enhancement processing is obtained;

[0012] The region images after enhancement processing are sequentially subjected to cross-partition fusion and color consistency calibration, so that the calibrated RGB image is obtained, and the calibrated RGB image is subjected to natural degree optimization processing, and the RGB enhanced image is output.

[0013] In a second aspect, the present application provides an adaptive image enhancement device in a low-light scene, comprising:

[0014] A scene determination module is configured to determine a low-light scene of the RGB image to obtain a corresponding scene category, wherein the scene category comprises a regular low-light scene and an extreme low-light scene;

[0015] A partition processing module is connected with the scene determination module and is configured to perform adaptive partition processing on the RGB image according to the scene category to obtain a plurality of region images, wherein the plurality of region images comprise a center region, a near-edge transition region and a far-edge low-light region;

[0016] A feature extraction module is connected with the partition processing module and is configured to extract a light feature, a noise feature and a detail feature from each region image of the plurality of region images, respectively, and generate a feature matrix corresponding to each region image according to the extracted light feature, noise feature and detail feature;

[0017] A strategy matching module is connected with the feature extraction module and is configured to dynamically match a corresponding image enhancement algorithm or algorithm combination for each region image from a preset enhancement strategy library through a preset matching strategy according to the scene category and the feature matrix;

[0018] An enhancement processing module is connected with the strategy matching module and is configured to use the image enhancement algorithm or algorithm combination matched for each region image to perform image enhancement processing on the corresponding region image, so that each region image after enhancement processing is obtained;

[0019] A fusion optimization module is connected with the enhancement processing module and is configured to sequentially perform cross-partition fusion and color consistency calibration on the region images after enhancement processing to obtain a calibrated RGB image, and perform natural degree optimization processing on the calibrated RGB image to output an RGB enhanced image.

[0020] The application provides a self-adaptive image enhancement method and device in a low-light scene, which realizes accurate identification and full-link adaptation of different light intensities by constructing a "regular-extreme" low-light scene layered judgment mechanism, effectively solving the core contradiction between noise suppression and detail preservation in an extreme low-light scene. An adaptive partitioning strategy of "spatial position and light gradient" is adopted, and a "light-noise-detail" three-dimensional feature analysis enabled by deep learning is combined to dynamically match the optimal enhancement algorithm for each partition. In addition, it significantly eliminates the stitching marks and color distortion through multi-scale cross-partition fusion and naturalness optimization technology, and improves the visual naturalness of the image. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1a A flowchart of a self-adaptive image enhancement method in a low-light scene provided by an embodiment of the application is shown.

[0022] Figure 1b Another flowchart of a self-adaptive image enhancement method in a low-light scene provided by an embodiment of the application is shown.

[0023] Figure 1c Still another flowchart of a self-adaptive image enhancement method in a low-light scene provided by an embodiment of the application is shown.

[0024] Figure 2 A circuit module structure diagram of a self-adaptive image enhancement device in a low-light scene provided by an embodiment of the application is shown.

[0025] Figure 3 An internal structure diagram of a computer device in an embodiment of the application is shown. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the application clearer, further detailed description of the application is made below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application.

[0027] As shown in Figure 1a , the application provides a self-adaptive image enhancement method in a low-light scene, which comprises:

[0028] Step S11: performing low-light scene judgment on an RGB image to obtain a corresponding scene category, wherein the scene category includes a regular low-light scene and an extreme low-light scene;

[0029] As shown in Figure 1b , step S11 can specifically include:

[0030] Step S111: Calculate the global brightness mean value, dynamic range value, relative color temperature index based on RGB distribution, and local dark area proportion in the RGB image, respectively.

[0031] In the present application, the calculation method of the global brightness mean value, dynamic range value, relative color temperature index based on RGB distribution, and local dark area proportion in the RGB image can be flexibly selected by those skilled in the art according to actual needs, which is not limited here.

[0032] Taking OpenCV library as an example, first, the RGB image is read by the cv2.imread() function, which parses the RGB image into a three-dimensional NumPy array in BGR format, where each element represents the intensity value of a specific pixel in the RGB image in the blue, green, and red channels, with a value range of 0-255.

[0033] After obtaining the three-dimensional NumPy array, the cv2.cvtColor() function is used with the cv2.COLOR_BGR2YUV parameter to convert the RGB image from BGR color space to YUV color space. This conversion process is based on the ITU-R BT.601 international standard, and its core is to use the luminance calculation formula Y=0.299×R+0.587×G+0.114×B, which takes into account the physiological characteristics of human eye sensitivity to different colors. After conversion, the Y channel value is directly extracted as the brightness value of each pixel.

[0034] Finally, the np.mean() function of the NumPy library is used to average the brightness values of all pixels, and the accurate global brightness mean value is obtained, which provides a key basis for low-light scene judgment.

[0035] Continuing the above example, after obtaining the brightness value of each pixel in the RGB image, the np.max() and np.min() functions of the NumPy library are used to calculate the maximum brightness value L max and the minimum brightness value L min in the brightness channel, respectively. The difference between the two is the dynamic range value D=L max -L min , which reflects the brightness difference between the brightest and darkest pixels in the image, and is an important indicator for evaluating image quality in low-light scenes.

[0036] The relative color temperature index calculation based on RGB distribution requires extracting color information from the RGB image and performing color temperature estimation. Continuing with the OpenCV library as an example, first, the RGB image is read by the cv2.imread() function to obtain a three-dimensional NumPy array in BGR format. Subsequently, the np.mean() function of the NumPy library is used to calculate the average intensity values of the red (R), green (G), and blue (B) channels, respectively, to obtain the average intensity value of the red channel R avg , the average intensity value of the green channel G avg , and the average intensity value of the blue channel B avg . Based on these average intensity values, the sum of the RGB average intensity values sum RGB =R avg +G avg +B avg is calculated, and further the normalized red ratio r=R avg / sum RGB and the blue ratio b=B avg / sum RGB are obtained. Finally, the empirical formula method based on RGB ratio is used for color temperature estimation. When r>b, the relative color temperature index based on RGB distribution is calculated using the formula T=1000×(r-0.33) / (0.33-b), where 0.33 is the theoretical approximation of the normalized RGB value under ideal white light (1 / 3); when this condition is not met, the default color temperature value (such as 5000K) is returned. This method is based on the statistical relationship between color temperature and RGB channel ratio, has high calculation efficiency, and can effectively distinguish between warm and cold light environments, providing important color feature reference for low-light scene classification.

[0037] It should be noted that the essence of color temperature is to describe the color characteristics of the light source, and its change mainly reflects in the relative proportion of red and blue components in the spectrum. Low color temperature (warm light): such as incandescent lamp, candlelight, red component much more than blue component (r>b); high color temperature (cold light): such as the sky on a sunny day, fluorescent lamp, blue component more than red component (b>r); and the green component is relatively stable in most natural and artificial light sources, more affecting the brightness perception of lighting rather than the color temperature perception. Therefore, when estimating the color temperature, the red ratio r and the blue ratio b are mainly considered.

[0038] Continuing the above example, after obtaining the brightness value of each pixel in the RGB image, a dark area brightness threshold (e.g., the dark area brightness threshold is 15) is preset, and the number of pixels with a brightness value less than the dark area brightness threshold in the brightness channel, i.e., the number of dark area pixels, is counted by using the np.sum() function of the NumPy library. At the same time, the total number of pixels of the RGB image is obtained by using the size attribute of the brightness channel. Finally, the proportion of the local dark area in the RGB image is the ratio of the number of dark area pixels to the total number of pixels, i.e., the proportion of the local dark area in the RGB image = the number of dark area pixels / the total number of pixels.

[0039] Step S112: If the global brightness average is less than the first brightness threshold, it is determined that the scene category is an extremely low-light scene.

[0040] Wherein, for the value of the first brightness threshold, a person skilled in the art can flexibly set it according to actual needs, for example: the brightness range of an 8-bit RGB image is 0-255, and the first brightness threshold can be 10. Specifically, if the global brightness average L avg is less than 10 (i.e., L avg < 10), it is determined that the scene category is an extremely low-light scene.

[0041] Step S113: If the global brightness average is greater than or equal to the first brightness threshold, the following judgment is performed:

[0042] If any of the following conditions is met, it is determined that the scene category is a regular low-light scene: the global brightness average is less than the second brightness threshold and the dynamic range value is less than the dynamic range threshold; or the global brightness average is less than the third brightness threshold and the relative color temperature index based on the RGB distribution is less than the color temperature threshold; or the proportion of the local dark area in the RGB image is greater than the dark area proportion threshold.

[0043] Wherein, for the value of the second brightness threshold, the value of the dynamic range threshold, the value of the third brightness threshold, the value of the color temperature threshold, and the value of the dark area proportion threshold, a person skilled in the art can flexibly set them according to actual needs, which are not limited here. For example: the brightness range of an 8-bit RGB image is 0-255, the second brightness threshold can be 30, the dynamic range threshold can be 40, the third brightness threshold can be 25, the color temperature threshold can be 5000K, and the dark area proportion threshold can be 40%. Specifically, if the global brightness average L avg is less than 30 and the dynamic range value D is less than 40 (i.e., L avg < 30 and D < 40); or the global brightness average L avg is less than 25 and the relative color temperature index T based on the RGB distribution is less than 5000K (i.e., L avgP1<40% and T>5000K) or P1>40% (i.e. P1>40%), the scene category is determined as a regular low-light scene.

[0044] Step S114: If none of the above conditions is met, the RGB image is subjected to a slight contrast optimization.

[0045] The slight contrast stretching algorithm used is a slight contrast stretching algorithm in the prior art, with a contrast gain of 1.1 to 1.2.

[0046] Step S12: According to the scene category, the RGB image is subjected to adaptive partitioning to obtain a plurality of region images, including a center region, a near-edge transition region and a far-edge low-light region.

[0047] Optionally, as shown in FIG. 12, step S12 specifically includes: Figure 1c

[0048] Step S121: Based on a spatial position partitioning method, the RGB image is partitioned into a center region and an edge region.

[0049] The spatial position partitioning method includes: obtaining a center point of the RGB image and a diagonal length of the RGB image; and partitioning a circular center region from the RGB image with the center point of the RGB image as the center and the one-third length of the diagonal length of the RGB image as the radius, and taking the remaining region after the partitioning of the RGB image as the edge region. Of course, other partitioning methods can also be selected by those skilled in the art, which are not limited herein.

[0050] Step S122: Obtain a lens distortion parameter of the camera, and perform distortion pre-correction on the edge region using the lens distortion parameter.

[0051] Lens distortion is an inherent optical property of the camera lens, which causes straight lines in the image to appear curved in the edge region. In a panoramic camera, this distortion is particularly evident at the image edges. The lens distortion parameter of the camera can be obtained through the SDK (English full name: Software Development Kit, development tool set) provided by the camera manufacturer, including radial distortion coefficients such as first-order radial distortion coefficient K1 and second-order radial distortion coefficient K2.

[0052] Distortion pre-correction is to compensate for this distortion in reverse through a mathematical model, so that the image is closer to the real scene. In an optional embodiment of the present application, if a Brown-Conrady distortion model is used for distortion pre-correction, the distortion formula is: x corrected =x×(1+k1×r²+k2×r 4 ), y corrected ​= y x (1 + k1 x r2+ k2 x r 4 ), wherein (x, y) is the normalized original coordinate of a pixel in the edge region, r is the distance of the pixel to the center point of the RGB image, and k1 and k2 are lens distortion parameters.

[0053] Suppose that the center point of the RGB image is (0, 0), the normalized original coordinate of a pixel in the edge region is (0.8, 0), and the lens distortion parameters are k1 = -0.0003 and k2 = 0.00001. 、 The normalized distance of the pixel to the center point of the RGB image is .

[0054] The coordinate of the pixel after the distortion pre-correction is x corrected = x x (1 + k1 x r2+ k2 x r 4 ) = 0.8 x [1 + (-0.0003) x 0.8 2+ 0.00001 x 0.8 4 ] = 0.799849677, and y corrected = y x (1 + k1 x r2+ k2 x r 4 ) = 0 x [1 + (-0.0003) x 0.8 2+ 0.00001 x 0.8 4 ] = 0, i.e., the normalized original coordinate of the pixel after the distortion pre-correction is (0.799849677, 0). Other pixels can be processed in the same way, which is not described herein again. In addition, if the original coordinates in the RGB image are used, the normalized original coordinates after the distortion pre-correction can also be converted by the inverse normalization method in the prior art, which is not described herein again.

[0055] Step S123: calculating the brightness difference between the brightness value of each pixel in the corrected edge region and the brightness average of the center region, and based on the brightness difference, subdividing the edge region into a near-edge transition region and a far-edge low-light region;

[0056] In an optional embodiment of the present application, step S123 specifically comprises: obtaining the brightness value of each pixel in the corrected edge region, and calculating the brightness average of the center region; calculating the brightness difference between the brightness value of each pixel in the corrected edge region and the brightness average of the center region; comparing the brightness difference with a preset fourth brightness threshold value; if the brightness difference is less than or equal to the fourth brightness threshold value, dividing the corresponding pixel in the edge region into the near-edge transition region; if the brightness difference is greater than the fourth brightness threshold value, dividing the corresponding pixel in the edge region into the far-edge low-light region.

[0057] The value of the fourth brightness threshold value can be selected flexibly by those skilled in the art according to actual needs, which is not limited herein. For example, the fourth brightness threshold value is preferably 50.

[0058] Taking the OpenCV library as an example, firstly, the RGB image is read using the `cv2.imread()` function. This function returns a three-dimensional NumPy array (matrix) with dimensions represented as [height, width, number of channels], where the channel order is BGR. Next, the `cv2.cvtColor()` function, along with the `cv2.COLOR_BGR2YUV` parameter, converts the RGB image from the BGR color space to the YUV color space.

[0059] After conversion, extracting the first channel (Y channel) from the 3D YUV matrix yields a 2D luminance matrix, where each element represents the luminance value of the corresponding pixel. Based on this 2D luminance matrix, the average luminance value L of all pixels in the central region is calculated using the np.mean() function. center , such as L center =125.4.

[0060] Subsequently, the brightness value L of each pixel corresponding to the edge region in the two-dimensional brightness matrix is ​​traversed. pixel Calculate the brightness value L for each pixel. pixel The average brightness L of the central region center The brightness difference (i.e., the absolute difference) between them is ΔL = |L pixel -125.4|. If the preset fourth brightness threshold is 50, compare the brightness difference ΔL of each pixel with the fourth brightness threshold and complete the division: for example, if a pixel L pixel1 =98.3, ΔL=|98.3-125.4|=27.1≤50, this pixel is classified as a near-edge transition region; another pixel L pixel2 =63.8, ΔL=|63.8-125.4|=61.6>50, therefore this pixel is classified as a low-light region at the far edge. This process allows for precise subdivision of edge regions.

[0061] Step S124: When the scene category is an extreme low light scene, perform noise pre-detection on the central area, the near edge transition area and the far edge low light area respectively, and mark the high noise area.

[0062] In an optional embodiment of the present invention, step S124 specifically includes: using a pixel window of a preset size (such as a 3×3 pixel window) to slide in the central region, the near-edge transition region, and the far-edge low-light region respectively; performing filtering processing on the pixels in the pixel window at each position, and calculating the variance of the pixel grayscale difference before and after filtering as the noise intensity. The calculation formula is: ,in, The number of pixels within the pixel window. Let be the grayscale value of the j-th pixel after filtering. is the gray value of the jth pixel before filtering; the noise intensity is compared with a preset first noise intensity threshold T noise1 ; if the noise intensity is greater than the first noise intensity threshold T noise1 , the region corresponding to the pixel window is marked as a high-noise region.

[0063] The size of the pixel window and the first noise intensity threshold can be flexibly set by those skilled in the art according to actual needs, and are not limited herein. For example, the size of the pixel window is preferably a 3x3 pixel window, and the first noise intensity threshold is preferably 4.

[0064] Suppose that noise pre-detection is performed on a far-edge low-light region, a 3x3 pixel window and a mean filtering method are adopted, and the preset first noise intensity threshold T noise =4.

[0065] A 3x3 window is taken from the far-edge low-light region, and the pixel gray value matrix before filtering is: After 3x3 mean filtering, the pixel gray value of the window becomes: .

[0066] The variance of the pixel gray difference before and after filtering is calculated as the noise intensity ≈7.35.

[0067] The noise intensity ≈7.35 is compared with the first noise intensity threshold T noise =4: since the noise intensity is greater than the first noise intensity threshold T noise (i.e., 7.35>4), the region corresponding to the 3x3 pixel window is marked as a high-noise region.

[0068] Step S13: For each of the plurality of region images, light features, noise features and detail features are extracted respectively, and a feature matrix corresponding to each of the plurality of region images is generated according to the extracted light features, noise features and detail features.

[0069] Optionally, in step S13, for each of the plurality of region images, the light features are extracted, including:

[0070] Step S1311: Based on the scene category, a dark part brightness threshold and a brightness standard deviation threshold are selected respectively.

[0071] Step S1312: Based on the selected dark part brightness threshold, dark part pixels in each of the plurality of region images are screened out, to obtain a number of dark part pixels in each of the plurality of region images.

[0072] Step S1313: Calculate the ratio of the number of dark pixels in each of the plurality of region images to the total number of pixels in the corresponding region image, respectively, to obtain the dark pixel proportion of the corresponding region image, and quantize the dark pixel proportion to a specific type according to the dark pixel proportion and a preset dark portion proportion threshold value;

[0073] Step S1314: Calculate the brightness standard deviation of each of the plurality of region images, respectively, and compare each brightness standard deviation with a selected brightness standard deviation threshold value, and mark the illumination uniformity state of the corresponding region image according to the comparison result;

[0074] Step S1315: Take the specific type corresponding to the dark pixel proportion and the illumination uniformity state of the plurality of region images as the illumination feature.

[0075] The values of the dark portion brightness threshold value and the brightness standard deviation threshold value can be flexibly set by those skilled in the art according to actual needs, which are not limited here. Based on different scene categories, the value of the dark portion brightness threshold value is different. For a conventional low-light scene, the dark portion brightness threshold value selected based on this scene category can be 15, and the brightness standard deviation threshold value can be 8; for an extreme low-light scene, the dark portion brightness threshold value selected based on this scene category can be 8, and the brightness standard deviation threshold value can be 5.

[0076] In the present application, the preset dark portion proportion threshold value includes a first dark portion proportion threshold value (such as 80%), a second dark portion proportion threshold value (such as 60%), and a third dark portion proportion threshold value (such as 30%), from which a plurality of interval ranges can be obtained; the dark pixel proportion is quantized to a specific type according to the dark pixel proportion and the plurality of interval ranges. Among them, the interval range corresponding to the extremely high dark portion proportion is greater than the first dark portion proportion threshold value; the interval range corresponding to the high dark portion proportion is less than or equal to the first dark portion proportion threshold value and greater than the second dark portion proportion threshold value; the interval range corresponding to the medium dark portion proportion is less than or equal to the second dark portion proportion threshold value and greater than the third dark portion proportion threshold value; the interval range corresponding to the low dark portion proportion is less than the third dark portion proportion threshold value.

[0077] Suppose the current scene category is a conventional low-light scene, the dark portion brightness threshold value selected based on this scene category is 15, and the brightness standard deviation threshold value is 8. Taking the far edge low-light region as an example, the total number of pixels in this region is 10000.

[0078] First, based on the selected dark part brightness threshold 15, the dark part pixels in the region are screened, and the number of dark part pixels is counted to be 7500, and the dark part pixel ratio is calculated to be 7500 / 10000=0.75 (i.e. 75%). Since 60% (the second dark part ratio threshold) < 75% (the dark part pixel ratio) < 80% (the first dark part ratio threshold), the specific type corresponding to the dark part pixel ratio is high dark part ratio.

[0079] Next, the brightness standard deviation of the region is calculated according to the formula , wherein is the brightness standard deviation of the region, is the total number of pixels in the region, is the brightness value of the i-th pixel in the region, is the average value of the brightness of all pixels in the region. Assuming that the brightness standard deviation of the region is 12. Compare the calculated brightness standard deviation with the selected brightness standard deviation threshold, since 12>8, mark the light uniformity state of the region as "light non-uniform".

[0080] Finally, the light features of the far edge low light region are composed of "high dark part ratio" and "light non-uniformity".

[0081] Similarly, the light features of the center region and the near edge transition region can also be obtained according to the above calculation process, which will not be described here.

[0082] Optionally, in step S13, for each of the plurality of region images, noise features are extracted, including:

[0083] Step S1321: based on the scene category, a corresponding second noise intensity threshold is selected;

[0084] Step S1322: filter through a preset window, and calculate the variance of the pixel gray scale difference before and after filtering of each of the plurality of region images;

[0085] Step S1323: compare the calculated variance with the selected second noise intensity threshold, and determine a noise intensity preliminary value according to the comparison result;

[0086] Step S1324: input each of the plurality of region images into a lightweight noise classification model to obtain corresponding noise type information, and quantize the noise type information into a specific noise type;

[0087] Step S1325: the noise intensity preliminary value and the specific noise type of the plurality of region images are taken as noise features.

[0088] Wherein, the second noise intensity threshold value can be set flexibly by the person skilled in the art according to actual needs, which is not limited here. Based on the difference of scene categories, the second noise intensity threshold value will be different. For example: if the scene category is a regular low-light scene, the second noise intensity threshold value selected based on this scene category is 3; if the scene category is an extreme low-light scene, the second noise intensity threshold value selected based on this scene category is 2.

[0089] Wherein, the types of noise intensity preliminary values can include: high noise, low noise; the types of specific noise types can include: Gaussian noise, mixed noise. The person skilled in the art can choose flexibly according to actual needs, which is not limited here.

[0090] Assuming that the current scene category is a regular low-light scene, the second noise intensity threshold value selected based on this scene category is 3.

[0091] Taking the far-edge low-light area as an example, first, mean filtering is performed through a 3x3 preset window, and the variance of the pixel gray scale difference before and after filtering of the area is calculated. Assuming that the calculated variance is 4.5.

[0092] Compare the calculated variance value with the second noise intensity threshold value, since 4.5>3, the noise intensity preliminary value of the area is determined as "high noise".

[0093] Input the image block of the far-edge low-light area into the lightweight noise classification model based on MobileNetV3 architecture, and the model outputs the noise type information of the area as "Gaussian noise is the main one", and quantizes the noise type information as the specific noise type of Gaussian noise.

[0094] Finally, the noise features of the far-edge low-light area are composed of "high noise" and "Gaussian noise is the main one".

[0095] Similarly, the noise features of the center area and the near-edge transition area can also be obtained according to the above calculation process, which is not described here.

[0096] Optionally, in step S13, for each of the plurality of region images, a detail feature is extracted, including:

[0097] Step S1331: based on the scene category, respectively selecting a gradient threshold value, a detail proportion threshold value and a Laplacian variance threshold value;

[0098] Step S1332: calculating the edge gradient value of each pixel in each of the plurality of region images by an edge detection operator;

[0099] Step S1333: Calculate the pixel ratio of each edge gradient value greater than the selected gradient threshold value, and compare each pixel ratio with the detail ratio threshold value, and mark the detail state information according to the comparison result;

[0100] Step S1334: Calculate the Laplacian variance of each of the plurality of region images, and compare each Laplacian variance with the Laplacian variance threshold value, and mark the definition state information according to the comparison result;

[0101] Step S1335: Take the detail state information and the definition state information as the detail feature.

[0102] It should be noted that the gradient threshold value, the detail ratio threshold value and the Laplacian variance threshold value will be different based on different scene categories. For example, if the scene category is a regular low-light scene, the gradient threshold value selected based on this scene category can be 5, the detail ratio threshold value can be 8%, and the Laplacian variance threshold value can be 15; if the scene category is an extreme low-light scene, the gradient threshold value selected based on this scene category can be 3, the detail ratio threshold value can be 5%, and the Laplacian variance threshold value can be 10.

[0103] Suppose the current scene category is a regular low-light scene, the gradient threshold value selected based on this scene category is 5, the detail ratio threshold value is 8%, and the Laplacian variance threshold value is 15. Taking the far edge low-light region as an example, suppose the total number of pixels in this region is 10000, the edge gradient value of each pixel in this region is calculated by the Sobel operator, and the number of pixels with gradient value greater than 5 is 600, and the pixel ratio is 6%. Compare the pixel ratio with the detail ratio threshold value, since 6%<8%, mark the detail state information as "low detail". The Laplacian variance of the region is 12, and the Laplacian variance is compared with the Laplacian variance threshold value, since 12<15, mark the definition state information as "detail blur". Finally, the detail feature of the far edge low-light region is composed of "low detail" and "detail blur".

[0104] Similarly, the detail features of the center region and the near edge transition region can also be obtained according to the above calculation process, which will not be described here.

[0105] It should be noted that steps S1311 to S1316, steps S1321 to S1325, steps S1331 to S1335 are only for convenience of description and are not shown in the figure.

[0106] Step S14: According to the scene category and the feature matrix, a corresponding image enhancement algorithm or algorithm combination is dynamically matched for each regional image from the preset enhancement strategy library through a preset matching strategy;

[0107] Optionally, step S14 specifically includes:

[0108] Step S141: If the scene category is an extreme low-light scene, a first decision tree model is called to match an extreme low-light exclusive algorithm combination from the enhancement strategy library based on the feature matrix;

[0109] Step S142: If the scene category is a regular low-light scene, a second decision tree model is called to select a preset number of candidate algorithms or algorithm combinations from the enhancement strategy library based on the feature matrix;

[0110] The first decision tree model can adopt a CART (Classification and Regression Tree) decision tree model in the prior art; the second decision tree model can adopt a C4.5 decision tree model in the prior art, which is an extension and optimization of the ID3 algorithm. Of course, a person skilled in the art can also select according to actual needs, which is not limited here.

[0111] Step S143: The comprehensive weight score of each candidate algorithm or algorithm combination is calculated based on a weight voting mechanism;

[0112] Step S144: The candidate algorithm or algorithm combination with the highest comprehensive weight score is selected as the image enhancement algorithm or algorithm combination for the corresponding regional image.

[0113] Optionally, the preset matching strategy is based on a preset corresponding relationship between the feature matrix and the enhancement algorithm combination, and the preset corresponding relationship in the enhancement strategy library includes:

[0114] ①When the specific type corresponding to the dark pixel proportion indicated by the feature matrix is high dark proportion, the noise intensity preliminary judgment value is high noise, the specific noise type is Gaussian noise, and the detail state information is low detail, the algorithm combination of the non-local mean denoising algorithm and the multi-scale Retinex image enhancement algorithm is matched; wherein the non-local mean denoising algorithm with a denoising window of 5x5 is adopted, and the multi-scale Retinex image enhancement algorithm with a scale parameter of {5, 15, 30} is adopted.

[0115] ②When the feature matrix indicates that the specific type corresponding to the dark pixel proportion is high dark proportion, the specific noise type is mixed noise, and the detail state information is low detail, the algorithm combination of the adaptive median filtering algorithm and the multi-scale Retinex image enhancement algorithm is matched; wherein, the adaptive median filtering algorithm with a filter window of 3x3 to 7x7 is adopted, and the multi-scale Retinex image enhancement algorithm with a scale parameter of {5, 15, 30} is adopted.

[0116] ③When the feature matrix indicates that the specific type corresponding to the dark pixel proportion is high dark proportion, the noise intensity preliminary judgment value is high noise, and the definition state information is detail blur, the algorithm combination of the dark channel prior denoising algorithm, the non-local mean denoising algorithm, and the multi-scale Retinex image enhancement algorithm is matched.

[0117] ④When the feature matrix indicates that the specific type corresponding to the dark pixel proportion is medium dark proportion, the noise intensity preliminary judgment value is low noise, and the light uniformity state is light unevenness, the partition gamma correction algorithm is matched; wherein, the value range of the gamma coefficient is 0.6-1.4, and the specific value is dynamically allocated according to the brightness information of each block.

[0118] ⑤When the feature matrix indicates that the specific type corresponding to the dark pixel proportion is medium dark proportion, the noise intensity preliminary judgment value is high noise, and the detail state information is high detail, the algorithm combination of the bilateral filter denoising algorithm and the adaptive gamma correction algorithm is matched; wherein, when the algorithm combination of the bilateral filter denoising algorithm and the adaptive gamma correction algorithm is adopted, the parameter configuration is as follows: the spatial domain standard deviation of the bilateral filter is set to 10, the value domain standard deviation is set to 20, and the gamma correction coefficient is dynamically adjusted in the range of 0.8 to 1.2 according to the image characteristics.

[0119] ⑥When the feature matrix indicates that the specific type corresponding to the dark pixel proportion is low dark proportion, the noise intensity preliminary judgment value is low noise, and the detail state information is high detail, the mild contrast stretching algorithm is matched; wherein, the contrast gain of the mild contrast stretching algorithm adopted is set to 1.1 to 1.3.

[0120] ⑦When the feature matrix indicates that the specific type corresponding to the dark pixel proportion is low dark proportion, the noise intensity preliminary judgment value is high noise, and the detail state information is high detail, the combination algorithm of the Gaussian filter denoising algorithm and the mild contrast stretching algorithm is matched; wherein, the standard deviation of the Gaussian function in the Gaussian filter denoising algorithm adopted is 1.5, and the contrast gain of the mild contrast stretching algorithm adopted is set to 1.1 to 1.3.

[0121] ⑧When the feature matrix indicates that the specific type corresponding to the dark pixel ratio is high dark portion ratio, the noise intensity preliminary judgment value is low noise, the illumination uniformity state is uneven illumination, and the detail state information is low detail, the algorithm combination of the matching layered brightness adjustment algorithm and the texture generation enhancement algorithm is matched; wherein, the combination algorithm is configured with a three-level brightness layering (dark / medium / bright) mechanism, and is assisted by texture generation enhancement, and the weight coefficient thereof acts on the interval of 0.6-0.8.

[0122] ⑨When the feature matrix indicates that the specific type corresponding to the dark pixel ratio is medium dark portion ratio, the specific noise type is mixed noise, and the definition state information is detail blur, the algorithm combination of the matching guided filter denoising algorithm and the detail enhancement algorithm is matched; wherein, in the guided filter denoising algorithm, the window radius of the guided filter is set to 3; in the detail enhancement algorithm, the weight coefficient of the Unsharp Mask detail enhancement is set to 0.3.

[0123] ⑩When the scene category is an extreme low light scene and the feature matrix indicates that the specific type corresponding to the dark pixel ratio is extremely high dark portion ratio, the noise intensity preliminary judgment value is low noise or the specific noise type is mixed noise, and the detail state information is extremely low detail, the algorithm combination of the matching multi-frame RGB alignment denoising algorithm, the layered brightness enhancement algorithm and the U-Net noise repair algorithm is matched.

[0124] It should be noted that the various algorithms involved in the preset matching strategy are all algorithms in the prior art, and a person skilled in the art can flexibly select them according to actual needs, which will not be described here.

[0125] Step S15: using the image enhancement algorithm or algorithm combination matched for each region image to perform image enhancement processing on the corresponding region image;

[0126] Step S16: sequentially performing cross-partition fusion and color consistency calibration on the region images after the enhancement processing to obtain the calibrated RGB image, and performing natural degree optimization processing on the calibrated RGB image to output the RGB enhanced image.

[0127] Optionally, step S16 specifically includes:

[0128] Step S161: performing pre-smoothing filter processing on the edge pixel band of the adjacent region image to reduce the edge brightness difference;

[0129] Optionally, step S161 specifically includes:

[0130] Step S1611: identifying the edges of the adjacent region image, and selecting the region with a preset width on both sides of each edge as the edge pixel band;

[0131] The preset width can be 15 pixels to 20 pixels, which can be selected by those skilled in the art according to actual needs, and is not limited herein.

[0132] Step S1612: According to the scene category, the window size, the number of iterations and the brightness difference control target of the pre-smoothing filter are selected, wherein the window size of the pre-smoothing filter includes a first size and a second size, and the brightness difference control target includes a first target value and a second target value.

[0133] The first size is 5x5, the second size is 7x7, the first target value is 3, and the second target value is 2, which can be selected by those skilled in the art according to actual needs, and is not limited herein.

[0134] Step S1613: If the scene category is a regular low-light scene, a filter kernel of the first size is used to perform filtering processing on each edge pixel band once, so that the brightness difference of the edge of the adjacent image area is less than the first target value.

[0135] Step S1614: If the scene category is an extreme low-light scene, a filter kernel of the second size is used to perform at least two times of iterative filtering processing on each edge pixel band, so that the brightness difference of the edge of the adjacent area image is less than the second target value, wherein the second size is greater than the first size, and the second target value is less than the first target value.

[0136] Step S162: Multi-scale fusion is performed on each area image under a plurality of different preset resolutions, wherein the number of resolution levels used by the extreme low-light scene is greater than the number of resolution levels used by the regular low-light scene.

[0137] Optionally, step S162 specifically includes:

[0138] Step S1621: According to the scene category, the number of resolution levels used for fusion, and the resolution and fusion weight of each level are determined.

[0139] Step S1622: If the scene category is a regular low-light scene, the number of resolution levels is a first number, and a first set of resolutions and a first set of weights are used for fusion.

[0140] Step S1623: If the scene category is an extreme low-light scene, the number of resolution levels is a second number, and a second set of resolutions and a second set of weights are used for fusion; wherein the second number is greater than the first number.

[0141] Step S1624: According to the determined number of resolution levels and the resolution of each level, a corresponding multi-resolution image sequence is generated for each area image after image enhancement processing.

[0142] Step S1625: At each determined resolution level, the region images corresponding to the resolution level are fused to generate a primary fusion RGB image of the level;

[0143] The region images can be fused by using a Poisson fusion algorithm in the prior art, and of course, other fusion algorithms can be flexibly selected by those skilled in the art according to actual needs, which are not limited herein.

[0144] Step S1626: The primary fusion RGB images of each resolution level are weighted and superimposed according to the corresponding fusion weights to generate a final fusion RGB image.

[0145] In a conventional low-light scene, the first number is 3, that is, the number of resolution levels is 3, and the resolutions of the levels are the original resolution of the RGB image, 1 / 2 of the original resolution of the RGB image, and 1 / 4 of the original resolution of the RGB image (i.e., a first resolution set), and the fusion weights of the above levels are 0.6, 0.3, and 0.1 (i.e., a first weight set) in turn; in an extreme low-light scene, the second number is 4, the resolutions of the levels are the original resolution of the RGB image, 1 / 2 of the original resolution of the RGB image, 1 / 4 of the original resolution of the RGB image, and 1 / 8 of the original resolution of the RGB image (i.e., a second resolution set) in turn, and the fusion weights of the above levels are 0.5, 0.3, 0.15, and 0.05 (a second weight set) in turn.

[0146] Taking a conventional low-light scene (3 resolution levels) as an example, the specific process is as follows:

[0147] First resolution level fusion (1 / 4 resolution): the 1 / 4 resolution region images of the center region, the 1 / 4 resolution region images of the near-edge transition region, and the 1 / 4 resolution region images of the far-edge low-light region are three region images of the same scale, which are spliced together according to their geographical positions by using a Poisson fusion algorithm to output a complete first primary fusion RGB image F1 of 1 / 4 resolution.

[0148] Second resolution level fusion (1 / 2 resolution): the 1 / 2 resolution region images of the center region, the 1 / 2 resolution region images of the near-edge transition region, and the 1 / 2 resolution region images of the far-edge low-light region are three region images of the same scale, which are spliced together according to their geographical positions by using a Poisson fusion algorithm to output a complete second primary fusion RGB image F2 of 1 / 2 resolution.

[0149] Third resolution level fusion (original resolution): three region images of the same scale, i.e., the original resolution region image of the center region, the original resolution region image of the near-edge transition region, and the original resolution region image of the far-edge low-light region, are spliced together according to their geographical positions by using a Poisson fusion algorithm, and a complete third primary fusion RGB image of original resolution, i.e., F3, is output.

[0150] Finally, F1, F2, and F3 are weighted and superimposed according to the first weight set to generate a fused RGB image, i.e., the fused RGB image = 0.1xF1+0.3xF2+0.6xF3. Other scenes are sequentially similar, and details are not repeated here.

[0151] Step S163: The color consistency of the fused RGB image is calibrated so that the color mean values of the near-edge transition region and the far-edge low-light region are the same as the color mean value of the center region, and a calibrated RGB image is obtained.

[0152] Optionally, step S163 specifically includes:

[0153] Step S1631: A pixel set corresponding to the center region of the pre-fusion RGB image in the fused RGB image is extracted, and the color mean value thereof is calculated as a calibration reference.

[0154] Step S1632: The color mean values of the pixel sets corresponding to the near-edge transition region and the far-edge low-light region in the pre-fusion RGB image in the fused RGB image are respectively calculated, and the deviations of the color mean values from the calibration reference are calculated.

[0155] Step S1633: The fused RGB image is color adjusted according to the deviations until each deviation is less than a preset color deviation threshold, and a calibrated RGB image is obtained.

[0156] Step S164: The calibrated RGB image is input into a naturalness evaluation model to obtain a naturalness similarity score, and the parameters of the calibrated RGB image are adjusted and optimized according to the naturalness similarity score.

[0157] The naturalness evaluation model adopts a MobileNetV3 model in the prior art. The training data used by the model contains 30,000 groups of normal light panoramic RGB images, and 20,000 groups of “low light-normal light” paired samples are additionally supplemented for extreme low light scenes. The model outputs the naturalness similarity S of the calibrated RGB image and the normal light image. In a conventional low light scene, when the naturalness similarity S is less than 0.8, the parameters of the calibrated RGB image are adjusted; in an extreme low light scene, when the naturalness similarity S is less than 0.75, the parameters of the calibrated RGB image are adjusted.

[0158] It should be noted that steps S141 to S144, steps S161 to S164, steps S1611 to S1614, steps S16121 to S1626, steps S16131 to S1633 are only for ease of description and are not shown in the figure.

[0159] In the present application, the three independently enhanced region images are seamlessly fused into one complete RGB enhanced image, which not only eliminates the splicing traces, but also through color calibration and naturalness optimization, the final enhancement effect not only improves the picture quality, but also maintains a high degree of visual realism and naturalness.

[0160] The adaptive image enhancement method in low-light scenes provided by the present application realizes accurate identification and full-link adaptation of different light intensities by constructing a "regular-extreme" low-light scene layered judgment mechanism, effectively solving the core contradiction between noise suppression and detail preservation in extreme low-light scenes. And using the adaptive partitioning strategy of "spatial position and light gradient", combined with the "light-noise-detail" three-dimensional feature analysis enabled by deep learning, the optimal enhancement algorithm is dynamically matched for each partition. In addition, it also significantly eliminates splicing traces and color distortion through multi-scale cross-partition fusion and naturalness optimization technology, improving the visual naturalness of the image.

[0161] Based on the same inventive concept, the embodiments of the present application also provide an adaptive image enhancement device in low-light scenes for implementing the adaptive image enhancement method in low-light scenes as described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more adaptive image enhancement device embodiments in low-light scenes provided below can refer to the limitations of the adaptive image enhancement method in low-light scenes in the above, which will not be repeated here.

[0162] As shown in Figure 2 The present application provides an adaptive image enhancement device in low-light scenes, which comprises a scene judgment module 21, a partition processing module 22, a feature extraction module 23, a strategy matching module 24, an enhancement processing module 25 and a fusion optimization module 26; wherein,

[0163] The scene judgment module 21 is used for low-light scene judgment of the RGB image to obtain the corresponding scene category, wherein the scene category includes regular low-light scene and extreme low-light scene;

[0164] The partition processing module 22 is connected with the scene judgment module 21 and is used for adaptive partitioning processing of the RGB image according to the scene category to obtain a plurality of region images, wherein the plurality of region images include a center region, a near-edge transition region and a far-edge low-light region;

[0165] The feature extraction module 23 is connected with the partition processing module 22, configured to extract, for each of the plurality of region images, illumination features, noise features and detail features respectively, and generate a feature matrix corresponding to each of the region images according to the extracted illumination features, noise features and detail features;

[0166] The strategy matching module 24 is connected with the feature extraction module 23, configured to dynamically match, according to the scene category and the feature matrix, a corresponding image enhancement algorithm or algorithm combination for each of the region images from a preset enhancement strategy library through a preset matching strategy;

[0167] The enhancement processing module 25 is connected with the strategy matching module 24, configured to perform image enhancement processing on the corresponding region images by using the image enhancement algorithm or algorithm combination matched for each of the region images, to obtain the region images after enhancement processing.

[0168] The fusion optimization module 26 is connected with the enhancement processing module 25, configured to sequentially perform cross-partition fusion and color consistency calibration on the region images after enhancement processing, to obtain the calibrated RGB image, and perform naturalness optimization processing on the calibrated RGB image, to output the RGB enhanced image.

[0169] Optionally, the scene determination module 21 is specifically configured to: calculate a global brightness mean value, a dynamic range value, a relative color temperature index based on RGB distribution and a local dark area proportion in the RGB image respectively; if the global brightness mean value is less than a first brightness threshold, determine that the scene category is an extreme low-light scene; if the global brightness mean value is greater than or equal to the first brightness threshold, perform the following judgment: if any of the following conditions is met, determine that the scene category is a regular low-light scene: the global brightness mean value is less than a second brightness threshold and the dynamic range value is less than a dynamic range threshold; or, the global brightness mean value is less than a third brightness threshold and the relative color temperature index based on the RGB distribution is less than a color temperature threshold; or, the local dark area proportion in the RGB image is greater than a dark area proportion threshold; if none of the conditions is met, perform mild contrast optimization processing on the RGB image.

[0170] Optionally, the partition processing module 22 is specifically configured to: divide the RGB image into a center region and an edge region based on a spatial position division method; obtain a lens distortion parameter of the camera, and perform distortion pre-correction on the edge region by using the lens distortion parameter; calculate a brightness difference value between a brightness value of each pixel in the corrected edge region and a brightness mean value of the center region, and subdivide the edge region into a near-edge transition region and a far-edge low-light region based on the brightness difference value; when the scene category is the extreme low-light scene, perform noise pre-detection on the center region, the near-edge transition region and the far-edge low-light region respectively, and mark a high-noise region.

[0171] Optionally, the feature extraction module 23 extracts the illumination feature for each of the plurality of region images, including: selecting a dark part brightness threshold and a brightness standard deviation threshold based on the scene category, respectively; filtering out the dark part pixels in each of the plurality of region images based on the selected dark part brightness threshold, to obtain the number of dark part pixels in each of the plurality of region images; calculating the ratio of the number of dark part pixels in each of the plurality of region images to the total number of pixels in the corresponding region image, to obtain the dark part pixel ratio of the corresponding region image, and quantifying the dark part pixel ratio to a specific type according to the dark part pixel ratio and a preset dark part ratio threshold; calculating the brightness standard deviation of each of the plurality of region images, and comparing each brightness standard deviation with the selected brightness standard deviation threshold, to mark the illumination uniformity state of the corresponding region image according to the comparison result; and taking the specific type corresponding to the dark part pixel ratio and the illumination uniformity state of the plurality of region images as the illumination feature.

[0172] Optionally, the feature extraction module 23 extracts the noise feature for each of the plurality of region images, including: selecting a corresponding second noise intensity threshold based on the scene category; calculating the variance of the pixel gray scale difference before and after filtering for each of the plurality of region images by filtering through a preset window; comparing the calculated variance with the selected second noise intensity threshold, to determine a noise intensity preliminary value according to the comparison result; inputting each of the plurality of region images into a lightweight noise classification model to obtain corresponding noise type information, and quantifying the noise type information to a specific noise type; and taking the noise intensity preliminary value and the specific noise type of the plurality of region images as the noise feature.

[0173] Optionally, the feature extraction module 23 extracts the detail feature for each of the plurality of region images, including: selecting a gradient threshold, a detail ratio threshold and a Laplacian variance threshold based on the scene category, respectively; calculating the edge gradient value of each pixel in each of the plurality of region images through an edge detection operator; calculating the pixel ratio of each edge gradient value greater than the selected gradient threshold, and comparing each pixel ratio with the detail ratio threshold, to mark the detail state information according to the comparison result; calculating the Laplacian variance of each of the plurality of region images, and comparing each Laplacian variance with the Laplacian variance threshold, to mark the definition state information according to the comparison result; and taking the detail state information and the definition state information as the detail feature.

[0174] Optionally, the policy matching module 24 is specifically configured to: if the scene category is an extreme low-light scene, call a first decision tree model to match an extreme low-light exclusive algorithm combination from the enhancement strategy library based on the feature matrix; if the scene category is a regular low-light scene, call a second decision tree model to filter a preset number of candidate algorithms or algorithm combinations from the enhancement strategy library based on the feature matrix; calculate a comprehensive weight score of each candidate algorithm or algorithm combination based on a weight voting mechanism; and select a candidate algorithm or algorithm combination with the highest comprehensive weight score as the image enhancement algorithm or algorithm combination for the corresponding regional image.

[0175] Optionally, the fusion optimization module 26 is specifically configured to: perform pre-smoothing filter processing on edge pixel bands of the adjacent regional images to reduce edge brightness differences; perform multi-scale fusion on the regional images at a plurality of different preset resolutions, wherein the number of resolution levels used for the extreme low-light scene is greater than the number of resolution levels used for the regular low-light scene; perform color consistency calibration on the fused RGB image to make the average values of the colors of the near-edge transition region, the far-edge low-light region, and the central region the same, thereby obtaining a calibrated RGB image; and input the calibrated RGB image into a naturalness evaluation model to obtain a naturalness similarity score, and adjust and optimize the parameters of the calibrated RGB image according to the score result.

[0176] Optionally, the pre-smoothing filter processing on the edge pixel bands of the adjacent regional images to reduce the edge brightness differences includes: identifying edges of the adjacent regional images and selecting regions with a preset width on both sides of each edge as edge pixel bands; selecting a window size, an iteration number, and a brightness difference control target of the pre-smoothing filter according to the scene category, wherein the window size of the pre-smoothing filter includes a first size and a second size, and the brightness difference control target includes a first target value and a second target value; if the scene category is a regular low-light scene, performing one-time filter processing on each edge pixel band using a filter kernel with the first size to make the brightness difference of the edges of the adjacent image regions less than the first target value; and if the scene category is an extreme low-light scene, performing at least two iteration filter processing on each edge pixel band using a filter kernel with the second size to make the brightness difference of the edges of the adjacent regional images less than the second target value, wherein the second size is greater than the first size, and the second target value is less than the first target value.

[0177] Optionally, the multi-scale fusion is performed on the region images at different preset resolutions, wherein the number of resolution levels used for the extremely low-light scene is greater than the number of resolution levels used for the regular low-light scene, including: determining the number of resolution levels, and the resolution and fusion weight of each level, for the fusion according to the scene category; if the scene category is a regular low-light scene, the number of resolution levels is a first number, and a first set of resolutions and a first set of weights are used for the fusion; if the scene category is an extremely low-light scene, the number of resolution levels is a second number, and a second set of resolutions and a second set of weights are used for the fusion; wherein the second number is greater than the first number; generating a corresponding multi-resolution image sequence for each region image after the image enhancement processing according to the determined number of resolution levels and the resolution of each level; fusing the region images corresponding to each resolution level to generate a primary fusion RGB image of the level at each determined resolution level; and weighting and superimposing the primary fusion RGB images of each resolution level according to the corresponding fusion weights to generate a final fusion RGB image.

[0178] Optionally, the fused RGB image is subjected to color consistency calibration, so that the average value of the color of the near-edge transition region and the far-edge low-light region is the same as the average value of the color of the central region, to obtain a calibrated RGB image, including: extracting a pixel set corresponding to the central region of the RGB image before the fusion from the fused RGB image, and calculating the average value of the color thereof as a calibration reference; calculating the deviation of the average value of the color of the pixel set corresponding to the near-edge transition region and the far-edge low-light region of the RGB image before the fusion from the calibration reference; and performing color adjustment on the fused RGB image according to the deviation until each deviation is less than a preset color deviation threshold, to obtain the calibrated RGB image.

[0179] The adaptive image enhancement device in a low-light scene provided by the application realizes accurate identification and full-link adaptation of different light intensities by constructing a "regular-extreme" low-light scene layered judgment mechanism, effectively solving the core contradiction between noise suppression and detail preservation in an extremely low-light scene. And an adaptive partitioning strategy of "spatial position and light gradient" is adopted, and combined with the "light-noise-detail" three-dimensional feature analysis enabled by deep learning, the optimal enhancement algorithm is dynamically matched for each partition. In addition, it also significantly eliminates the stitching marks and color distortion through multi-scale cross-partition fusion and naturalness optimization technology, and improves the visual naturalness of the image.

[0180] It should be noted that in the application, a plurality of includes two or more.

[0181] It should be understood that although the steps in the flowcharts related to the embodiments described above are shown in sequence according to the arrows, the steps are not necessarily executed in sequence according to the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other sequences. Moreover, at least some of the steps in the flowcharts related to the embodiments described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of the steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least some of the other steps or steps or stages in the other steps.

[0182] The various modules in the devices in the present application can be implemented wholly or partially by software, hardware and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above modules.

[0183] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram thereof can be as shown in Figure 3 The computer device includes a processor, a memory and a network interface connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store data required for or generated by the above adaptive image enhancement method in a low light scene. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement an adaptive image enhancement method in a low light scene.

[0184] In one embodiment, a computer device is provided, which can be a terminal, and an internal structure diagram thereof can be as shown in Figure 3As shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used for wired or wireless communication with external terminals. Wireless communication can be achieved through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by the processor to implement a self-adaptive image enhancement method in a low-light scene. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad provided on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0185] Those skilled in the art can understand that, Figure 3 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. A specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0186] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the above method embodiments.

[0187] In one embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0188] In one embodiment, a computer program product is provided, including a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0189] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0190] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0191] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0192] The above embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application.

Claims

1. An adaptive image enhancement method in low-light scenes, characterized in that, The method comprises the following steps: perform low-light scene judgment on the RGB image to obtain a corresponding scene category, wherein the scene category comprises a regular low-light scene and an extreme low-light scene; perform adaptive partition processing on the RGB image according to the scene category to obtain a plurality of regional images, wherein the plurality of regional images comprise a center region, a near-edge transition region and a far-edge low-light region; extract a light feature, a noise feature and a detail feature from each of the plurality of regional images, respectively, and generate a feature matrix corresponding to each of the plurality of regional images according to the extracted light feature, noise feature and detail feature; according to the scene category and the feature matrix, dynamically match a corresponding image enhancement algorithm or algorithm combination for each of the plurality of regional images from a preset enhancement strategy library through a preset matching strategy; perform image enhancement processing on the corresponding regional image by using the image enhancement algorithm or algorithm combination matched for each of the plurality of regional images to obtain each regional image after enhancement processing; perform cross-partition fusion and color consistency calibration on each regional image after enhancement processing in sequence to obtain a calibrated RGB image, and perform naturalness optimization processing on the calibrated RGB image to output an RGB enhanced image.

2. The method of claim 1, wherein, The method of performing low-light scene judgment on the RGB image to obtain a corresponding scene category comprises the following steps: respectively calculate a global brightness mean value, a dynamic range value, a relative color temperature index based on RGB distribution and a local dark area proportion in the RGB image; if the global brightness mean value is less than a first brightness threshold, the scene category is determined as an extreme low-light scene; if the global brightness mean value is greater than or equal to the first brightness threshold, the following judgment is performed: if any of the following conditions is met, the scene category is determined as a regular low-light scene: the global brightness mean value is less than a second brightness threshold and the dynamic range value is less than a dynamic range threshold; or the global brightness mean value is less than a third brightness threshold and the relative color temperature index based on RGB distribution is less than a color temperature threshold; or the local dark area proportion in the RGB image is greater than a dark area proportion threshold; if none of the conditions is met, perform slight contrast optimization processing on the RGB image.

3. The method of claim 1, wherein, The method of performing adaptive partition processing on the RGB image according to the scene category to obtain a plurality of regional images, wherein the plurality of regional images comprise a center region, a near-edge transition region and a far-edge low-light region, comprises the following steps: divide the RGB image into a center region and an edge region based on a spatial position division method; obtain a lens distortion parameter of a camera, and perform distortion pre-correction on the edge region by using the lens distortion parameter; calculate a brightness difference value between a brightness value of each pixel in the corrected edge region and a brightness mean value of the center region, and subdivide the edge region into a near-edge transition region and a far-edge low-light region based on the brightness difference value; when the scene category is an extreme low-light scene, perform noise pre-detection on the center region, the near-edge transition region and the far-edge low-light region, respectively, and mark a high-noise region.

4. The method of claim 1, wherein, The method of extracting a light feature from each of the plurality of regional images comprises the following steps: select a dark part brightness threshold and a brightness standard deviation threshold, respectively, based on the scene category; Screening dark pixels in each of the plurality of region images based on the selected dark part brightness threshold value, to obtain a number of dark pixels in each of the plurality of region images; Respectively calculating a ratio of the number of dark pixels in each of the plurality of region images to a total number of pixels in the corresponding region image, to obtain a dark pixel proportion of the corresponding region image, and quantifying the dark pixel proportion into a specific type according to the dark pixel proportion and a preset dark part proportion threshold value; Respectively calculating a brightness standard deviation of each of the plurality of region images, and comparing each brightness standard deviation with a selected brightness standard deviation threshold value, to mark a light uniformity state of the corresponding region image according to a comparison result; Taking the specific type of the dark pixel proportion and the light uniformity state of the plurality of region images as a light feature.

5. The method of claim 1, wherein, For each of the plurality of region images, extracting a noise feature, including: Selecting a corresponding second noise intensity threshold value based on the scene category; Calculating a variance of pixel gray scale difference values before and after filtering of each of the plurality of region images through preset window filtering; Comparing the calculated variance with the selected second noise intensity threshold value, to determine a noise intensity preliminary value according to a comparison result; Inputting each of the plurality of region images into a lightweight noise classification model, to obtain corresponding noise type information, and quantifying the noise type information into a specific noise type; Taking the noise intensity preliminary value and the specific noise type of the plurality of region images as the noise feature.

6. The method of claim 1, wherein, For each of the plurality of region images, extracting a detail feature, including: Respectively selecting a gradient threshold value, a detail proportion threshold value and a Laplacian variance threshold value based on the scene category; Calculating edge gradient values of each pixel in each of the plurality of region images through an edge detection operator; Calculating a pixel proportion of each edge gradient value greater than the selected gradient threshold value, and comparing each pixel proportion with the detail proportion threshold value, to mark detail state information according to a comparison result; Calculating a Laplacian variance of each of the plurality of region images, and comparing each Laplacian variance with the Laplacian variance threshold value, to mark definition state information according to a comparison result; Taking the detail state information and the definition state information as the detail feature.

7. The method of claim 1, wherein, According to the scene category and the feature matrix, a corresponding image enhancement algorithm or algorithm combination is dynamically matched for each region image from a preset enhancement strategy library through a preset matching strategy, including: If the scene category is an extreme low light scene, a first decision tree model is called, and an extreme low light exclusive algorithm combination is matched from the enhancement strategy library based on the feature matrix; If the scene category is a regular low light scene, a second decision tree model is called, and a preset number of candidate algorithms or algorithm combinations are screened from the enhancement strategy library based on the feature matrix; Based on a weight voting mechanism, a comprehensive weight score of each candidate algorithm or algorithm combination is calculated; The candidate algorithm or algorithm combination with the highest comprehensive weight score is selected as the image enhancement algorithm or algorithm combination of the corresponding region image.

8. The method of claim 1, wherein, The enhanced region images are sequentially subjected to cross-partition fusion and color consistency calibration to obtain a calibrated RGB image, and the calibrated RGB image is subjected to naturalness optimization processing to output an RGB enhanced image, comprising: performing pre-smoothing filter processing on the edge pixel bands of the adjacent region images to reduce the edge brightness difference; performing multi-scale fusion on the region images under multiple different preset resolutions, wherein the number of resolution levels used in an extreme low-light scene is greater than the number of resolution levels used in a regular low-light scene; performing color consistency calibration on the fused RGB image, so that the mean values of the colors of the near-edge transition region and the far-edge low-light region are the same as the mean value of the color of the center region, to obtain a calibrated RGB image; inputting the calibrated RGB image into a naturalness evaluation model to obtain a naturalness similarity score, and adjusting and optimizing the parameters of the calibrated RGB image according to the score result.

9. The method of claim 8, wherein, The pre-smoothing filter processing on the edge pixel bands of the adjacent region images to reduce the edge brightness difference comprises: identifying the edges of the adjacent region images and selecting a region of a preset width on both sides of each edge as an edge pixel band; selecting a pre-smoothing filter window size, an iteration number, and a brightness difference control target according to the scene category, wherein the pre-smoothing filter window size includes a first size and a second size, and the brightness difference control target includes a first target value and a second target value; if the scene category is a regular low-light scene, a filter kernel of the first size is used to perform one-time filter processing on each edge pixel band to make the brightness difference of the edges of the adjacent image regions less than the first target value; if the scene category is an extreme low-light scene, a filter kernel of the second size is used to perform at least two iterations of filter processing on each edge pixel band to make the brightness difference of the edges of the adjacent region images less than the second target value, wherein the second size is greater than the first size, and the second target value is less than the first target value.

10. An adaptive image enhancement device in low-light scenes, characterized by, comprising: a scene determination module for determining a low-light scene of an RGB image to obtain a corresponding scene category, wherein the scene category includes a regular low-light scene and an extreme low-light scene; a partition processing module connected to the scene determination module, for adaptively partitioning the RGB image according to the scene category to obtain a plurality of region images, the plurality of region images including a center region, a near-edge transition region, and a far-edge low-light region; a feature extraction module connected to the partition processing module, for extracting illumination features, noise features, and detail features from each of the plurality of region images, and generating a feature matrix corresponding to each region image according to the extracted illumination features, noise features, and detail features; a strategy matching module connected to the feature extraction module, for dynamically matching a corresponding image enhancement algorithm or algorithm combination for each region image from a preconfigured enhancement strategy library according to the scene category and the feature matrix through a preconfigured matching strategy; The enhancement processing module is connected with the strategy matching module, and is configured to perform image enhancement processing on the corresponding regional images by using an image enhancement algorithm or a combination of algorithms matched for each regional image, to obtain the regional images after the image enhancement processing; The fusion optimization module is connected with the enhancement processing module, and is configured to sequentially perform cross-partition fusion and color consistency calibration on the regional images after the image enhancement processing, to obtain the calibrated RGB images, and perform naturalness optimization processing on the calibrated RGB images, and output the RGB enhanced image.

Images