An image enhancement method and device, electronic equipment and readable storage medium

Abstract

The application discloses an image enhancement method and device, electronic equipment and readable storage medium, comprising: amplifying the pixel intensity of the obtained original low-light image according to a pre-amplification coefficient to obtain a pre-amplification image; performing multiple downsampling operations on the pre-amplification image to obtain a corresponding downsampling image for each downsampling operation; inputting each downsampling image into a corresponding scale calculation model to output a scale calculation image corresponding to the downsampling image; the scale calculation model is used for at least one convolution processing calculation on the downsampling image; all the scale calculation images are fused to obtain a fusion image, and then convolution processing is performed, and a target image is output through upsampling operation, the target image has higher image quality relative to the original low-light image, the low-light image is enhanced, manual processing of staff is not required, work efficiency is improved, and the problem of low work efficiency in the prior art is solved.

Description

Technical Field

[0001] This application specifically relates to an image enhancement method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] To enhance low-light images, an image enhancement method is needed.

[0003] In the prior art, staff manually enhance low-light images using image editing software (such as Adobe Photoshop).

[0004] In the process of developing this application, the inventors discovered at least the following problems in the prior art: low work efficiency is caused by workers manually enhancing low-light images using image editing software. Summary of the Invention

[0005] This application aims to provide an image enhancement method, apparatus, electronic device, and readable storage medium, which at least solves the problem of low work efficiency caused by workers manually enhancing low-light images using image editing software in the prior art.

[0006] To solve the above-mentioned technical problems, this application is implemented as follows:

[0007] In a first aspect, embodiments of this application provide an image enhancement method, the method comprising:

[0008] The pixel intensity of the original low-light image is magnified according to the pre-magnification factor to obtain a pre-magnified image.

[0009] The pre-enlarged image is subjected to multiple downsampling operations to obtain a downsampled image corresponding to each downsampling operation; the sampling factor corresponding to each of the multiple downsampling operations decreases sequentially.

[0010] Each downsampled image is input into a corresponding scale calculation model, and a scale-calculated image corresponding to the downsampled image is output; the scale calculation model is used to perform at least one convolution operation on the downsampled image; the sampling factor and the number of convolution operations performed by the scale calculation model are positively correlated.

[0011] All the scale-calculated images are fused together to obtain a fused image, which is then subjected to convolution processing and upsampling to output the target image.

[0012] Secondly, embodiments of this application also provide an image enhancement apparatus, the apparatus comprising:

[0013] The pre-magnification module is used to amplify the pixel intensity of the acquired original low-light image according to the pre-magnification factor to obtain a pre-magnified image.

[0014] The downsampling module is used to perform multiple downsampling operations on the pre-enlarged image to obtain a downsampled image corresponding to each downsampling operation; the sampling factor of each of the multiple downsampling operations decreases sequentially.

[0015] The scaling module is used to input each downsampled image into the corresponding scaling model and output a scaled image corresponding to the downsampled image; the scaling model is used to perform at least one convolution processing calculation on the downsampled image; the sampling factor and the number of convolution processing calculations of the scaling model are positively correlated.

[0016] The fusion module is used to fuse all the scale-calculated images, and after obtaining the fused image, it performs convolution processing and outputs the target image after upsampling.

[0017] Thirdly, embodiments of this application also provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0018] Fourthly, embodiments of this application also provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the method described in the first aspect.

[0019] In this embodiment, the pixel intensity of the acquired original low-light image is magnified according to a pre-magnification factor to obtain a pre-magnified image. Multiple downsampling operations are performed on the pre-magnified image to obtain a downsampled image corresponding to each downsampling operation. The sampling factor for each downsampling operation decreases sequentially. Each downsampled image is input into a corresponding scale calculation model, which outputs a scale-calculated image corresponding to the downsampled image. The scale calculation model performs at least one convolution operation on the downsampled image. The sampling factor and the number of convolution operations in the scale calculation model are positively correlated. All the scale-calculated images are fused to obtain a fused image, which is then convolved and upsampled to output a target image. The target image has higher image quality than the original low-light image, achieving low-light image enhancement. This eliminates the need for manual enhancement using image editing software, improving work efficiency and solving the problem of low efficiency caused by manual enhancement of low-light images using image editing software. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the steps of an image enhancement method provided in an embodiment of this application;

[0021] Figure 2 This is a flowchart illustrating the specific steps of an image enhancement method provided in an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of the image enhancement process corresponding to an image enhancement method provided in an embodiment of this application;

[0023] Figure 4 This is a block diagram of an image enhancement device provided in an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] It should be noted that in the embodiments of this application, the original low-light image is a low-light image, such as an image under short exposure or an image under extremely low ambient light under long exposure. The target image is an image obtained by enhancing the original low-light image, that is, by processing the problems of low brightness, low contrast, noise, artifacts and other issues existing in the original low-light image with insufficient lighting, and improving the visual quality of the original low-light image.

[0027] The image enhancement method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0028] Figure 1 This is a flowchart illustrating the steps of an image enhancement method provided in an embodiment of this application, as follows: Figure 1 As shown, the method may include:

[0029] Step 101: Magnify the pixel intensity of the original low-light image according to the pre-magnification factor to obtain a pre-magnified image.

[0030] It's important to note that pixel intensity refers to the brightness of a pixel in an image. Pixel brightness values ​​range from 0 to 255. Pixels closer to 255 are brighter, those closer to 0 are darker, and the rest are considered midtones. This distinction in brightness is absolute; pixels near 255 are highlights, pixels near 0 are shadows, and midtones are around 128.

[0031] Specifically, in some embodiments, the original low-light image is obtained through staff input, such as by staff uploading images or by staff taking pictures.

[0032] Optionally, in some embodiments, the pre-magnification factor is used as the magnification factor of the pixel intensity of the original low-light image, such that the pixel intensity of each pixel in the original low-light image is multiplied by the pre-magnification factor to obtain the pixel intensity of each pixel in the pre-magnified image, and the pixel intensity of each pixel in the pre-magnified image is greater than the pixel intensity of the corresponding pixel in the original low-light image. For example, if the pixel intensity of the (a, b)th pixel in the original low-light image is 10 and the pre-magnification factor is 20, then the pixel intensity of the corresponding (a, b)th pixel in the pre-magnified image is 200 (obtained by 10 × 20). The pre-magnification factor is calculated based on the weighted average of the pixel intensities of all pixels in the original low-light image and the preset magnification parameter.

[0033] After obtaining the pre-magnification factor, the pixel intensity of each pixel in the original low-light image is multiplied by the pre-magnification factor to obtain the pixel intensity of each pixel in the pre-magnified image. That is, the pixel intensity of the original low-light image is magnified to obtain the pre-magnified image, so as to achieve preliminary image enhancement processing of the original low-light image. After this step, the pixel intensity of the pre-magnified image is improved compared with the pixel intensity of the original low-light image. From the perspective of human vision, the overall brightness of the pre-magnified image is improved compared with the original low-light image, thus improving the visual quality of the image.

[0034] Step 102: Perform multiple downsampling operations on the pre-enlarged image to obtain the downsampled image corresponding to each downsampling operation.

[0035] The sampling multiples for each of the multiple downsampling operations decrease sequentially.

[0036] Specifically, in some embodiments, the downsampling operation is an operation that reduces the resolution of the pre-enlarged image. That is, the resolution of the pre-enlarged image is divided by a downsampling factor to obtain the resolution of the downsampled image. The larger the downsampling factor, the smaller the resolution of the downsampled image. For example, a downsampling factor of 2 means dividing the resolution of the pre-enlarged image by 2, resulting in an image with half the resolution of the pre-enlarged image as the downsampled image. For instance, if the resolution of the pre-enlarged image is 640×320, after a downsampling operation of factor 2 (640 divided by 2 (resulting in 320) and 320 divided by 2 (resulting in 160), a downsampled image with a resolution of 320×160 is obtained.

[0037] Optionally, in some embodiments, the number of downsampling operations can be 2, 3, 4, 5, 6, etc. For example, if the resolution of the pre-enlarged image is 640×320, the number of downsampling operations is 4, namely: performing a downsampling operation of 4 times on the pre-enlarged image to obtain a downsampled image A with a resolution of 160×80, performing a downsampling operation of 8 times on the pre-enlarged image to obtain a downsampled image B with a resolution of 80×40, performing a downsampling operation of 16 times on the pre-enlarged image to obtain a downsampled image C with a resolution of 40×20, and performing a downsampling operation of 32 times on the pre-enlarged image to obtain a downsampled image D with a resolution of 20×10.

[0038] By performing multiple downsampling operations on the pre-enlarged image, a downsampled image corresponding to each downsampling operation is obtained. This is the first step in further enhancing the original low-light image. The resulting multiple downsampled images will be input into the corresponding scale calculation models for further processing.

[0039] Step 103: Input each downsampled image into the corresponding scale calculation model and output the scale calculation image corresponding to the downsampled image.

[0040] The scale calculation model is used to perform at least one convolution calculation on the downsampled image; the sampling factor and the number of convolution calculations performed by the scale calculation model are positively correlated.

[0041] Specifically, in some embodiments, the larger the sampling factor of the downsampled image, the more times the convolution processing calculation is performed in the scale calculation model corresponding to the downsampled image; the convolution processing calculation can be a single-layer convolution processing calculation, a multi-layer convolution processing calculation, a hybrid calculation including a single channel shuffling convolution processing calculation and a single group convolution processing calculation, or a residual dense block calculation including multiple convolution layers for nonlinear correction, etc.

[0042] It's important to note that single-layer and multi-layer convolutional processing are two different convolution methods, primarily differing in the number of convolutional layers and the number of parameters. A single-layer convolutional neural network model refers to a convolutional neural network model with only one convolutional layer. This model is suitable for processing simple images or feature extraction. A single-layer convolutional processing model contains only one convolutional layer and one activation function; the input data is processed by the convolution operation to output a feature map. Due to the fewer layers, the number of parameters is smaller, and the training speed is relatively fast. However, single-layer convolutional processing lacks sufficient feature extraction capabilities and may fail to capture complex image representations. Multi-layer convolutional processing, on the other hand, refers to a convolutional neural network model containing multiple convolutional layers, where each convolutional layer adds non-linear operations such as activation functions and batch normalization. Multi-layer convolutional processing can improve feature extraction capabilities, resulting in better performance. However, due to the larger number of convolutional layers and the corresponding increase in parameters, the training speed is slower. However, convolutional processing with multiple convolutional layers can capture more image features and has a better effect on processing complex image tasks.

[0043] For example, the pre-enlarged image has a resolution of 640×320, and the downsampling operation is performed 4 times: downsampling the pre-enlarged image by a factor of 4 to obtain a downsampled image A with a resolution of 160×80; downsampling the pre-enlarged image by a factor of 8 to obtain a downsampled image B with a resolution of 80×40; downsampling the pre-enlarged image by a factor of 16 to obtain a downsampled image C with a resolution of 40×20; and downsampling the pre-enlarged image by a factor of 32 to obtain a downsampled image D with a resolution of 20×10.

[0044] Each downsampled image is input into its corresponding scale calculation model, and the output is a scale-calculated image corresponding to the downsampled image. For example, a downsampled image A with a resolution of 160×80 is input into the corresponding scale calculation model A for one convolution operation, and the output is the corresponding scale-calculated image A; a downsampled image B with a resolution of 80×40 is input into the corresponding scale calculation model B for two convolution operations, and the output is the corresponding scale-calculated image B; a downsampled image C with a resolution of 40×20 is input into the corresponding scale calculation model C for three convolution operations, and the output is the corresponding scale-calculated image C; a downsampled image D with a resolution of 20×10 is input into the corresponding scale calculation model D for four convolution operations, and the output is the corresponding scale-calculated image D.

[0045] By inputting each downsampled image into the corresponding scale calculation model, the output scale calculation image corresponding to the downsampled image is the second step to further enhance the original low-light image. The resulting multiple scale calculation images will then undergo further fusion processing.

[0046] Step 104: Fuse all the scale-calculated images to obtain the fused image, perform convolution processing, and output the target image after upsampling.

[0047] Specifically, in some embodiments, fusing all the scale-computed images can be done in steps. For example, the first image to be fused is fused with the second image to be fused to obtain a fused image, and then this fused image is fused with the third image to be fused. During the fusion of all the scale-computed images, the scale-computed images can be further processed before fusion, and the fused image can be further processed. For example, if scale-computed images A, B, C, and D need to be fused to output a target image, then scale-computed image C and its corresponding downsampled image C are fused to obtain a fused image C1, and then the fused image C1 is subjected to a convolution process. The scaling process is as follows: Image C2 is obtained by performing a single convolution operation on the downsampled image D corresponding to the scaled image D. Image D1 is then fused with the scaled image D to obtain the fused image D2. The fused image D2 is then subjected to two convolution operations and one upsampling operation to obtain image D3. Image C2 and image D3 are then fused to obtain image E1. The scaled image B is then subjected to a single convolution operation to obtain image B1. Image E2, obtained by performing a single convolution operation and one upsampling operation on image E1, is then fused with image B1 to obtain image E3. Image E3 and image A are then fused to obtain image E4. Image E4 is then subjected to a single convolution operation and one upsampling operation to obtain the target image.

[0048] The target image obtained by fusing all the scaled images has higher visual quality compared to the pre-magnified image; the target image also shows a more significant improvement in visual quality compared to the original low-light image.

[0049] In summary, the embodiments of this application can achieve the following: First, the pixel intensity of the acquired original low-light image is magnified according to a pre-magnification factor to obtain a pre-magnified image. Then, multiple downsampling operations are performed on the pre-magnified image to obtain a downsampled image corresponding to each downsampling operation. The sampling factor for each downsampling operation decreases sequentially. Each downsampled image is input into a corresponding scale calculation model, which outputs a scale-calculated image corresponding to the downsampled image. The scale calculation model performs at least one convolution calculation on the downsampled image. The sampling factor and the number of convolution calculations in the scale calculation model are positively correlated. All the scale-calculated images are fused to obtain a fused image, which is then convolved and upsampled to output a target image. The target image has higher image quality than the original low-light image, achieving low-light image enhancement. This eliminates the need for manual enhancement using image editing software, improving work efficiency and solving the problem of low work efficiency caused by manual enhancement using image editing software.

[0050] Figure 2 This is a flowchart illustrating the specific steps of an image enhancement method provided in an embodiment of this application, as follows: Figure 2 As shown, the method may include:

[0051] Step 201: Magnify the pixel intensity of the original low-light image according to the pre-magnification factor to obtain a pre-magnified image.

[0052] The implementation method of this step is similar to that of "Step 101" above, and will not be repeated here.

[0053] Optionally, in some embodiments, before performing "step 201", the method further includes:

[0054] Step 200: Obtain the pre-magnification factor based on the pixel intensity of the original low-light image.

[0055] Specifically, in some embodiments, a pre-magnification factor is calculated based on the pixel intensity of all pixels in the original low-light image, using a pre-magnification factor calculation method, as the basis for preliminary image enhancement processing of the original low-light image.

[0056] Optionally, in some embodiments, "step 200" includes the following steps (step 2001, step 2002, step 2003, step 2004):

[0057] Step 2001: Obtain the pixel intensity of each pixel in the original low-light image.

[0058] Specifically, in some embodiments, the pixel intensity of each pixel in the original low-light image is read by parsing the original low-light image. For example, the pixel intensity of the (c, d)th pixel in the original low-light image is read as 50, that is, the brightness of the pixel of the (c, d)th pixel in the original low-light image is 50.

[0059] By analyzing the original low-light image, we can obtain the feature information of each pixel, including the pixel intensity of each pixel, which can provide data for further processing of the original low-light image to enhance it.

[0060] Step 2002: Obtain the weight of each pixel in the original low-light image based on the pixel intensity of each pixel in the original low-light image.

[0061] Specifically, in some embodiments, based on the pixel intensity of each pixel in the original low-light image, a weight calculation method is designed to calculate the weight of each pixel in the original low-light image, which serves as the basis for further processing of the original low-light image. For example, the weight of the (i, j)th pixel in the original low-light image is calculated according to the following mathematical expression:

[0062]

[0063] Where, ω i,j Let b be the weight of the (i, j)th pixel in the original low-light image, k ∈ [1, n]. k For the edge of the k-th container, b k-1 x is the edge of the (k-1)th container. i,j Let be the pixel intensity of the (i, j)th pixel in the original low-light image.

[0064] Optionally, in some embodiments, the k-th container edge is calculated according to the following mathematical expression:

[0065]

[0066] Among them, b k b0 is the edge of the k-th container, and b0 is the edge of the 0-th container, where k∈[1,n].

[0067] For example, the pixel intensity range [0, 1] of a pixel is quantized into n containers, where k ranges from [1, n]. Assuming n is 100, the n containers can be represented as: [0, 0.01], [0.01, 0.02], [0.02, 0.03], ..., [0.97, 0.98], [0.98, 0.99], [0.99, 1].

[0068] Assuming k = 1, the pixel intensity range is [0, 0.01]. Substituting this into the mathematical expression...

[0069]

[0070] get:

[0071]

[0072] That is, the weight ω of the (i, j)th pixel in the original low-light image. i,j =1;

[0073] Assuming k = 100, the pixel intensity range is [0.99, 1]. Substituting this into the mathematical expression...

[0074]

[0075] get:

[0076]

[0077] That is, the weight ω of the (i, j)th pixel in the original low-light image. i,j Approximately 0;

[0078] As demonstrated by the examples above, the lower the pixel intensity of a pixel in the original low-light image, the closer its weight is to 1; conversely, the higher the pixel intensity, the closer its weight is to 0. This design ensures that pixels with higher pixel intensity have a lower weight, while pixels with lower pixel intensity have a higher weight. Consequently, pixels with lower pixel intensity have a greater impact on the initial processing of the original low-light image than pixels with higher pixel intensity. In subsequent steps, the pixel intensity of the original low-light image is amplified according to the pre-amplification factor obtained based on the weights, with a greater emphasis on amplifying the pixel intensity of the dark areas. This results in a pre-amplified image with higher visual quality, reducing the possibility of overly focusing on amplifying the pixel intensity of the bright areas and causing overly bright areas to appear as pops in the pre-amplified image.

[0079] Step 2003: Obtain a weighted average value based on the weights and pixel intensities of all pixels in the original low-light image.

[0080] Specifically, in some embodiments, based on the weights and pixel intensities of all pixels in the original low-light image, a weighted average of the pixel intensities of all pixels in the original low-light image can be obtained as the basis for further processing of the original low-light image. The weighted average is calculated according to the following mathematical expression:

[0081]

[0082] Where P is the weighted average, ω i,j Let x be the weight of the (i, j)th pixel in the original low-light image. i,j Let be the pixel intensity of the (i, j)th pixel in the original low-light image.

[0083] Step 2004: Obtain the pre-amplification coefficient based on the weighted average value and the preset amplification parameters.

[0084] Specifically, in some embodiments, a pre-magnification factor is calculated based on a weighted average and a preset magnification parameter, serving as the basis for further processing of the original low-light image. The pre-magnification factor is calculated according to the following mathematical expression:

[0085]

[0086] Where T is the pre-amplification factor, m is the preset amplification parameter, and ω i,j Let x be the weight of the (i, j)th pixel in the original low-light image. i,j Let be the pixel intensity of the (i, j)th pixel in the original low-light image.

[0087] It should be noted that the preset magnification parameters are parameters set by the staff in advance, for example, the preset magnification parameter can be 0.5.

[0088] By performing steps 2001, 2002, 2003, and 2004 in the embodiments of this application, the following can be achieved: obtaining the pixel intensity of each pixel in the original low-light image; obtaining the weight of each pixel in the original low-light image based on the pixel intensity of each pixel; obtaining a weighted average value based on the weights and pixel intensities of all pixels in the original low-light image; and obtaining a pre-magnification coefficient based on the weighted average value and a preset magnification parameter, which serves as the basis for the initial enhancement processing of the original low-light image.

[0089] Optionally, in some embodiments, the weight of the pixel is negatively correlated with the pixel intensity; the pre-amplification factor is negatively correlated with the weighted average value.

[0090] Specifically, in some embodiments, the relationship between the pixel weight and the pixel intensity is negatively correlated as shown in the mathematical expression for the weight above, and the relationship between the pre-amplification factor and the weighted average is negatively correlated as shown in the mathematical expression for the pre-amplification factor above.

[0091] The relationship between pixel weight and pixel intensity, and the relationship between pre-magnification factor and weighted average value, serve as the basis for the initial enhancement processing of the original low-light image.

[0092] Step 202: Perform multiple downsampling operations on the pre-enlarged image to obtain the downsampled image corresponding to each downsampling operation.

[0093] The sampling multiples for each of the multiple downsampling operations decrease sequentially.

[0094] The implementation method of this step is similar to that of "Step 102" above, and will not be repeated here.

[0095] Optionally, in some embodiments, the sampling factor and the resolution of the corresponding downsampled image are negatively correlated.

[0096] Specific examples are as described above and will not be repeated here.

[0097] Optionally, in some embodiments, "step 202" specifically includes the following steps (step 2021, step 2022, step 2023):

[0098] Step 2021: Perform a first downsampling operation on the pre-enlarged image to obtain a first downsampled image.

[0099] Specifically, in some embodiments, the sampling factor for the first downsampling can be 2, that is, the resolution of the pre-enlarged image is divided by 2, and the resulting image with half the resolution of the pre-enlarged image is the first downsampled image. For example, if the resolution of the pre-enlarged image is 640×320, after a first downsampling operation with a factor of 2, that is, 640 divided by 2 (resulting in 320) and 320 divided by 2 (resulting in 160), a first downsampled image with a resolution of 320×160 is obtained.

[0100] Step 2022: Perform a second downsampling operation on the pre-enlarged image to obtain a second downsampled image.

[0101] Specifically, in some embodiments, the sampling factor for the second downsampling can be 8, that is, the resolution of the pre-enlarged image is divided by 8, and the resulting image with 1 / 8 of the resolution of the pre-enlarged image is the second downsampled image. For example, if the resolution of the pre-enlarged image is 640×320, after a second downsampling operation with a factor of 8, that is, 640 divided by 8 (getting 80) and 320 divided by 8 (getting 40), a second downsampled image with a resolution of 80×40 is obtained.

[0102] Step 2023: Perform a third downsampling operation on the pre-enlarged image to obtain a third downsampled image.

[0103] Specifically, in some embodiments, the sampling factor for the third downsampling can be 32, that is, dividing the resolution of the pre-enlarged image by 32 to obtain an image with 1 / 32 of the resolution of the pre-enlarged image as the third downsampled image. For example, if the resolution of the pre-enlarged image is 640×320, after a third downsampling operation with a factor of 32, that is, 640 divided by 32 (getting 20) and 320 divided by 32 (getting 10), a third downsampled image with a resolution of 20×10 is obtained.

[0104] Optionally, in some embodiments, the sampling factor of the first downsampling operation is less than the sampling factor of the second downsampling operation; the sampling factor of the second downsampling operation is less than the sampling factor of the third downsampling operation. For example, the sampling factor of the first downsampling operation is 2, the sampling factor of the second downsampling operation is 8, and the sampling factor of the third downsampling operation is 32.

[0105] By performing steps 2021, 2022, and 2023 in the embodiments of this application, the pre-magnified image is downsampled by different sampling factors, resulting in downsampled images of different resolutions. This is beneficial for subsequent refined image enhancement processing, so that the final fused target image has higher visual quality.

[0106] Optionally, in some embodiments, the scale calculation model corresponding to the first downsampled image includes one convolutional processing calculation to output a first-scale calculated image; the scale calculation model corresponding to the second downsampled image includes two convolutional processing calculations to output a high-quality second-scale calculated image; and the scale calculation model corresponding to the third downsampled image includes one hybrid calculation and one residual dense block calculation to output the highest-quality third-scale calculated image among the three.

[0107] The embodiments of this application enable further refined enhancement processing for each downsampled image, that is, performing corresponding refined convolution processing calculations for different downsampled images, which helps to make the final target image have higher visual quality.

[0108] It should be noted that Residual Dense Block (RDB) can make full use of all the layered features of the image to be processed.

[0109] Optionally, in some embodiments, the hybrid computation is performed before the residual dense block computation.

[0110] By performing hybrid computation before the residual dense block computation, the third downsampled image is preprocessed, which is beneficial for the finer processing of the subsequent residual dense block computation.

[0111] Step 203: Input each downsampled image into the corresponding scale calculation model and output the scale calculation image corresponding to the downsampled image.

[0112] The scale calculation model is used to perform at least one convolution calculation on the downsampled image; the sampling factor and the number of convolution calculations performed by the scale calculation model are positively correlated.

[0113] The implementation method of this step is similar to that of "Step 103" above, and will not be repeated here.

[0114] Optionally, in some embodiments, "step 203" specifically includes the following steps (step 2031, step 2032, step 2033):

[0115] Step 2031: Input the first downsampled image into the corresponding scale calculation model to perform a single-layer convolutional processing calculation, and output the first scale calculation image.

[0116] It should be noted that convolution processing in a single convolutional layer can improve the visual quality of the final output target image.

[0117] Outputting the image at the first scale facilitates subsequent refined image enhancement processing.

[0118] Step 2032: Input the second downsampled image into the corresponding scale calculation model to perform two convolutional processing calculations with multiple convolutional layers, and output the second scale calculation image.

[0119] It should be noted that convolution processing computation, by performing a convolution operation between the image and a convolution kernel, achieves operations such as sharpening filtering and Gaussian filtering on the image, thereby improving the visual quality of the image. Sharpening filtering enhances the edges and details of the image through sharpening coefficients in the convolution kernel, thus achieving image sharpening; Gaussian filtering smooths the image through a Gaussian function in the convolution kernel, thus achieving image denoising. Therefore, convolution processing computation with multiple convolutional layers can improve the visual quality of the final output target image. Outputting the image at a second scale is beneficial for subsequent further refined image enhancement processing.

[0120] Step 2033: Input the third downsampled image into the corresponding scale calculation model to perform one hybrid calculation and one residual dense block calculation, and output the third scale calculation image.

[0121] The hybrid computation includes one channel shuffling convolution processing computation and one group convolution processing computation; the residual dense block computation includes multiple convolutional layer computations for nonlinear correction.

[0122] It should be noted that since the hybrid computation includes one channel shuffling convolution processing calculation and one group convolution processing calculation, it can enhance the edge and detail of the image and reduce image noise. The residual dense block computation includes multiple convolutional layer calculations for nonlinear correction, that is, multi-layer convolution processing calculations, which can capture more image features. Therefore, the hybrid computation and residual dense block computation can improve the visual quality of the final output target image.

[0123] Outputting the image at the third scale facilitates further refined image enhancement processing.

[0124] By executing steps 2031, 2032, and 2033 in the embodiments of this application, a first-scale calculated image, a second-scale calculated image, and a third-scale calculated image with higher visual quality can be output, which is beneficial for subsequent further refined image enhancement processing of the first-scale calculated image, the second-scale calculated image, and the third-scale calculated image.

[0125] Step 204: Fuse all the scale-calculated images to obtain the fused image, perform convolution processing, and output the target image after upsampling.

[0126] The implementation method of this step is similar to that of "Step 104" above, and will not be repeated here.

[0127] Optionally, in some embodiments, "step 204" specifically includes the following steps (step 2041, step 2042, step 2043):

[0128] Step 2041: Fuse the third-scale calculated image and the second-scale calculated image to obtain a first fused image.

[0129] It should be noted that fusing the third-scale computed image and the second-scale computed image can improve the visual quality of the final output target image.

[0130] Fusion refers to combining the features of two images with the same resolution. During the fusion process, the image resolution does not change, but the number of channels increases. The more channels there are, the more image features can be extracted, and the higher the visual quality of the final output target image. For example, image feature X1 = (length pixels, width pixels, channel 1), image feature X2 = (length pixels, width pixels, channel 2), and the fused image feature X3 = (length pixels, width pixels, channel 1 + channel 2).

[0131] Optionally, in some embodiments, "step 2041" specifically includes the following steps (step 2041a, step 2041b, step 2041c, step 2041d):

[0132] Step 2041a: Fuse the image obtained by mixing the third-scale calculated image with the third downsampled image to obtain the third fused image.

[0133] Specifically, in some embodiments, the third downsampled image is first subjected to a mixing calculation, which includes a channel shuffling convolution processing calculation and a group convolution processing calculation. The resulting image is then fused with the third-scale calculated image. By fusing the third-scale calculated image and the third downsampled image obtained through the mixing calculation, the number of channels of the image features is increased, thereby improving the amount of information extracted from the image features and thus improving the visual quality of the final output target image.

[0134] Since the time required for convolution processing on high-resolution images is greater than that for low-resolution images, and the third downsampled image has a lower resolution than the first and second downsampled images, a hybrid computation is adopted. This involves performing one channel shuffling convolution and one group convolution on the third downsampled image to account for time costs and distribute as many convolution processing computations as possible to the lower-resolution third downsampled image.

[0135] Step 2041b: Fuse the second scale-calculated image with the second downsampled image to obtain a fourth fused image.

[0136] It should be noted that fusing the second-scale calculated image and the second downsampled image increases the number of channels for image features, thereby increasing the amount of information extracted from the image features and thus improving the visual quality of the final output target image.

[0137] Step 2041c: Upsample the third fused image to obtain the first upsampled image.

[0138] Specifically, in some embodiments, the upsampling factor for the third fused image can be 4, that is, multiplying the resolution of the third fused image by 4 to obtain an image with 4 times the resolution of the third fused image as the first upsampled image. For example, if the resolution of the third fused image is 20×10, after an upsampling operation of 4, that is, 20 multiplied by 4 (getting 80) and 10 multiplied by 4 (getting 40), a first upsampled image with a resolution of 80×40 is obtained.

[0139] Step 2041c can be used to upsample the third fused image to obtain the first upsampled image, so that the first upsampled image and the fourth fused image have the same resolution, which facilitates further fusion processing of the first upsampled image and the fourth fused image.

[0140] Step 2041d: The image obtained by performing convolution processing on the first upsampled image through a single convolutional layer is fused with the image obtained by performing convolution processing on the fourth fused image through a single convolutional layer to obtain the first fused image.

[0141] Specifically, in some embodiments, the first upsampled image is first subjected to a single-layer convolutional processing calculation to obtain image 1; the fourth fused image is then subjected to a single-layer convolutional processing calculation to obtain image 2; and image 1 and image 2 are then fused to obtain the first fused image. Fusing image 1 and image 2 can improve the visual quality of the final output target image.

[0142] By performing steps 2041a, 2041b, 2041c, and 2041d in the embodiments of this application, further refined image enhancement processing is performed on the third-scale calculated image and the second-scale calculated image. The resulting first fused image will have higher visual quality, which is beneficial for further image enhancement processing.

[0143] Step 2042: Fuse the first fused image and the first scale-calculated image to obtain the second fused image.

[0144] It should be noted that fusing the first fused image and the first scale calculated image can improve the visual quality of the final output target image.

[0145] Optionally, in some embodiments, "step 2042" specifically includes the following steps (step 2042a, step 2042b):

[0146] Step 2042a: The image obtained after the first fused image has undergone one mixing calculation and one convolutional processing calculation of multiple convolutional layers is upsampled to obtain a second upsampled image.

[0147] Optionally, in some embodiments, the upsampling factor of the image obtained by performing an upsampling operation on the first fused image after one mixing calculation and one convolutional processing layer can be 4. That is, the resolution of the image obtained by performing an upsampling operation on the first fused image after one mixing calculation and one convolutional processing layer can be multiplied by 4 to obtain an image with 4 times the resolution of the image obtained by performing an upsampling operation on the first fused image after one mixing calculation and one convolutional processing layer. For example, if the resolution of the image obtained by performing an upsampling operation on the first fused image after one mixing calculation and one convolutional processing layer is 80×40, then after performing an upsampling operation with a factor of 4, that is, multiplying 80 by 4 (to obtain 320) and 40 by 4 (to obtain 160), a second upsampling image with a resolution of 320×160 is obtained.

[0148] Specifically, in some embodiments, the first fused image is subjected to a mixing calculation and a convolutional processing calculation of multiple convolutional layers to obtain image 3; then image 3 is upsampled to obtain a second upsampled image.

[0149] Compared to convolutional processing with a single convolutional layer, convolutional processing with multiple convolutional layers can capture more image features, thus improving the visual quality of the final output target image.

[0150] Step 2042a can be used to obtain a second upsampled image, which has the same resolution as the first scale calculated image, thus facilitating further fusion processing of the second upsampled image and the first scale calculated image.

[0151] Step 2042b: Fuse the second upsampled image with the first scale-calculated image to obtain the second fused image.

[0152] It should be noted that fusing the second upsampled image and the first scale calculated image can improve the visual quality of the final output target image.

[0153] By performing steps 2042a and 2042b in the embodiments of this application, further refined image enhancement processing can be performed on the first fused image and the first scale calculated image to obtain the second fused image, which is beneficial for further image enhancement processing.

[0154] It should be noted that for the same convolutional processing calculation, the time required for convolutional processing calculation on high-resolution images is greater than that required for convolutional processing calculation on low-resolution images. Furthermore, for the same image, a single-layer convolutional processing calculation with multiple convolutional layers takes more time than a single-layer convolutional processing calculation. Therefore, in this embodiment, as many convolutional processing calculations as possible are allocated to low-resolution images to reduce the overall process time. For example, if the first fused image has a lower resolution than the second fused image, then a single-layer convolutional processing calculation is allocated to the first fused image for processing, while a single-layer convolutional processing calculation is allocated to the second fused image for processing.

[0155] Step 2043: Upsample the image obtained by processing the second fused image through a single convolutional layer to obtain the target image.

[0156] Optionally, in some embodiments, the upsampling factor of the image obtained after the second fused image undergoes a single-layer convolutional processing is 2. That is, the resolution of the image obtained after the second fused image undergoes a single-layer convolutional processing is multiplied by 2, resulting in an image with twice the resolution of the image obtained after the second fused image undergoes a single-layer convolutional processing. For example, if the resolution of the image obtained after the second fused image undergoes a single-layer convolutional processing is 320×160, after an upsampling operation of 2 (320 multiplied by 2 to get 640) and 160 multiplied by 2 (getting 320), a target image with a resolution of 640×320 is obtained.

[0157] Specifically, in some embodiments, the second fused image is first processed by convolution of a single convolutional layer to obtain image 4; then image 4 is upsampled to obtain the target image.

[0158] Step 2043 can be used to obtain a target image that has the same resolution as the original low-light image and has higher visual quality compared to the original low-light image.

[0159] By executing steps 2041, 2042, and 2043 in the embodiments of this application, further image enhancement refinement processing is performed on all scale-calculated images. The target image obtained by fusing all scale-calculated images has the same resolution as the original low-light image, and the target image has higher visual quality than the original low-light image, thereby achieving low-light image enhancement.

[0160] Optionally, in some embodiments, the upsampling factor of the image obtained by upsampling the second fused image after processing it through a single convolutional layer is equal to the sampling factor of the first downsampling; the product of the upsampling factor of the image obtained by upsampling the second fused image after processing it through a single convolutional layer and the upsampling factor of the image obtained by upsampling the first fused image after processing it through a mixing calculation and a multi-layer convolutional layer is equal to the sampling factor of the second downsampling; the product of the upsampling factor of the image obtained by upsampling the second fused image after processing it through a single convolutional layer, the upsampling factor of the image obtained by upsampling the first fused image after processing it through a mixing calculation and a multi-layer convolutional layer, and the upsampling factor of the third fused image is equal to the sampling factor of the third downsampling, to ensure that the final target image has the same resolution as the original low-light image.

[0161] Optionally, in some embodiments, before performing a residual dense block computation on the image obtained after the third downsampled image undergoes a mixing computation, the default number of convolutional layer computations in the residual dense block computation is adjusted to a target number; the target number is less than the default number.

[0162] It should be noted that in residual dense block computation, excessive use of convolutional layers with nonlinear correction will reduce the accuracy of image enhancement processing, while insufficient use of convolutional layers with nonlinear correction will reduce the visual quality of the image. In the embodiments of this application, compared to the default number of convolutional layers with nonlinear correction, the target number of convolutional layers with nonlinear correction reduces the side effects of negative clipping while maintaining sufficient nonlinear correction, which is beneficial to improving the visual quality of the image.

[0163] Optionally, in some embodiments, the total number of operations for performing a first downsampling operation on the pre-enlarged image and for inputting the first downsampled image into the corresponding scale calculation model for convolution processing is calculated according to the following mathematical expression:

[0164]

[0165] Where M1 represents the total number of operations for performing the first downsampling operation on the pre-enlarged image and inputting the first downsampled image into the corresponding scale calculation model for convolution processing; the resolution of the original low-light image is H×W (H is the number of pixels in length, and W is the number of pixels in width); r1 is the downsampling factor; and the size of the convolution kernel is... (i.e., k1×k1), where I1 is the number of input channels, O1 is the number of output channels, and N1 is the number of convolutional layers.

[0166] Optionally, in some embodiments, the total number of operations for performing a second downsampling operation on the pre-enlarged image and inputting the second downsampling image into the corresponding scale calculation model for convolution processing is calculated according to the following mathematical expression:

[0167]

[0168] Where M2 represents the total number of operations for performing the second downsampling operation on the pre-enlarged image and inputting the second downsampled image into the corresponding scale calculation model for convolution processing; the resolution of the original low-light image is H×W (H is the number of pixels in length, and W is the number of pixels in width); r2 is the downsampling factor; and the size of the convolution kernel is... (i.e., k2×k2), where I2 is the number of input channels, O2 is the number of output channels, and N2 is the number of convolutional layers.

[0169] Optionally, in some embodiments, the total number of operations for performing a third downsampling operation on the pre-enlarged image and inputting the third downsampling image into the corresponding scale calculation model for convolution processing is calculated according to the following mathematical expression:

[0170]

[0171] Where M3 represents the total number of operations for performing the third downsampling operation on the pre-enlarged image and inputting the third downsampled image into the corresponding scale calculation model for convolution processing; the resolution of the original low-light image is H×W (H is the number of pixels in length, and W is the number of pixels in width); r3 is the downsampling factor; and the size of the convolution kernel is... (i.e., k3×k3), I3 is the channel width of the grouped convolution, N3 is the number of convolutional layers, S3 is the growth factor calculated for each convolutional process in the residual dense block, l∈[1, η], and η is the number of convolutional layers calculated for each convolutional process in the residual dense block.

[0172] Specifically, in some embodiments, for mathematical expressions:

[0173]

[0174] Set r1 to 2, the size of the convolution kernel (i.e., k1×k1) is 7 2 If I1 is 4, O1 is 12, and N1 is 1, then:

[0175]

[0176] For mathematical expressions:

[0177]

[0178] Set r2 to 8, the size of the convolution kernel (i.e., k2×k2) is 3 2 If I2 is 64, O2 is 64, and N2 is 1, then:

[0179]

[0180] For mathematical expressions:

[0181]

[0182] Set r3 to 32, the size of the convolution kernel (i.e., k3×k3) is 3, I3 is 64, N3 is 3, S3 is 32, l∈[1, η], η is 5, then we have:

[0183]

[0184] In the embodiments of this application, the total number of operations for calculating the total number of operations for performing a first downsampling operation on the pre-enlarged image and inputting the first downsampling image into the corresponding scale calculation model for convolution processing, the total number of operations for performing a second downsampling operation on the pre-enlarged image and inputting the second downsampling image into the corresponding scale calculation model for convolution processing, and the total number of operations for performing a third downsampling operation on the pre-enlarged image and inputting the third downsampling image into the corresponding scale calculation model for convolution processing are all within the same operand range (570×H×W to 600×H×W), and can be performed in parallel.

[0185] Optionally, in some embodiments, reference is made to Figure 3The image enhancement process corresponding to the image enhancement method can include: X1, inputting the original low-light image; X2, 2x downsampling; X3, 8x downsampling; X4, 32x downsampling; X5, single-layer convolutional layer calculation; X6, multi-layer convolutional layer calculation; X7, multi-layer convolutional layer calculation; X8, fusion; X9, single-layer convolutional layer calculation; X10, hybrid calculation; X11, residual dense block calculation; X12, fusion; X13, 4x upsampling; X14, single-layer convolutional layer calculation; X15, fusion; X16, hybrid calculation; X17, multi-layer convolutional layer calculation; X18, 4x upsampling; X19, fusion; X 20. Single-layer convolutional calculation; X21. 2x upsampling; X22. Output target image, wherein the hybrid calculation includes one channel shuffling convolutional processing calculation and one group convolutional processing calculation; the residual dense block calculation includes multiple convolutional layer calculations for nonlinear correction. Through the embodiments of this application, the target image can have higher image quality than the original low-light image, realizing the enhancement of low-light images. It eliminates the need for staff to manually enhance low-light images using image editing software, improves work efficiency, and solves the problem of low work efficiency caused by staff manually enhancing low-light images using image editing software.

[0186] In summary, the embodiments of this application can achieve the following: First, the pixel intensity of the acquired original low-light image is magnified according to a pre-magnification factor to obtain a pre-magnified image. Then, multiple downsampling operations are performed on the pre-magnified image to obtain a downsampled image corresponding to each downsampling operation. The sampling factor for each downsampling operation decreases sequentially. Each downsampled image is input into a corresponding scale calculation model, which outputs a scale-calculated image corresponding to the downsampled image. The scale calculation model performs at least one convolution calculation on the downsampled image. The sampling factor and the number of convolution calculations in the scale calculation model are positively correlated. All the scale-calculated images are fused to obtain a fused image, which is then convolved and upsampled to output a target image. The target image has higher image quality than the original low-light image, achieving low-light image enhancement. This eliminates the need for manual enhancement using image editing software, improving work efficiency and solving the problem of low work efficiency caused by manual enhancement using image editing software.

[0187] Figure 4 This is a block diagram of an image enhancement device provided in an embodiment of this application, such as... Figure 4 As shown, the device 300 includes:

[0188] The pre-magnification module 301 is used to amplify the pixel intensity of the acquired original low-light image according to the pre-magnification factor to obtain a pre-magnified image.

[0189] The downsampling module 302 is used to perform multiple downsampling operations on the pre-magnified image to obtain a downsampled image corresponding to each downsampling operation; the sampling factor of each of the multiple downsampling operations decreases sequentially.

[0190] The scaling module 303 is used to input each downsampled image into the corresponding scaling model and output a scaling image corresponding to the downsampled image; the scaling model is used to perform at least one convolution processing calculation on the downsampled image; the sampling factor and the number of convolution processing calculations of the scaling model are positively correlated.

[0191] The fusion module 304 is used to fuse all the scale-calculated images, and after obtaining the fused image, it performs convolution processing and outputs the target image after upsampling.

[0192] Optionally, the downsampling module 302 specifically includes:

[0193] The first downsampling submodule is used to perform a first downsampling operation on the pre-enlarged image to obtain a first downsampled image;

[0194] The second downsampling submodule is used to perform a second downsampling operation on the pre-enlarged image to obtain a second downsampled image;

[0195] The third downsampling submodule is used to perform a third downsampling operation on the pre-enlarged image to obtain a third downsampled image.

[0196] The sampling factor of the first downsampling operation is less than the sampling factor of the second downsampling operation; the sampling factor of the second downsampling operation is less than the sampling factor of the third downsampling operation.

[0197] Optionally, the scale calculation model corresponding to the first downsampled image includes one convolutional processing calculation to output a first-scale calculated image; the scale calculation model corresponding to the second downsampled image includes two convolutional processing calculations to output a high-quality second-scale calculated image; and the scale calculation model corresponding to the third downsampled image includes one hybrid calculation and one residual dense block calculation to output the highest-quality third-scale calculated image among the three.

[0198] Optionally, the scale calculation module 303 specifically includes:

[0199] The first scale calculation submodule is used to input the first downsampled image into the corresponding scale calculation model to perform a single-layer convolutional processing calculation and output the first scale calculated image.

[0200] The second scale calculation submodule is used to input the second downsampled image into the corresponding scale calculation model to perform two multi-layer convolution processing calculations and output the second scale calculated image.

[0201] The third-scale calculation submodule is used to input the third downsampled image into the corresponding scale calculation model to perform a hybrid calculation and a residual dense block calculation, and output the third-scale calculated image; the hybrid calculation includes a channel shuffling convolution processing calculation and a grouped convolution processing calculation; the residual dense block calculation includes multiple convolutional layer calculations for nonlinear correction.

[0202] Optionally, the fusion module 304 specifically includes:

[0203] The first fusion submodule is used to fuse the third-scale calculated image and the second-scale calculated image to obtain a first fused image;

[0204] The second fusion submodule is used to fuse the first fused image and the first scale-calculated image to obtain the second fused image;

[0205] The third fusion submodule is used to upsample the image obtained by processing the second fused image through a single convolutional layer to obtain the target image.

[0206] Optionally, the first fusion submodule specifically includes:

[0207] The first fusion module is used to fuse the image obtained by mixing the third-scale calculated image and the third downsampled image to obtain the third fused image;

[0208] The second fusion module is used to fuse the second scale-calculated image with the second downsampled image to obtain a fourth fused image;

[0209] The third fusion module is used to perform an upsampling operation on the third fused image to obtain a first upsampled image;

[0210] The fourth fusion module is used to fuse the image obtained by processing the first upsampled image through a single convolutional layer with the image obtained by processing the fourth fusion image through a single convolutional layer to obtain the first fusion image.

[0211] Optionally, the second fusion submodule specifically includes:

[0212] The fifth fusion module is used to perform an upsampling operation on the image obtained after the first fused image has undergone a mixing calculation and a convolutional processing calculation of multiple convolutional layers to obtain a second upsampled image;

[0213] The sixth fusion module is used to fuse the second upsampled image with the first scale-calculated image to obtain the second fused image.

[0214] Optionally, before performing a residual dense block computation on the image obtained after the third downsampled image undergoes a mixing computation, the default number of convolutional layer computations in the residual dense block computation is adjusted to a target number; the target number is less than the default number.

[0215] Optionally, the device 300 further includes:

[0216] The acquisition module is used to acquire the pixel intensity of each pixel in the original low-light image before obtaining the pre-magnified image by magnifying the pixel intensity of the acquired original low-light image according to the pre-magnification factor.

[0217] The weight calculation module is used to obtain the weight of each pixel in the original low-light image based on the pixel intensity of each pixel in the original low-light image.

[0218] The weighted average calculation module is used to obtain a weighted average based on the weights of all pixels and the pixel intensity of all pixels in the original low-light image.

[0219] The pre-amplification factor calculation module is used to obtain the pre-amplification factor based on the weighted average value and the preset amplification parameters.

[0220] Optionally, the weight of the pixel is negatively correlated with the pixel intensity; the pre-amplification coefficient is negatively correlated with the weighted average value.

[0221] The image enhancement device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0222] The image enhancement device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0223] The image enhancement device provided in this application embodiment can achieve Figure 1 , Figure 2 The various processes implemented by the image enhancement device in the method embodiment will not be described again here to avoid repetition.

[0224] In summary, in this embodiment, a pre-magnified image is obtained by amplifying the pixel intensity of the original low-light image according to a pre-magnification factor; multiple downsampling operations are performed on the pre-magnified image to obtain a downsampled image corresponding to each downsampling operation; the sampling factor corresponding to each downsampling operation decreases sequentially; each downsampled image is input into a corresponding scale calculation model to output a scale calculation image corresponding to the downsampled image; the scale calculation model is used to perform at least one convolution calculation on the downsampled image; the sampling factor and the number of convolution calculations in the scale calculation model are positively correlated; all the scale calculation images are fused to obtain a fused image, which is then subjected to convolution processing and upsampling to output a target image. The target image has higher image quality than the original low-light image, thus achieving low-light image enhancement. This eliminates the need for manual enhancement of low-light images using image editing software, improving work efficiency and solving the problem of low work efficiency caused by manual enhancement of low-light images using image editing software.

[0225] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described image enhancement method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0226] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0227] Figure 5 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0228] The electronic device 400 includes, but is not limited to, components such as: radio frequency unit 401, network module 402, audio output unit 403, input unit 404, sensor 405, display unit 406, user input unit 407, interface unit 408, memory 409, and processor 410.

[0229] Those skilled in the art will understand that the electronic device 400 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 410 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0230] The processor 410 is used to amplify the pixel intensity of the acquired original low-light image according to the pre-magnification factor to obtain a pre-magnified image.

[0231] The pre-enlarged image is subjected to multiple downsampling operations to obtain a downsampled image corresponding to each downsampling operation; the sampling factor corresponding to each of the multiple downsampling operations decreases sequentially.

[0232] Each downsampled image is input into a corresponding scale calculation model, and a scale-calculated image corresponding to the downsampled image is output; the scale calculation model is used to perform at least one convolution operation on the downsampled image; the sampling factor and the number of convolution operations performed by the scale calculation model are positively correlated.

[0233] All the scale-calculated images are fused together to obtain a fused image, which is then subjected to convolution processing and upsampling to output the target image.

[0234] In this embodiment, the pixel intensity of the acquired original low-light image is magnified according to a pre-magnification factor to obtain a pre-magnified image. Multiple downsampling operations are performed on the pre-magnified image to obtain a downsampled image corresponding to each downsampling operation. The sampling factor for each downsampling operation decreases sequentially. Each downsampled image is input into a corresponding scale calculation model, which outputs a scale-calculated image corresponding to the downsampled image. The scale calculation model performs at least one convolution operation on the downsampled image. The sampling factor and the number of convolution operations in the scale calculation model are positively correlated. All the scale-calculated images are fused to obtain a fused image, which is then convolved and upsampled to output a target image. The target image has higher image quality than the original low-light image, achieving low-light image enhancement. This eliminates the need for manual enhancement using image editing software, improving work efficiency and solving the problem of low efficiency caused by manual enhancement of low-light images using image editing software.

[0235] Optionally, the processor 410 is further configured to perform multiple downsampling operations on the pre-enlarged image to obtain a downsampled image corresponding to each downsampling operation, including: performing a first downsampling operation on the pre-enlarged image to obtain a first downsampled image; performing a second downsampling operation on the pre-enlarged image to obtain a second downsampled image; performing a third downsampling operation on the pre-enlarged image to obtain a third downsampled image; wherein the sampling factor of the first downsampling operation is less than the sampling factor of the second downsampling operation; and the sampling factor of the second downsampling operation is less than the sampling factor of the third downsampling operation.

[0236] Optionally, the processor 410 is further configured to: have the scale calculation model corresponding to the first downsampled image include one convolutional processing calculation to output a first scale calculation image; have the scale calculation model corresponding to the second downsampled image include two convolutional processing calculations to output a high-quality second scale calculation image; and have the scale calculation model corresponding to the third downsampled image include one hybrid calculation and one residual dense block calculation to output the highest quality third scale calculation image among the three.

[0237] Optionally, the processor 410 is further configured to input each of the downsampled images into a corresponding scale calculation model and output a scale calculation image corresponding to the downsampled image, including: inputting the first downsampled image into the corresponding scale calculation model to perform a single-layer convolutional processing calculation and outputting a first scale calculation image; inputting the second downsampled image into the corresponding scale calculation model to perform two multi-layer convolutional processing calculations and outputting a second scale calculation image; inputting the third downsampled image into the corresponding scale calculation model to perform a hybrid calculation and a residual dense block calculation and outputting a third scale calculation image; the hybrid calculation includes a channel shuffling convolutional processing calculation and a grouped convolutional processing calculation; the residual dense block calculation includes multiple convolutional layer calculations for nonlinear correction.

[0238] Optionally, the processor 410 is further configured to fuse all the scale-calculated images, obtain a fused image, perform convolution processing, and output a target image after upsampling, including: fusing the third scale-calculated image and the second scale-calculated image to obtain a first fused image; fusing the first fused image and the first scale-calculated image to obtain a second fused image; and upsampling the image obtained after the second fused image undergoes convolution processing through a single convolutional layer to obtain the target image.

[0239] Optionally, the processor 410 is further configured to fuse the third-scale calculated image and the second-scale calculated image to obtain a first fused image, comprising: fusing the image obtained by mixing the third-scale calculated image and the third downsampled image to obtain a third fused image; fusing the second-scale calculated image and the second downsampled image to obtain a fourth fused image; performing an upsampling operation on the third fused image to obtain a first upsampled image; and fusing the image obtained by performing a single-layer convolutional processing calculation on the first upsampled image with the image obtained by performing a single-layer convolutional processing calculation on the fourth fused image to obtain the first fused image.

[0240] Optionally, the processor 410 is further configured to fuse the first fused image and the first scale-calculated image to obtain a second fused image, comprising: performing an upsampling operation on the image obtained after the first fused image has undergone a mixing calculation and a convolution processing calculation of multiple convolutional layers to obtain a second upsampled image; and fusing the second upsampled image with the first scale-calculated image to obtain the second fused image.

[0241] Optionally, the processor 410 is further configured to adjust the default number of convolutional layer calculations in the residual dense block calculation to a target number before performing a residual dense block calculation on the image obtained after the third downsampled image has undergone a mixing calculation; the target number is less than the default number.

[0242] Optionally, the processor 410 is further configured to, before amplifying the pixel intensity of the acquired original low-light image according to the pre-magnification factor to obtain the pre-magnified image, further include: acquiring the pixel intensity of each pixel of the original low-light image; acquiring the weight of each pixel of the original low-light image based on the pixel intensity of each pixel of the original low-light image; acquiring a weighted average based on the weights and pixel intensities of all pixels of the original low-light image; and acquiring the pre-magnification factor based on the weighted average and a preset magnification parameter.

[0243] Optionally, the processor 410 is further configured such that the weight of the pixel is negatively correlated with the pixel intensity of the pixel; and the pre-amplification coefficient is negatively correlated with the weighted average value.

[0244] In this embodiment, the pixel intensity of the acquired original low-light image is magnified according to a pre-magnification factor to obtain a pre-magnified image. Multiple downsampling operations are performed on the pre-magnified image to obtain a downsampled image corresponding to each downsampling operation. The sampling factor for each downsampling operation decreases sequentially. Each downsampled image is input into a corresponding scale calculation model, which outputs a scale-calculated image corresponding to the downsampled image. The scale calculation model performs at least one convolution operation on the downsampled image. The sampling factor and the number of convolution operations in the scale calculation model are positively correlated. All the scale-calculated images are fused to obtain a fused image, which is then convolved and upsampled to output a target image. The target image has higher image quality than the original low-light image, achieving low-light image enhancement. This eliminates the need for manual enhancement using image editing software, improving work efficiency and solving the problem of low efficiency caused by manual enhancement of low-light images using image editing software.

[0245] It should be understood that, in this embodiment, the input unit 404 may include a graphics processing unit (GPU) 4041 and a microphone 4042. The GPU 4041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 406 may include a display panel 4061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 407 includes at least one of a touch panel 4071 and other input devices 4072. The touch panel 4071 is also called a touch screen. The touch panel 4071 may include a touch detection device and a touch controller. Other input devices 4072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0246] The memory 409 can be used to store software programs and various data. The memory 409 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 409 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 409 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0247] Processor 410 may include one or more processing units; optionally, processor 410 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 410.

[0248] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image enhancement method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0249] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0250] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image enhancement method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0251] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0252] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image enhancement method, characterized in that, The method includes: The pixel intensity of the original low-light image is magnified according to the pre-magnification factor to obtain a pre-magnified image; the pre-magnification factor is obtained based on the weighted average of the pixel intensity of all pixels in the original low-light image and a preset magnification parameter. The pre-enlarged image is subjected to multiple downsampling operations to obtain a downsampled image corresponding to each downsampling operation; the sampling factor corresponding to each of the multiple downsampling operations decreases sequentially. Each downsampled image is input into a corresponding scale calculation model, and a scale-calculated image corresponding to the downsampled image is output; the scale calculation model is used to perform at least one convolution operation on the downsampled image; the sampling factor and the number of convolution operations performed by the scale calculation model are positively correlated. The third-scale calculated image, obtained by performing a mixing calculation and a residual dense block calculation on the third downsampled image, is fused with the second-scale calculated image, obtained by performing two convolution processes on the second downsampled image, to obtain the first fused image. The first scale calculated image obtained by convolution processing the first downsampled image is fused with the first fused image to obtain the second fused image; The second fused image is convolved and then upsampled to output the target image.

2. The method according to claim 1, characterized in that, The step of performing multiple downsampling operations on the pre-enlarged image to obtain a downsampled image corresponding to each downsampling operation includes: Perform a first downsampling operation on the pre-enlarged image to obtain the first downsampled image; Perform a second downsampling operation on the pre-enlarged image to obtain the second downsampled image; A third downsampling operation is performed on the pre-enlarged image to obtain the third downsampled image; The sampling factor of the first downsampling operation is less than the sampling factor of the second downsampling operation; the sampling factor of the second downsampling operation is less than the sampling factor of the third downsampling operation.

3. The method according to claim 1, characterized in that, The hybrid computation includes one channel shuffling convolution processing computation and one group convolution processing computation; the residual dense block computation includes multiple convolutional layer computations for nonlinear correction.

4. The method according to claim 1, characterized in that, The step of convolving the second fused image and upsampling it to output the target image includes: The target image is obtained by upsampling the second fused image after it has undergone convolution processing through a single convolutional layer.

5. The method according to claim 1, characterized in that, The first fused image is obtained by fusing the third-scale calculated image (obtained by performing a mixing calculation and a residual dense block calculation on the third downsampled image) with the second-scale calculated image (obtained by performing two convolution processes on the second downsampled image), including: The image obtained by mixing the third-scale calculated image and the third downsampled image is fused to obtain the third fused image; The second scale-calculated image and the second downsampled image are fused to obtain a fourth fused image; The third fused image is upsampled to obtain the first upsampled image; The image obtained by performing convolution processing on the first upsampled image through a single convolutional layer is fused with the image obtained by performing convolution processing on the fourth fused image through a single convolutional layer to obtain the first fused image.

6. The method according to claim 1, characterized in that, The step of fusing the first scale calculated image obtained by performing a convolution process on the first downsampled image with the first fused image to obtain the second fused image includes: The image obtained by performing a mixing calculation and a convolutional processing of multiple convolutional layers on the first fused image is upsampled to obtain the second upsampled image. The second upsampled image is fused with the first scale-calculated image to obtain the second fused image.

7. The method according to claim 1, characterized in that, Before performing a residual dense block computation on the image obtained after the third downsampled image undergoes a mixing computation, the default number of convolutional layer computations in the residual dense block computation is adjusted to the target number, so that the nonlinear correction convolutional layer computation reduces the side effects of negative clipping; the target number is less than the default number.

8. The method according to claim 1, characterized in that, Before amplifying the pixel intensity of the acquired original low-light image according to a pre-magnification factor to obtain a pre-magnified image, the method further includes: Obtain the pixel intensity of each pixel in the original low-light image; The weight of each pixel in the original low-light image is obtained based on the pixel intensity of each pixel; the weight of the pixel is negatively correlated with the pixel intensity. The weighted average value is obtained based on the weights of all pixels and the pixel intensity of all pixels in the original low-light image. The pre-amplification coefficient is obtained based on the weighted average value and the preset amplification parameter; the pre-amplification coefficient is negatively correlated with the weighted average value.

9. An image enhancement device, characterized in that, The device includes: The pre-magnification module is used to amplify the pixel intensity of the acquired original low-light image according to the pre-magnification factor to obtain a pre-magnified image; the pre-magnification factor is obtained based on the weighted average of the pixel intensity of all pixels in the original low-light image and a preset amplification parameter. The downsampling module is used to perform multiple downsampling operations on the pre-enlarged image to obtain a downsampled image corresponding to each downsampling operation; the sampling factor of each of the multiple downsampling operations decreases sequentially. The scaling module is used to input each downsampled image into the corresponding scaling model and output a scaled image corresponding to the downsampled image; the scaling model is used to perform at least one convolution processing calculation on the downsampled image; the sampling factor and the number of convolution processing calculations of the scaling model are positively correlated. The first fusion submodule is used to fuse the third-scale calculated image obtained by the third downsampled image through one mixing calculation and one residual dense block calculation, and the second-scale calculated image obtained by the second downsampled image through two convolution processing to obtain the first fused image; The second fusion submodule is used to fuse the first scale calculated image obtained by convolution processing the first downsampled image with the first fusion image to obtain the second fusion image. The third fusion submodule is used to perform convolution processing on the second fused image and output the target image after upsampling.

10. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the image enhancement method as described in any one of claims 1 to 8.

11. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the image enhancement method as described in any one of claims 1 to 8.

Images