Low-light image enhancement method, related equipment and readable storage medium

Abstract

This application discloses a low-light image enhancement method, related equipment, and readable storage medium. In this solution, a decomposition model is pre-trained based on a training image pair consisting of an incompletely paired training low-light image and a training normal-light image of the same scene. After obtaining the low-light image to be enhanced, the low-light image is input into the decomposition model to obtain an illumination map. Finally, the enhanced image is obtained using the illumination map. Because the decomposition model is trained based on incompletely paired image pairs, such image pairs are easy to obtain in large quantities, resulting in better decomposition model training results and more accurate illumination maps. Therefore, using the illumination map to enhance low-light images can achieve better enhancement results.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to a low-light image enhancement method, related equipment, and a readable storage medium. Background Art

[0002] Low-light image enhancement is to address problems such as low brightness, low contrast, noise, and artifacts in images with insufficient lighting, thereby improving the visual quality of the image.

[0003] In existing technology, low-light images can be decomposed based on a trained decomposition model to obtain an illumination map and a reflectance map. The illumination map and reflectance map are then enhanced separately, and the enhanced illumination map and reflectance map are reconstructed to obtain the enhanced image. In this technology, the decomposition model is trained based on perfectly paired low-light and normal-light images. However, perfectly paired low-light and normal-light images are rare and difficult to obtain, resulting in poor performance of the trained decomposition model and, in turn, poor low-light image enhancement.

[0004] Therefore, how to improve the enhancement effect of low-light images has become a technical problem that needs to be solved urgently by those skilled in the art. Summary of the Invention

[0005] In view of the above problems, this application proposes a low-light image enhancement method, related equipment and readable storage medium. The specific solution is as follows:

[0006] A low-light image enhancement method, the method comprising:

[0007] Acquire a low-light image to be enhanced;

[0008] Inputting the low-light image into a decomposition model, wherein the decomposition model decomposes the low-light image to obtain a lighting map; the decomposition model is trained based on a training image pair, wherein the training image pair includes a training low-light image and a training normal-light image, wherein the training low-light image and the training normal-light image are two incompletely paired images of the same scene;

[0009] Based on the illumination map, an enhanced image is obtained.

[0010] Optionally, the training loss of the decomposition model includes:

[0011] The consistency loss between the feature map of the reflectance map of the training low-light image output by the decomposition model and the feature map of the training low-light image processed by histogram equalization.

[0012] Optionally, the training loss also includes:

[0013] Any one or more of the discriminator loss between the enhanced image obtained based on the illumination map of the training low-light image output by the decomposition model and the training normal-light image, the consistency loss between the reflectance map of the training low-light image output by the decomposition model and the reflectance map of the training normal-light image output by the decomposition model, and the reconstruction loss of the illumination map of the training low-light image output by the decomposition model and the reflectance map of the training low-light image output by the decomposition model.

[0014] Optionally, obtaining an enhanced image based on the illumination map includes:

[0015] determining an exposure map for the low-light image based on the light map;

[0016] determining a map of regions with good illumination in the low-light image and a map of regions with good illumination in the exposure image;

[0017] The map of the region with good illumination in the low-light image and the map of the region with good illumination in the exposure image are reconstructed to obtain an enhanced image.

[0018] Optionally, determining an exposure map of the low-light image based on the light map includes:

[0019] Determining low-light pixels in the low-light image based on the light map;

[0020] Brightness transformation is performed on low-light pixels in the low-light image to obtain an exposure map of the low-light image.

[0021] Optionally, determining low-light pixels in the low-light image based on the light map includes:

[0022] Pixels in the low-light image corresponding to pixels in the illumination image whose pixel values ​​are less than a preset threshold are determined as low-light pixels in the low-light image.

[0023] Optionally, determining the map of regions with good illumination in the low-light image and the map of regions with good illumination in the exposure image includes:

[0024] Determining a first weight matrix and a second weight matrix corresponding to the low-light image;

[0025] determining a map of regions with good illumination in the low-light image based on the first weight matrix and the low-light image;

[0026] Based on the second weight matrix and the exposure map, a map of regions with good illumination in the exposure map is determined.

[0027] Optionally, determining a first weight matrix and a second weight matrix corresponding to the low-light image includes:

[0028] Calculating the weight of each pixel in the illumination map using the pixel value of each pixel in the illumination map as a base and a preset value as an exponent, and combining the weights of each pixel in the illumination map to obtain the first weight matrix;

[0029] Normalize the first weight matrix to obtain the second weight matrix.

[0030] A low-light image enhancement device, comprising:

[0031] an acquisition unit, configured to acquire a low-light image to be enhanced;

[0032] a decomposition unit, configured to input the low-light image into a decomposition model, wherein the decomposition model decomposes the low-light image to obtain an illumination map; the decomposition model is trained based on a training image pair, wherein the training image pair includes a training low-light image and a training normal-light image, wherein the training low-light image and the training normal-light image are two incompletely paired images of the same scene;

[0033] An enhancement unit is configured to obtain an enhanced image based on the illumination map.

[0034] Optionally, the training loss of the decomposition model includes:

[0035] The consistency loss between the feature map of the reflectance map of the training low-light image output by the decomposition model and the feature map of the training low-light image processed by histogram equalization.

[0036] Optionally, the training loss also includes:

[0037] Any one or more of the discriminator loss between the enhanced image obtained based on the illumination map of the training low-light image output by the decomposition model and the training normal-light image, the consistency loss between the reflectance map of the training low-light image output by the decomposition model and the reflectance map of the training normal-light image output by the decomposition model, and the reconstruction loss of the illumination map of the training low-light image output by the decomposition model and the reflectance map of the training low-light image output by the decomposition model.

[0038] Optionally, the enhancement unit includes:

[0039] an exposure map determining unit, configured to determine an exposure map of the low-light image based on the light map;

[0040] a good illumination area map determining unit, configured to determine a good illumination area map in the low-light image and a good illumination area map in the exposure image;

[0041] The reconstruction unit is used to reconstruct the image of the region with good illumination in the low-light image and the image of the region with good illumination in the exposure image to obtain an enhanced image.

[0042] Optionally, the exposure map determining unit includes:

[0043] a low-light pixel determination unit, configured to determine low-light pixels in the low-light image based on the light map;

[0044] The brightness conversion unit is used to perform brightness conversion on low-light pixels in the low-light image to obtain an exposure image of the low-light image.

[0045] Optionally, the low-light pixel determination unit is specifically configured to:

[0046] Pixels in the low-light image corresponding to pixels in the illumination image whose pixel values ​​are less than a preset threshold are determined as low-light pixels in the low-light image.

[0047] Optionally, the good illumination area map determining unit includes:

[0048] a weight matrix determining unit, configured to determine a first weight matrix and a second weight matrix corresponding to the low-light image;

[0049] a first good illumination area map determining unit, configured to determine a good illumination area map in the low-light image based on the first weight matrix and the low-light image;

[0050] The second good illumination area map determining unit is configured to determine a good illumination area map in the exposure map based on the second weight matrix and the exposure map.

[0051] Optionally, the weight matrix determining unit includes:

[0052] a first weight matrix determining unit, configured to calculate the weight of each pixel in the illumination map using the pixel value of each pixel in the illumination map as a base and a preset value as an exponent, and to combine the weights of each pixel in the illumination map to obtain the first weight matrix;

[0053] The second weight matrix determining unit is configured to perform normalization calculation on the first weight matrix to obtain the second weight matrix.

[0054] A low-light image enhancement device comprising a memory and a processor;

[0055] The memory is used to store programs;

[0056] The processor is used to execute the program to implement the various steps of the low-light image enhancement method described above.

[0057] A readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the computer program implements the various steps of the low-light image enhancement method described above.

[0058] By means of the above technical solution, the present application discloses a low-light image enhancement method, related equipment, and readable storage medium. In this solution, a decomposition model is pre-trained based on a training image pair comprising an incompletely paired training low-light image and a training normal-light image of the same scene. After obtaining the low-light image to be enhanced, the low-light image is input into the decomposition model to obtain an illumination map. Finally, the enhanced image is obtained using the illumination map. Since the decomposition model is trained based on incompletely paired image pairs, such image pairs are easy to obtain in large quantities, resulting in better decomposition model training results and more accurate illumination maps. Therefore, using the illumination map to enhance the low-light image can achieve better enhancement effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0060] Figure 1 A schematic diagram of the process of the low-light image enhancement method disclosed in an embodiment of the present application;

[0061] Figure 2 A schematic diagram of the structure of the decomposition model disclosed in the embodiment of this application;

[0062] Figure 3 A schematic diagram of the training process of the decomposition model disclosed in the embodiment of this application;

[0063] Figure 4 A schematic diagram of obtaining an enhanced image based on the illumination map disclosed in an embodiment of the present application;

[0064] Figure 5 This is a schematic structural diagram of a low-light image enhancement device disclosed in an embodiment of the present application;

[0065] Figure 6 This is a hardware structure block diagram of a low-light image enhancement device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0067] Next, the low-light image enhancement method provided by this application is introduced through the following embodiments.

[0068] Reference Figure 1 , Figure 1 This is a flow chart of a low-light image enhancement method disclosed in an embodiment of the present application. The method may include:

[0069] Step S101: Acquire a low-light image to be enhanced.

[0070] In this application, the low-light image to be enhanced can be the original low-light image captured by a camera device (such as a camera, a terminal device with a camera, etc.) in any scene (such as a traffic scene, a monitoring scene, etc.), or it can be the low-light image obtained after pre-processing the original image (such as image cropping, etc.). This application does not impose any restrictions on this.

[0071] Step S102: Input the low-light image into a decomposition model, and the decomposition model decomposes the low-light image to obtain an illumination map; the decomposition model is trained based on a training image pair, and the training image pair includes a training low-light image and a training normal-light image, and the training low-light image and the training normal-light image are two incompletely paired images of the same scene.

[0072] In this application, the decomposition model can adopt various forms of neural network structures, such as convolutional neural network, which is not limited in this application. Figure 2 , Figure 2 This is a structural diagram of the decomposition model disclosed in an embodiment of the present application. The decomposition network consists of a Conv layer, multiple Conv+Relu layers and a Sigmoid layer.

[0073] It should be noted that, compared to decomposition models in the prior art, the decomposition model of this application is trained based on incompletely paired low-light images and normal-light images from the same scene. This improves the decomposition effect of the decomposition model of this application compared to the decomposition model in the prior art. The training method of the decomposition model will be explained in the following examples and will not be described in detail here.

[0074] Step S103: obtaining an enhanced image based on the illumination map.

[0075] In this application, after determining the illumination map of the low-light image, an enhanced image can be obtained based on the illumination map. The specific implementation will be described in detail in the following embodiments and will not be described in detail here.

[0076] This embodiment discloses a method for low-light image enhancement. In this solution, a decomposition model is pre-trained based on a training image pair consisting of an incompletely paired training low-light image and a training normal-light image from the same scene. After obtaining the low-light image to be enhanced, the low-light image is input into the decomposition model to obtain an illumination map. Finally, the enhanced image is obtained using the illumination map. Because the decomposition model is trained based on incompletely paired image pairs, such image pairs are easy to obtain in large quantities, resulting in better decomposition model training results and more accurate illumination maps. Therefore, using the illumination map to enhance low-light images can achieve better enhancement results.

[0077] It should be noted that, as an implementable method, the decomposition model of the present application can be trained by using the training method of the decomposition model in the prior art, that is, by constructing a reflection Figure 1 The decomposition network is optimized by using a loss function consistent with the original image quality and a reconstruction loss based on the Retinex theory. That is, the consistency loss between the reflectance map of the training low-light image output by the decomposition model and the reflectance map of the training normal-light image output by the decomposition model, and the reconstruction loss between the illumination map of the training low-light image output by the decomposition model and the reflectance map of the training low-light image output by the decomposition model are used as a joint loss to train the decomposition network. However, this method is applicable to the case where the input images are completely paired, while in this application, the input images are not completely paired. If the above training method is used for training, it may lead to poor training results.

[0078] Therefore, in another embodiment of the present application, an improved training method is proposed to train the decomposition model.

[0079] As an implementable embodiment, in the improved training method proposed in the present application, the training loss of the decomposition model includes: the consistency loss between the feature map of the reflectance map of the training low-light image output by the decomposition model and the feature map of the training low-light image after histogram equalization processing.

[0080] It should be noted that the feature map of the reflectance map of the training low-light image can be obtained by extracting the feature map of the reflectance map of the training low-light image using a VGG network (such as VGG19, VGG16, etc.), and the feature map of the training low-light image processed by histogram equalization can be obtained by extracting the feature map of the training low-light image processed by histogram equalization using a VGG network (such as VGG19, VGG16, etc.).

[0081] As another possible implementation method, in the improved training method proposed in the present application, the training loss of the decomposition model, in addition to including the consistency loss between the feature map of the reflectance map of the training low-light image output by the decomposition model and the feature map of the training low-light image after histogram equalization processing, may also include: the discriminator loss of the enhanced image obtained based on the illumination map of the training low-light image output by the decomposition model and the training normal-light image, the consistency loss between the reflectance map of the training low-light image output by the decomposition model and the reflectance map of the training normal-light image output by the decomposition model, and any one or more of the reconstruction loss of the illumination map of the training low-light image output by the decomposition model and the reflectance map of the training low-light image output by the decomposition model.

[0082] For easier understanding, please refer to Figure 3 , Figure 3 Schematic diagram of the training process of the decomposition model disclosed in the embodiment of this application. Figure 3 As shown, during the training process, a set of training normal light images J and training low light images I under the same scene are input into the decomposition network. The decomposition network decomposes the training normal light image J to obtain the illumination map I of the training normal light image. normal And the reflection map R of the normal light image used for training normal , the decomposition network decomposes the low-light image I for training and obtains the illumination map I of the low-light image for training low and the reflectance map R of the training low-light image low , where I result is the enhanced image corresponding to the low-light image used for training.

[0083] The consistency loss between the feature map of the reflectance map of the training low-light image output by the decomposition model and the feature map of the training low-light image processed by histogram equalization is L fm .

[0084] Its loss function can be expressed as: L fm =|F(Rlow )-F(H(I))|1

[0085] Here, H(*) represents the histogram equalization operation used to extract content information, and F(*) represents the feature map used to extract the corresponding image. |*|1 represents the absolute value loss. This feature map-based consistency loss effectively addresses the issue of incomplete image matching.

[0086] The discriminator loss between the enhanced image obtained based on the illumination map of the training low-light image output by the decomposition model and the training normal-light image is L LSGAN (D). In this application, the discriminator loss can be generated using least squares, and its loss function can be expressed as:

[0087]

[0088] Here, x represents the normal illumination image, z represents the generated enhanced image, and D represents the discriminator used to determine whether the enhanced result is true or false. The discriminator loss can ensure the quality of the generated enhanced image to a certain extent.

[0089] The consistency loss between the reflectance map of the training low-light image output by the decomposition model and the reflectance map of the training normal-light image output by the decomposition model is L ci . Its loss function can be expressed as: L ci =|R low -R normal |2

[0090] Among them, |*|2 represents the loss obtained by taking pixel-by-pixel difference on the image and then squaring it. This loss can supervise the structural information of the reflection map at the pixel level, thereby effectively ensuring the consistency of the reflection map.

[0091] The reconstruction loss of the illumination map of the training low-light image output by the decomposition model and the reflectance map of the training low-light image output by the decomposition model is L recon . Its loss function can be expressed as:

[0092] Here, |*|1 represents the absolute loss applied pixel by pixel on the image. The reconstruction loss further ensures the effectiveness of the decomposition network.

[0093] It should be noted that the more comprehensive the training loss is, the better the decomposition model training effect will be. In practical applications, different application requirements can be considered and the above different training losses can be selected as the joint training loss. This application does not impose any restrictions on this. For example, if the above four training losses are selected as the joint training loss, the joint training loss is:

[0094]

[0095] Among them, α1, α2, α3, and α4 are the weights corresponding to each training loss.

[0096] In another embodiment of the present application, a specific implementation method for obtaining an enhanced image based on the illumination map is introduced, and the method may include the following steps:

[0097] Step S201: determining an exposure map of the low-light image based on the light map.

[0098] As an implementable method, the low-light pixels in the low-light image can be determined based on the illumination map; the low-light pixels in the low-light image are subjected to brightness transformation to obtain an exposure map of the low-light image. Specifically, the pixels in the low-light image corresponding to the pixels whose pixel values ​​in the illumination map are less than a preset threshold are determined as the low-light pixels in the low-light image. The preset threshold can be determined based on different scene requirements, and this application does not impose any restrictions. For example, the preset threshold can be 0.5, then the formula for extracting low-light pixels is as follows: T=I low <0.5

[0099] The specific expression of the brightness transformation function is as follows:

[0100]

[0101] Where a and b are camera parameters, which are fixed parameters for most cameras (a = 0.3293, b = 1.1258), and k is the exposure rate. The exposure rate k is obtained by solving the bounded maximization of entropy:

[0102]

[0103] Step S202: determining a map of regions with good illumination in the low-light image and a map of regions with good illumination in the exposure image.

[0104] As an implementable embodiment, determining the map of areas with good illumination in the low-light image and the map of areas with good illumination in the exposure image includes: determining a first weight matrix and a second weight matrix corresponding to the low-light image; determining the map of areas with good illumination in the low-light image based on the first weight matrix and the low-light image; and determining the map of areas with good illumination in the exposure image based on the second weight matrix and the exposure image.

[0105] Determining the map of regions with good illumination in the low-light image based on the first weight matrix and the low-light image may involve performing a convolution operation on the first weight matrix and the low-light image to obtain the map of regions with good illumination in the low-light image. Determining the map of regions with good illumination in the exposure image based on the second weight matrix and the exposure image may involve performing a convolution operation on the second weight matrix and the exposure image to obtain the map of regions with good illumination in the exposure image.

[0106] As an implementation method, the weight of each pixel in the illumination map can be calculated with the pixel value of each pixel in the illumination map as the base and a preset value as the exponent, and the weight combination of each pixel in the illumination map obtains the first weight matrix; the first weight matrix is ​​normalized to obtain the second weight matrix.

[0107] It should be noted that the weight matrix is ​​positively correlated with the scene illumination, so the power is used to construct the calculation formula. When the exponent is 0, no enhancement is performed; when the exponent is 1, both underexposed and well-exposed pixels are enhanced; when the exponent is greater than 1, pixels may become saturated and suffer from loss of detail. Therefore, an exponent of 0.5 is selected to enhance while ensuring well-exposed areas. Therefore, the calculation formula for the first weight matrix can be: The second weight matrix is ​​1-W.

[0108] Step S203: reconstructing the image of the region with good illumination in the low-light image and the image of the region with good illumination in the exposure image to obtain an enhanced image.

[0109] In this application, the well-illuminated area map in the low-light image and the well-illuminated area map in the exposure map can be fused to obtain an enhanced image. This method adaptively enhances low-light images by retaining well-exposed areas and only enhancing underexposed areas, resulting in a good enhanced image.

[0110] To understand steps S201 to S203, please refer to Figure 4 , Figure 4 This is a schematic diagram of obtaining an enhanced image based on the illumination map disclosed in an embodiment of the present application.

[0111] The low-light image enhancement device disclosed in an embodiment of the present application is described below. The low-light image enhancement device described below and the low-light image enhancement method described above can refer to each other.

[0112] Reference Figure 5 , Figure 5 This is a schematic diagram of the structure of a low-light image enhancement device disclosed in an embodiment of this application. Figure 5As shown, the low-light image enhancement device may include:

[0113] An acquisition unit 11 is configured to acquire a low-light image to be enhanced;

[0114] a decomposition unit 12 configured to input the low-light image into a decomposition model, wherein the decomposition model decomposes the low-light image to obtain a lighting map; the decomposition model is trained based on a training image pair, wherein the training image pair includes a training low-light image and a training normal-light image, wherein the training low-light image and the training normal-light image are two incompletely paired images of the same scene;

[0115] The enhancement unit 13 is configured to obtain an enhanced image based on the illumination map.

[0116] As an implementation method, the training loss of the decomposition model includes:

[0117] The consistency loss between the feature map of the reflectance map of the training low-light image output by the decomposition model and the feature map of the training low-light image processed by histogram equalization.

[0118] As an implementable embodiment, the training loss further includes:

[0119] Any one or more of the discriminator loss between the enhanced image obtained based on the illumination map of the training low-light image output by the decomposition model and the training normal-light image, the consistency loss between the reflectance map of the training low-light image output by the decomposition model and the reflectance map of the training normal-light image output by the decomposition model, and the reconstruction loss of the illumination map of the training low-light image output by the decomposition model and the reflectance map of the training low-light image output by the decomposition model.

[0120] As an implementable embodiment, the enhancement unit includes:

[0121] an exposure map determining unit, configured to determine an exposure map of the low-light image based on the light map;

[0122] a good illumination area map determining unit, configured to determine a good illumination area map in the low-light image and a good illumination area map in the exposure image;

[0123] The reconstruction unit is used to reconstruct the image of the region with good illumination in the low-light image and the image of the region with good illumination in the exposure image to obtain an enhanced image.

[0124] As an implementable embodiment, the exposure map determining unit includes:

[0125] a low-light pixel determination unit, configured to determine low-light pixels in the low-light image based on the light map;

[0126] The brightness conversion unit is used to perform brightness conversion on low-light pixels in the low-light image to obtain an exposure image of the low-light image.

[0127] As an implementable method, the low-light pixel determination unit is specifically configured to:

[0128] Pixels in the low-light image corresponding to pixels in the illumination image whose pixel values ​​are less than a preset threshold are determined as low-light pixels in the low-light image.

[0129] In one possible implementation manner, the good illumination area map determining unit includes:

[0130] a weight matrix determining unit, configured to determine a first weight matrix and a second weight matrix corresponding to the low-light image;

[0131] a first good illumination area map determining unit, configured to determine a good illumination area map in the low-light image based on the first weight matrix and the low-light image;

[0132] The second good illumination area map determining unit is configured to determine a good illumination area map in the exposure map based on the second weight matrix and the exposure map.

[0133] As an implementable embodiment, the weight matrix determining unit includes:

[0134] a first weight matrix determining unit, configured to calculate the weight of each pixel in the illumination map using the pixel value of each pixel in the illumination map as a base and a preset value as an exponent, and to combine the weights of each pixel in the illumination map to obtain the first weight matrix;

[0135] The second weight matrix determining unit is configured to perform normalization calculation on the first weight matrix to obtain the second weight matrix.

[0136] Reference Figure 6 , Figure 6 The hardware structure diagram of the low-light image enhancement device provided in the embodiment of the present application is shown in FIG. Figure 6 ,The hardware structure of the low-light image enhancement device may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

[0137] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;

[0138] The processor 1 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention;

[0139] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0140] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0141] Acquire a low-light image to be enhanced;

[0142] Inputting the low-light image into a decomposition model, wherein the decomposition model decomposes the low-light image to obtain a lighting map; the decomposition model is trained based on a training image pair, wherein the training image pair includes a training low-light image and a training normal-light image, wherein the training low-light image and the training normal-light image are two incompletely paired images of the same scene;

[0143] Based on the illumination map, an enhanced image is obtained.

[0144] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0145] The present application also provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0146] Acquire a low-light image to be enhanced;

[0147] Inputting the low-light image into a decomposition model, wherein the decomposition model decomposes the low-light image to obtain a lighting map; the decomposition model is trained based on a training image pair, wherein the training image pair includes a training low-light image and a training normal-light image, wherein the training low-light image and the training normal-light image are two incompletely paired images of the same scene;

[0148] Based on the illumination map, an enhanced image is obtained.

[0149] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0150] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0151] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0152] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A low-light image enhancement method, characterized in that: The method comprises: Acquire a low-light image to be enhanced; Inputting the low-light image into a decomposition model, the decomposition model decomposing the low-light image to obtain an illumination map; the decomposition model is trained based on a training image pair, the training image pair including a training low-light image and a training normal-light image, the training low-light image and the training normal-light image being two incompletely paired images of the same scene; the training loss of the decomposition model includes: a consistency loss between a feature map of a reflectance map of the training low-light image output by the decomposition model and a feature map of the training low-light image processed by histogram equalization; Based on the illumination map, an enhanced image is obtained.

2. The method according to claim 1, characterized in that The training loss also includes: Any one or more of the discriminator loss between the enhanced image obtained based on the illumination map of the training low-light image output by the decomposition model and the training normal-light image, the consistency loss between the reflectance map of the training low-light image output by the decomposition model and the reflectance map of the training normal-light image output by the decomposition model, and the reconstruction loss of the illumination map of the training low-light image output by the decomposition model and the reflectance map of the training low-light image output by the decomposition model.

3. The method according to claim 1, characterized in that Obtaining an enhanced image based on the illumination map includes: determining an exposure map for the low-light image based on the light map; determining a map of regions with good illumination in the low-light image and a map of regions with good illumination in the exposure image; The map of the region with good illumination in the low-light image and the map of the region with good illumination in the exposure image are reconstructed to obtain an enhanced image.

4. The method according to claim 3, characterized in that The determining, based on the illumination map, an exposure map of the low-light image includes: Determining low-light pixels in the low-light image based on the light map; Brightness transformation is performed on low-light pixels in the low-light image to obtain an exposure map of the low-light image.

5. The method according to claim 4, characterized in that The determining, based on the illumination map, low-light pixel points in the low-light image includes: Pixels in the low-light image corresponding to pixels in the illumination image whose pixel values ​​are less than a preset threshold are determined as low-light pixels in the low-light image.

6. The method according to claim 3, characterized in that The determining of the region map with good illumination in the low-light image and the region map with good illumination in the exposure image includes: Determining a first weight matrix and a second weight matrix corresponding to the low-light image; determining a map of regions with good illumination in the low-light image based on the first weight matrix and the low-light image; Based on the second weight matrix and the exposure map, a map of regions with good illumination in the exposure map is determined.

7. The method according to claim 6, characterized in that Determining the first weight matrix and the second weight matrix corresponding to the low-light image includes: Calculating the weight of each pixel in the illumination map using the pixel value of each pixel in the illumination map as a base and a preset value as an exponent, and combining the weights of each pixel in the illumination map to obtain the first weight matrix; Normalize the first weight matrix to obtain the second weight matrix.

8. A low-light image enhancement device, characterized in that: The device comprises: an acquisition unit, configured to acquire a low-light image to be enhanced; a decomposition unit, configured to input the low-light image into a decomposition model, wherein the decomposition model decomposes the low-light image to obtain an illumination map; the decomposition model is trained based on a training image pair, wherein the training image pair includes a training low-light image and a training normal-light image, wherein the training low-light image and the training normal-light image are two incompletely paired images of the same scene; and a training loss of the decomposition model includes a consistency loss between a feature map of a reflectance map of the training low-light image output by the decomposition model and a feature map of the training low-light image processed by histogram equalization; An enhancement unit is configured to obtain an enhanced image based on the illumination map.

9. A low-light image enhancement device, characterized in that: including memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement each step of the low-light image enhancement method according to any one of claims 1 to 7.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the low-light image enhancement method according to any one of claims 1 to 7 is implemented.

Images