Low-light image enhancement method and device, electronic equipment and readable medium

Abstract

Embodiments of the present disclosure disclose a low-light image enhancement method, device, electronic equipment and readable medium. A specific embodiment of the method comprises: acquiring a low-light image and a reference image; performing image enhancement processing on the low-light image; inputting the reference image into a reference image feature extraction layer; performing the following enhancement processing: inputting second low-light image feature information and a first reference image feature information in a reference image feature information sequence into a condition information generation module; inputting fused image feature information and the first low-light image feature information into a feature enhancement module; updating the first low-light image feature information; updating the reference image feature information sequence; based on the updated reference image feature information sequence, performing the enhancement processing again; determining the updated first low-light image feature information as enhanced low-light image feature information; and performing enhancement processing on the low-light image. The embodiment improves the enhancement effect of the low-light image.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer technology, and in particular, to a low-light image enhancement method and device, electronic equipment and a computer readable medium. BACKGROUND

[0002] Low-light image enhancement is a technique for image enhancement of low-light images. At present, when image enhancement is performed on low-light images, the commonly used method is to directly uniformly adjust the brightness and contrast of the low-light images to achieve the effect of enhancing the low-light images.

[0003] However, when the above method is used to enhance low-light images, the following technical problems often exist:

[0004] Directly uniformly adjusting the brightness and contrast of low-light images to achieve the effect of enhancing the low-light images is usually a global adjustment, which may change the relative brightness relationship of different color regions in the low-light images, thereby causing color distortion of the low-light images. At the same time, uniformly adjusting the global brightness and contrast of the low-light images ignores the detailed information in the low-light images. In the high-brightness region, it may cause detail overexposure and loss; in the low-brightness region, it may cause the low-light image to still appear blurred due to insufficient brightness improvement, thereby causing poor enhancement effect of the low-light image.

[0005] The above information disclosed in this BACKGROUND section is only for enhancing the understanding of the background of the present inventive concepts, and therefore, it can include information that does not form the prior art known to those of ordinary skill in the art. SUMMARY

[0006] The summary of the present disclosure is used to introduce the concepts in a brief manner, which will be described in detail in the specific embodiments section. The summary of the present disclosure is not intended to identify key or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

[0007] Some embodiments of the present disclosure propose a low-light image enhancement method, device, electronic equipment and computer readable medium to solve one or more of the technical problems mentioned in the above BACKGROUND section.

[0008] In a first aspect, some embodiments of the present disclosure provide a low-light image enhancement method, which comprises: acquiring a low-light image and a first preset number of reference images; inputting the low-light image into a first convolutional layer and a second convolutional layer included in a pre-trained image enhancement model to obtain first low-light image feature information and second low-light image feature information; for each reference image in the first preset number of reference images, based on the reference image, performing the following image enhancement processing on the low-light image: inputting the reference image into a second preset number of reference image feature extraction layers included in the image enhancement model to obtain a reference image feature information sequence corresponding to the reference image; based on the reference image feature information sequence and the first low-light image feature information, performing the following enhancement processing: inputting the second low-light image feature information and a first reference image feature information in the reference image feature information sequence into a conditional information generation module included in the image enhancement model to obtain fusion image feature information; inputting the fusion image feature information and the first low-light image feature information into a feature enhancement module included in the image enhancement model to obtain enhanced low-light image feature information; updating the first low-light image feature information based on the enhanced low-light image feature information; deleting the first reference image feature information from the reference image feature information sequence to update the reference image feature information sequence; in response to determining that the updated reference image feature information sequence is not empty, based on the updated reference image feature information sequence and the updated first low-light image feature information, performing the enhancement processing again; in response to determining that the updated reference image feature information sequence is empty, determining the updated first low-light image feature information as the enhanced low-light image feature information; based on the enhanced low-light image feature information, performing enhancement processing on the low-light image to obtain an enhanced low-light image consistent with the image style of the reference image.

[0009] In a second aspect, some embodiments of the present disclosure provide a low-light image enhancement device, the device comprising: an acquisition unit configured to acquire a low-light image and a first preset number of reference images; an input unit configured to input the low-light image into a first convolutional layer and a second convolutional layer included in a pre-trained image enhancement model to obtain first low-light image feature information and second low-light image feature information; an enhancement processing unit configured to, for each reference image in the first preset number of reference images, based on the reference image, perform the following image enhancement processing on the low-light image: input the reference image into a second preset number of reference image feature extraction layers included in the image enhancement model to obtain a reference image feature information sequence corresponding to the reference image; based on the reference image feature information sequence and the first low-light image feature information, perform the following enhancement processing: input the second low-light image feature information and a first reference image feature information in the reference image feature information sequence into a conditional information generation module included in the image enhancement model to obtain fusion image feature information; input the fusion image feature information and the first low-light image feature information into a feature enhancement module included in the image enhancement model to obtain enhanced low-light image feature information; update the first low-light image feature information based on the enhanced low-light image feature information; delete the first reference image feature information from the reference image feature information sequence to update the reference image feature information sequence; in response to determining that the updated reference image feature information sequence is not empty, perform the enhancement processing again based on the updated reference image feature information sequence and the updated first low-light image feature information; in response to determining that the updated reference image feature information sequence is empty, determine the updated first low-light image feature information as the enhanced low-light image feature information; and perform enhancement processing on the low-light image based on the enhanced low-light image feature information to obtain an enhanced low-light image consistent with the image style of the reference image.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the implementations of the first aspect.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program is executed by a processor to implement the method described in any of the implementations of the first aspect.

[0012] The above various embodiments of the present disclosure have the following beneficial effects: through the low-light image enhancement method of some embodiments of the present disclosure, the color distortion degree of the low-light image is reduced and the enhancement effect of the low-light image is improved. Specifically, the reason for the color distortion of the low-light image and the poor enhancement effect of the low-light image is that the brightness and contrast of the low-light image are directly and uniformly adjusted to achieve the effect of enhancing the low-light image. Directly and uniformly adjusting the brightness and contrast of the low-light image is usually global adjustment, which may change the relative brightness relationship of different color regions in the low-light image, thereby causing color distortion of the low-light image. At the same time, uniformly adjusting the global brightness and contrast of the low-light image ignores the detail information in the low-light image. In the high brightness region, it may cause detail overexposure and loss; in the low brightness region, it may cause the low-light image to still appear blurred due to insufficient brightness improvement, thereby causing poor enhancement effect of the low-light image. Based on this, the low-light image enhancement method of some embodiments of the present disclosure first acquires a low-light image and a first preset number of reference images. In this way, the first preset number of reference images for enhancing the low-light image can be obtained. Then, the low-light image is input into the first convolutional layer and the second convolutional layer included in the pre-trained image enhancement model to obtain first low-light image feature information and second low-light image feature information. In this way, the first low-light image feature information and the second low-light image feature information can represent the low-light image. The first low-light image feature information and the second low-light image feature information can include edge, texture, corner and other detail feature information of the low-light image. Then, for each reference image in the first preset number of reference images, based on the reference image, the following image enhancement processing is performed on the low-light image: first, the reference image is input into the second preset number of reference image feature extraction layers included in the image enhancement model to obtain reference image feature information corresponding to the reference image. In this way, the second preset number of reference image feature extraction layers can extract reference image feature information at different scales of the reference image to obtain a reference image feature information sequence. Based on the reference image feature information sequence and the first low-light image feature information, the following enhancement processing is performed: first, the second low-light image feature information and the first reference image feature information in the reference image feature information sequence are input into the conditional information generation module included in the image enhancement model to obtain fusion image feature information. In this way, the reference image feature information of the low-light image can be fused into the second low-light image feature information of the low-light image to obtain fusion image feature information for generating enhanced low-light image feature information. Through the conditional information fusion module, the color information of the reference image can be combined with the second low-light image feature information of the low-light image, thereby ensuring that the brightness is enhanced while the accuracy and naturalness of the image color are maintained.Then, the fusion image feature information and the first low-light image feature information are input into a feature enhancement module included in the image enhancement model to obtain enhanced low-light image feature information. In this way, the enhanced low-light image feature information for updating the first low-light image feature information can be obtained. Then, the first low-light image feature information is updated based on the enhanced low-light image feature information. In this way, the first low-light image feature information can be updated to contain both the feature information of the reference image (i.e., at least one reference image feature information) and the low-light image detail feature information. Then, the first reference image feature information is deleted from the reference image feature information sequence to update the reference image feature information sequence. Then, in response to determining that the updated reference image feature information sequence is not empty, the enhancement processing is performed again based on the updated reference image feature information sequence and the updated first low-light image feature information. Then, in response to determining that the updated reference image feature information sequence is empty, the updated first low-light image feature information is determined as the enhanced low-light image feature information. In this way, the enhanced low-light image feature information containing both the feature information of the reference image (i.e., at least one reference image feature information) and the low-light image detail feature information can be obtained through the above-mentioned enhancement processing. Based on the enhanced low-light image feature information, the low-light image is enhanced to obtain an enhanced low-light image consistent with the image style of the reference image. In this way, the low-light image can be enhanced through the enhanced low-light image feature information containing both the feature information of the reference image (i.e., at least one reference image feature information) and the low-light image detail feature information, and the color information of the reference image and the features of the low-light image are combined to enhance the brightness and contrast while maintaining the accuracy and naturalness of the color of the low-light image, reducing the degree of color distortion of the low-light image. At the same time, the low-light image detail feature information is considered in the process of enhancing the low-light image, reducing the possibility of overexposure in the high-brightness area and insufficient enhancement in the low-brightness area, and improving the enhancement effect of the low-light image. BRIEF DESCRIPTION OF DRAWINGS

[0013] The above-described and other features and advantages of the present disclosure will be more apparent from the following detailed description taken in conjunction with the accompanying drawings. Throughout the drawings, like or similar reference numerals are used to refer to like or similar elements. It should be understood that the drawings are schematic and elements and features are not necessarily to scale.

[0014] Figure 1 is a flowchart of some embodiments of a low-light image enhancement method according to the present disclosure;

[0015] Figure 2 is a structural schematic diagram of some embodiments of a low-light image enhancement device according to the present disclosure;

[0016] Figure 3is a structural schematic diagram of an electronic device suitable for implementing some embodiments of the present disclosure;

[0017] Figure 4 is a model structure schematic diagram of some embodiments of an image enhancement model according to the present disclosure;

[0018] Figure 5 is a module structure schematic diagram of some embodiments of a conditional information generation module included in an image enhancement model according to the present disclosure;

[0019] Figure 6 is a module structure schematic diagram of some embodiments of a feature enhancement module included in an image enhancement model according to the present disclosure. DETAILED DESCRIPTION

[0020] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein. Rather, these embodiments are provided so as to more completely and comprehensively understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes, and are not intended to limit the scope of protection of the present disclosure.

[0021] It should also be noted that, for ease of description, only parts related to the present application are shown in the drawings. The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0022] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0023] It should be noted that the adjectives "one", "multiple" mentioned in the present disclosure are illustrative and not limiting, and those skilled in the art should understand that, unless otherwise explicitly stated in the context, it should be understood as "one or more".

[0024] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0025] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0026] Figure 1 Flow 100 of some embodiments of a low-light image enhancement method according to the present disclosure is shown. The low-light image enhancement method includes the following steps:

[0027] At step 101, a low-light image and a first preset number of reference images are obtained.

[0028] In some embodiments, an execution subject (e.g., a computing device) of the low-light image enhancement method can obtain the low-light image and the first preset number of reference images through a wired connection or a wireless connection. The low-light image can be an image taken in a low-light environment (e.g., at night). In practice, the execution subject can obtain the low-light image through a camera. The execution subject can obtain the first preset number of reference images from a preset database. Each of the first preset number of reference images can be an image taken in a normal environment.

[0029] It should be noted that the wireless connection can include, but is not limited to, 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other now known or future developed wireless connection methods.

[0030] At step 102, the low-light image is input to a first convolutional layer and a second convolutional layer included in a pre-trained image enhancement model to obtain first low-light image feature information and second low-light image feature information.

[0031] In some embodiments, the execution subject can input the low-light image to the first convolutional layer and the second convolutional layer included in the pre-trained image enhancement model to obtain the first low-light image feature information and the second low-light image feature information. The first convolutional layer and the second convolutional layer can be convolutional layers for feature extraction of the low-light image. Figure 4 A model structure diagram of some embodiments of the image enhancement model of the present disclosure is shown. The model structure of the image enhancement model is as shown in Figure 4 The first convolutional layer and the second convolutional layer can be convolutional layers for feature extraction of the low-light image. The first low-light image feature information can represent a feature map of the low-light image. The first convolutional layer can be the same as the second convolutional layer, or the first convolutional layer can be different from the second convolutional layer. The second low-light image feature information can represent a feature map obtained by feature extraction of the low-light image. The image enhancement model can be a neural network model for image enhancement of the low-light image according to the first preset number of reference images to obtain each enhanced low-light image. The image enhancement model can include the first convolutional layer and the second convolutional layer, a conditional information generation module, and a feature enhancement module. The output information of the second convolutional layer and the reference image feature information are input to the conditional information generation module. The output information of the conditional information generation module and the output information of the first convolutional layer can be output to the feature enhancement module. The reference image feature information can be a feature map representing a reference image.

[0032] Step 103, for each reference image in the first preset number of reference images, based on the reference image, the low-light image is subjected to the following image enhancement processing:

[0033] Step 1031, input the reference image into the second preset number of reference image feature extraction layers included in the image enhancement model to obtain the reference image feature information sequence corresponding to the reference image.

[0034] In some embodiments, the execution subject can input the reference image into the second preset number of reference image feature extraction layers included in the image enhancement model to obtain the reference image feature information sequence corresponding to the reference image. The second preset number of reference image feature extraction layers can be each convolutional layer for feature extraction of the reference image. The reference image feature information sequence can represent the image features of the reference image at each scale. Each reference image feature information in the reference image feature information sequence can represent the features (such as edges, colors, contrasts, etc.) of the reference image extracted at a single scale.

[0035] Step 1032, based on the reference image feature information sequence and the first low-light image feature information, the following enhancement processing is performed:

[0036] Step 10321, input the second low-light image feature information and the first reference image feature information in the reference image feature information sequence into the conditional information generation module included in the image enhancement model to obtain the fusion image feature information.

[0037] In some embodiments, the execution subject can input the second low-light image feature information and the first reference image feature information in the reference image feature information sequence into the conditional information generation module included in the image enhancement model to obtain the fusion image feature information. Wherein, Figure 5 is a module structure diagram of some embodiments of the conditional information generation module included in the image enhancement model according to the present disclosure. The conditional information generation module is as shown in Figure 5 The conditional information generation module includes a low-light image feature pooling layer, a reference image feature pooling layer, a feature fusion layer, and a full connection layer. The low-light image feature pooling layer can be a pooling layer for maximum pooling processing of the second low-light image feature information. The reference image feature pooling layer can be a pooling layer for maximum pooling processing of the reference image feature information. The feature fusion layer can be a Concat layer for connection operation of the pooled second low-light image feature information and the pooled reference image feature information.

[0038] In some optional implementations of some embodiments, the execution subject can input the second low-light image feature information and the first reference image feature information in the sequence of reference image feature information into a conditional information generation module included in the image enhancement model to obtain fused image feature information:

[0039] In a first step, the second low-light image feature information is input into the low-light image feature pooling layer to obtain pooled low-light image feature information corresponding to the second low-light image feature information. The pooled low-light image feature information can be information obtained by performing maximum pooling on the second low-light image feature information.

[0040] In a second step, the first reference image feature information in the sequence of reference image feature information is determined as target reference image feature information.

[0041] In a third step, the target reference image feature information is input into the reference image feature pooling layer to obtain pooled reference image feature information corresponding to the target reference image feature information. The pooled reference image feature information can be information obtained by performing maximum pooling on the target reference image feature information.

[0042] In a fourth step, the pooled low-light image feature information and the pooled reference image feature information are input into the feature fusion layer to obtain fused image feature information. The fused feature information can be a feature map obtained by splicing the pooled low-light image feature information and the pooled reference image feature information. In practice, after the pooled low-light image feature information and the pooled reference image feature information are input into the feature fusion layer, the feature fusion layer can perform Num-dimensional or Channel-dimensional splicing on the pooled low-light image feature information and the pooled reference image feature information to obtain the fused image feature information. As an example, the pooled low-light image feature information can be (N, C1, H, W), and the pooled reference image feature information can be (N, C2, H, W). N can represent the number of feature maps represented by the low-light image feature information and the pooled reference image feature information. C1 can be the number of channels of the feature map represented by the pooled low-light image feature information, representing the feature information (such as edges, colors, contrasts, etc.) of the low-light image. C2 can be the number of channels represented by the pooled reference image feature information, representing the feature information (such as edges, colors, contrasts, etc.) of the reference image. H and W represent the height and width of the feature map represented by the low-light image feature information and the pooled reference image feature information, respectively. The fused image feature information can be (N, C1+C2, H, W).

[0043] In the fifth step, the fusion image feature information is input into the full connection layer to obtain image enhancement guidance information. The image enhancement guidance information can be the fusion image feature information after nonlinear transformation and integration.

[0044] In the sixth step, the fusion image feature information is generated based on the image enhancement guidance information and the pooled low-light image feature information. In practice, the execution subject can perform element-level addition on the feature map represented by the image enhancement guidance information and the feature map represented by the pooled low-light image feature information to obtain the fusion image feature information. The fusion image feature information can be a feature map obtained by performing element-level addition on the feature map represented by the image enhancement guidance information and the feature map represented by the pooled low-light image feature information.

[0045] In some optional implementations of some embodiments, the execution subject can generate the fusion image feature information based on the image enhancement guidance information and the pooled low-light image feature information by the following steps:

[0046] In the first step, the image enhancement guidance information and the pooled low-light image feature information are fused to obtain initial fusion image feature information. In practice, the execution subject can perform matrix multiplication on the feature map represented by the image enhancement guidance information and the feature map represented by the pooled low-light image feature information to obtain the initial fusion image feature information.

[0047] In the second step, the initial fusion image feature information and the image enhancement guidance information are added element by element to obtain the fusion image feature information.

[0048] In step 10322, the fusion image feature information and the first low-light image feature information are input into the feature enhancement module included in the image enhancement model to obtain enhanced low-light image feature information.

[0049] In some embodiments, the execution subject can input the fusion image feature information and the first low-light image feature information into the feature enhancement module included in the image enhancement model to obtain the enhanced low-light image feature information. Wherein, Figure 6 is a module structure diagram of some embodiments of the feature enhancement module included in the image enhancement model according to the present disclosure. As Figure 6The feature enhancement module includes a first feature enhancement layer 601, a second feature enhancement layer 602, and a third feature enhancement layer 603. The first feature enhancement layer 601 includes a first residual context module, a second residual context module, and a conditional aggregation module. The second residual context module receives a first enhanced feature map, and the conditional aggregation module receives information output by the second residual context module. Each of the first residual context module, the second residual context module, a second residual context module of the second feature enhancement layer, a second residual context module of the third feature enhancement layer, a first residual context module of the third feature enhancement layer, and a second residual context module of the third feature enhancement layer can divide the information input into the residual context module into two branches. One branch applies convolution, reshape, and tpgunay operations to the information input into the residual context module in sequence to obtain first branch feature information. The other branch applies a reshape operation to the information input into the residual context module to obtain second branch feature information. The feature information (the first branch feature information and the second branch feature information) generated by the two branches is fused, and matrix multiplication is applied to obtain global feature information. The global feature information is then convolved by two convolution layers to obtain global convolution feature information. The residual context module is used to extract features in the feature enhancement layer and suppress useless features from being further transmitted forward. The execution subject can determine the information obtained by applying convolution to the information input into the residual context module as initial feature information. Finally, the execution subject can add the initial feature information and the global convolution feature information at the element level to aggregate the global convolution feature information into the initial feature information. The conditional aggregation module of the first feature enhancement layer and the second feature enhancement layer can be used to introduce features of a reference image into a low-light image. The conditional aggregation module can include a fully connected layer and an activation function layer. The conditional aggregation module can fuse the image feature information and the received information. Each of the first feature aggregation module of the second feature enhancement layer, the second feature aggregation module of the second feature enhancement layer, the first feature aggregation module of the third feature enhancement layer, and the second feature aggregation module of the third feature enhancement layer includes a feature fusion part and a feature calibration part. The purpose of the feature fusion operation is to combine information from different feature enhancement layers (resolution streams) to generate new feature identifiers. The feature calibration operation uses these identifiers to readjust the feature mapping from different streams. The feature aggregation module can be used to fuse features of different resolution streams and aggregate features of different scales using a self-attention mechanism. The feature aggregation module can be a multi-scale feature fusion module.

[0050] In some optional implementations of some embodiments, the execution subject can input the fusion image feature information and the first low-light image feature information into a feature enhancement module included in the image enhancement model to obtain enhanced low-light image feature information by the following steps:

[0051] In a first step, the first low-light image feature information is down-sampled according to first preset down-sampling rate information, second preset down-sampling rate information, and third preset down-sampling rate information to obtain a first feature map, a second feature map, and a third feature map. For example, the first preset down-sampling rate information can be “down-sampling rate 2”, the second preset down-sampling rate information can be “down-sampling rate 4”, and the third preset down-sampling rate information can be “down-sampling rate 8”. In practice, the execution subject can perform down-sampling on the first low-light image feature information according to the first preset down-sampling rate information to obtain the third feature map. Then, the execution subject can perform down-sampling on the first feature map according to the second preset down-sampling rate information to obtain the second feature map. After that, the execution subject can perform down-sampling on the second feature map to obtain the first feature map.

[0052] In a second step, the first feature map is input into an enhancement layer one first residual context module of the first feature enhancement layer to obtain a first enhanced feature map. The first feature enhancement layer includes the enhancement layer one first residual context module, an enhancement layer one second residual context module, and an enhancement layer one conditional aggregation module. The enhancement layer one second residual context module receives the first enhanced feature map, the enhancement layer one conditional aggregation module receives information output by the enhancement layer one second residual context module, and the enhancement layer one conditional aggregation module performs fusion processing on the fusion image feature information and the received information.

[0053] In the third step, the second feature map is input into the second feature enhancement layer to obtain a second enhanced feature map. The second feature enhancement layer 602 includes the first residual context module of the second enhancement layer, the first feature aggregation module of the second enhancement layer, the second residual context module of the second enhancement layer, the second feature aggregation module of the second enhancement layer, and the conditional aggregation module of the second enhancement layer. The first feature aggregation module of the second enhancement layer receives the up-sampling information of the second enhanced feature map and the first enhanced feature map, and aggregates the received information. The second residual context module of the second enhancement layer receives the output information of the first feature aggregation module of the second enhancement layer. The second feature aggregation module of the second enhancement layer receives the output information of the second residual context module of the second enhancement layer and the up-sampling information of the output information of the conditional aggregation module of the first enhancement layer, and aggregates the received information. The conditional aggregation module of the second enhancement layer receives the output information of the second feature aggregation module of the second enhancement layer, and fuses the output information of the second feature aggregation module of the second enhancement layer and the received information to obtain the fused image feature information.

[0054] In the fourth step, the third feature map is input into the first residual context module of the third feature enhancement layer to obtain a third enhanced feature map. The third feature enhancement layer 603 includes the first residual context module of the third enhancement layer, the first feature aggregation module of the third enhancement layer, the second residual context module of the third enhancement layer, and the second feature aggregation module of the third enhancement layer. The first feature aggregation module of the third enhancement layer receives the output information of the first residual context module of the third enhancement layer and the up-sampling information of the output information of the first feature aggregation module of the second enhancement layer, and aggregates the received information. The second residual context module of the third enhancement layer receives the output information of the first feature aggregation module of the third enhancement layer. The second feature aggregation module of the third enhancement layer receives the output information of the second residual context module of the third enhancement layer and the up-sampling information of the output information of the conditional aggregation module of the second enhancement layer, and aggregates the received information to obtain the enhanced low-light image feature information.

[0055] In step 10323, the first low-light image feature information is updated based on the enhanced low-light image feature information.

[0056] In some embodiments, the execution subject can update the first low-light image feature information based on the enhanced low-light image feature information.

[0057] In some optional implementations of some embodiments, the execution subject can update the first low-light image feature information based on the enhanced low-light image feature information by the following steps:

[0058] In the first step, the enhanced low-light image feature information is input into a convolutional layer connected to the feature enhancement module to obtain updated low-light image feature information.

[0059] In the second step, the updated low-light image feature information is updated to the first low-light image feature information to update the first low-light image feature information.

[0060] In step 10324, the first reference image feature information is deleted from the reference image feature information sequence to update the reference image feature information sequence.

[0061] In some embodiments, the execution subject can delete the first reference image feature information from the reference image feature information sequence to update the reference image feature information sequence.

[0062] In step 10325, in response to determining that the updated reference image feature information sequence is not empty, the enhancement processing is performed again based on the updated reference image feature information sequence and the updated first low-light image feature information.

[0063] In some embodiments, the execution subject can perform the enhancement processing again based on the updated reference image feature information sequence and the updated first low-light image feature information in response to determining that the updated reference image feature information sequence is not empty.

[0064] In step 10326, in response to determining that the updated reference image feature information sequence is empty, the updated first low-light image feature information is determined as the enhanced low-light image feature information.

[0065] In some embodiments, the execution subject can determine the updated first low-light image feature information as the enhanced low-light image feature information in response to determining that the updated reference image feature information sequence is empty.

[0066] In step 1033, the low-light image is enhanced based on the enhanced low-light image feature information to obtain an enhanced low-light image consistent with the image style of the reference image.

[0067] In some embodiments, the execution subject can enhance the low-light image based on the enhanced low-light image feature information to obtain an enhanced low-light image consistent with the image style of the reference image.

[0068] In some optional implementations of some embodiments, the execution subject can enhance the low-light image based on the enhanced low-light image feature information to obtain an enhanced low-light image consistent with the image style of the reference image by the following steps:

[0069] In a first step, the enhanced residual image is generated based on the enhanced low-light image feature information. In practice, the execution subject can input the enhanced low-light image feature information into a pre-trained residual network to obtain an enhanced residual image corresponding to the enhanced low-light image feature information. The pre-trained residual network is used to represent the correspondence between the enhanced low-light image feature information and the enhanced residual image. The enhanced residual image can be a residual image for enhancing the low-light image.

[0070] In a second step, the low-light image is enhanced based on the enhanced residual image to obtain an enhanced low-light image consistent with the image style of the reference image. In practice, the execution subject can add the low-light image and the enhanced residual image pixel by pixel to obtain an enhanced low-light image consistent with the image style of the reference image. That is, for each low-light pixel point in the low-light image, the execution subject can determine the enhanced residual image pixel point in the enhanced residual image with the same position as the low-light pixel point as an enhanced pixel point. Then, the execution subject can determine the sum of the pixel value information represented by the low-light pixel point and the pixel value information represented by the enhanced pixel point as updated pixel value information. Then, the execution subject can determine the updated pixel value information as the pixel value information of the low-light pixel point to update the low-light pixel point. Finally, the execution subject can determine the low-light image after updating each low-light pixel point as an enhanced low-light image consistent with the image style of the reference image. As an example, the pixel value information of the low-light pixel point can be (18, 12, 5), the pixel value information of the enhanced pixel point in the enhanced residual image with the same position as the low-light low-light pixel point can be (2, 4, 2), and the updated pixel value information can be (20, 16, 7).

[0071] In the process of using the technical solutions to solve the problems mentioned in the background, the following problems often occur:

[0072] In the process of enhancing the low-light image to obtain each enhanced low-light image consistent with the style of each reference image, the enhanced low-light image after enhancement is not detected, and there may still be some images in each enhanced low-light image after enhancement that have poor quality problems such as darkness, blur, and loss of details, resulting in poor overall image enhancement effect of the low-light image.

[0073] In the face of the above technical problems, the following solutions are adopted:

[0074] In some optional implementations of some embodiments, after the above image enhancement processing of the low-light image based on the reference image for each reference image in the first preset number of reference images, the execution subject can further perform the following steps:

[0075] In a first step, each of the generated enhanced low-light images is determined as a to-be-detected enhanced low-light image.

[0076] In a second step, for each of the to-be-detected enhanced low-light images, the following detection steps are performed:

[0077] In a first sub-step, the to-be-detected enhanced low-light image is segmented to obtain a set of to-be-detected enhanced low-light image blocks. In practice, the execution subject can segment the to-be-detected enhanced low-light image by using an image segmentation technique to obtain the set of to-be-detected enhanced low-light image blocks.

[0078] In a second sub-step, each to-be-detected enhanced low-light image block in the set of to-be-detected enhanced low-light image blocks is subjected to frequency domain conversion processing to obtain frequency domain matrix information corresponding to the to-be-detected enhanced low-light image block. In practice, the execution subject can subject the to-be-detected enhanced low-light image block to frequency domain conversion processing by using a DCT transformation technique to obtain the frequency domain matrix information corresponding to the to-be-detected enhanced low-light image block. The frequency domain matrix information can be a discrete cosine transformation coefficient matrix.

[0079] In a third sub-step, for each of the obtained frequency domain matrix information, the frequency domain matrix information and the to-be-detected enhanced low-light image block corresponding to the frequency domain matrix information are input into a pre-trained image block quality detection model to obtain image block quality detection information. The image block quality detection model can be an image quality assessment (IQA) model that takes the frequency domain matrix information of an image as input and outputs image block quality detection information. The image block quality detection information can be a score of the to-be-detected enhanced low-light image block.

[0080] In a fourth sub-step, based on the obtained image block quality detection information, image quality score information of the to-be-detected enhanced low-light image is generated. In practice, the execution subject can determine the mean of the quality scores corresponding to the image block quality detection information as the image quality score information.

[0081] In a third step, each of the obtained image quality score information is determined as image quality score information.

[0082] In a fourth step, each image quality score information in the image quality score information set whose image quality score is less than or equal to a preset value is determined as a screening image quality score information.

[0083] In a fifth step, each of the generated enhanced low-light images is determined as an enhanced low-light image set.

[0084] Step six, for each of the screening image quality score information, the following processing is performed:

[0085] Sub-step one, determine the initial enhanced low-light image from the enhanced low-light images in the enhanced low-light image set corresponding to the screening image quality score information.

[0086] Sub-step two, perform super-resolution reconstruction processing on the initial enhanced low-light image to obtain a repaired enhanced low-light image. In practice, the execution subject can perform super-resolution reconstruction processing on the initial enhanced low-light image through an image super-resolution reconstruction algorithm to obtain a repaired enhanced low-light image.

[0087] Sub-step three, replace the initial enhanced low-light image in the enhanced low-light image set with the repaired enhanced low-light image to update the enhanced low-light image set.

[0088] Step seven, determine the updated enhanced low-light image set as the enhanced image set corresponding to the low-light image.

[0089] The technical solution and related content thereof serve as one inventive point of the embodiments of the present disclosure, and solve the technical problem of poor overall image enhancement effect of low-light images. Factors leading to poor overall image enhancement effect of low-light images are often as follows: when low-light images are enhanced, each enhanced low-light image consistent with the style of each reference image is obtained, and the enhanced low-light images are not detected in quality after enhancement, and some of the enhanced low-light images may still have problems such as poor quality of dimness, blurring, and loss of details, leading to poor overall image enhancement effect of low-light images. If the above factors are solved, the overall image enhancement effect of low-light images can be improved. To achieve this effect, first, each enhanced low-light image generated is determined as each to-be-detected enhanced low-light image. Then, for each to-be-detected enhanced low-light image in the to-be-detected enhanced low-light images, the following detection steps are performed: first, the to-be-detected enhanced low-light image is segmented to obtain a set of to-be-detected enhanced low-light image blocks. In this way, a set of to-be-detected enhanced low-light image blocks for quality detection of the to-be-detected enhanced low-light image can be obtained. Second, each to-be-detected enhanced low-light image block in the set of to-be-detected enhanced low-light image blocks is subjected to frequency domain conversion processing to obtain frequency domain matrix information corresponding to the to-be-detected enhanced low-light image block. Third, for each frequency domain matrix information in the obtained frequency domain matrix information, the frequency domain matrix information and the to-be-detected enhanced low-light image block corresponding to the frequency domain matrix information are input into a pre-trained image block quality detection model to obtain image block quality detection information. In this way, the frequency domain matrix information of the image block can be input into the image block quality detection model to detect the quality of the image block and obtain the image block quality detection information. Third, based on the obtained image block quality detection information, image quality score information of the to-be-detected enhanced low-light image is generated. In this way, the quality of each to-be-detected enhanced low-light image can be detected through the above detection steps to obtain each image quality score information of each to-be-detected enhanced low-light image. Then, the obtained image quality score information is determined as a set of image quality score information. Next, each image quality score information in the set of image quality score information whose image quality score represented by the image quality score information is less than or equal to a preset value is determined as each screening image quality score information. In this way, each screening image quality score information with poor image quality can be determined. Then, each enhanced low-light image generated is determined as a set of enhanced low-light images. After that, for each screening image quality score information in the screening image quality score information, the following processing is performed: first, the enhanced low-light image corresponding to the screening image quality score information in the set of enhanced low-light images is determined as an initial enhanced low-light image. In this way, the initial enhanced low-light image with poor image quality corresponding to the screening image quality score information can be determined from the set of enhanced low-light images.Then, the initial enhanced low-light image is subjected to super-resolution reconstruction processing to obtain a repaired enhanced low-light image. In this way, the initial enhanced low-light image with poor image quality can be subjected to super-resolution reconstruction processing, and the image quality of the initial enhanced low-light image can be improved. Next, the initial enhanced low-light image in the set of enhanced low-light images is replaced by the repaired enhanced low-light image to update the set of enhanced low-light images. Then, the updated set of enhanced low-light images is determined as the set of enhanced images corresponding to the low-light image. Because the low-light image is subjected to image enhancement processing to obtain each enhanced low-light image, each enhanced low-light image is subjected to image quality detection to obtain each image quality score information. According to each image quality score information, the enhanced low-light image with poor image quality in each enhanced low-light image is subjected to super-resolution reconstruction processing to obtain the set of enhanced images corresponding to the low-light image. The image quality of the enhanced low-light image with poor image quality in each enhanced low-light image is improved, and the overall image enhancement effect of the low-light image is improved.

[0090] The above various embodiments of the present disclosure have the following beneficial effects: through the low-light image enhancement method of some embodiments of the present disclosure, the color distortion degree of the low-light image is reduced and the enhancement effect of the low-light image is improved. Specifically, the reason for the color distortion of the low-light image and the poor enhancement effect of the low-light image is that the brightness and contrast of the low-light image are directly and uniformly adjusted to achieve the effect of enhancing the low-light image. Directly and uniformly adjusting the brightness and contrast of the low-light image is usually global adjustment, which may change the relative brightness relationship of different color regions in the low-light image, thereby causing color distortion of the low-light image. At the same time, uniformly adjusting the global brightness and contrast of the low-light image ignores the detail information in the low-light image. In the high brightness region, it may cause detail overexposure and loss; in the low brightness region, it may cause the low-light image to still appear blurred due to insufficient brightness improvement, thereby causing poor enhancement effect of the low-light image. Based on this, the low-light image enhancement method of some embodiments of the present disclosure first acquires a low-light image and a first preset number of reference images. In this way, the first preset number of reference images for enhancing the low-light image can be obtained. Then, the low-light image is input into the first convolutional layer and the second convolutional layer included in the pre-trained image enhancement model to obtain first low-light image feature information and second low-light image feature information. In this way, the first low-light image feature information and the second low-light image feature information can represent the low-light image. The first low-light image feature information and the second low-light image feature information can include edge, texture, corner and other detail feature information of the low-light image. Then, for each reference image in the first preset number of reference images, based on the reference image, the following image enhancement processing is performed on the low-light image: first, the reference image is input into the second preset number of reference image feature extraction layers included in the image enhancement model to obtain reference image feature information corresponding to the reference image. In this way, the second preset number of reference image feature extraction layers can extract reference image feature information at different scales of the reference image to obtain a reference image feature information sequence. Based on the reference image feature information sequence and the first low-light image feature information, the following enhancement processing is performed: first, the second low-light image feature information and the first reference image feature information in the reference image feature information sequence are input into the conditional information generation module included in the image enhancement model to obtain fusion image feature information. In this way, the reference image feature information of the low-light image can be fused into the second low-light image feature information of the low-light image to obtain fusion image feature information for generating enhanced low-light image feature information. Through the conditional information fusion module, the color information of the reference image can be combined with the second low-light image feature information of the low-light image, thereby ensuring that the brightness is enhanced while the accuracy and naturalness of the image color are maintained.Then, the fusion image feature information and the first low-light image feature information are input into a feature enhancement module included in the image enhancement model to obtain enhanced low-light image feature information. In this way, the enhanced low-light image feature information for updating the first low-light image feature information can be obtained. Then, the first low-light image feature information is updated based on the enhanced low-light image feature information. In this way, the first low-light image feature information can be updated to contain both the feature information of the reference image (i.e., at least one reference image feature information) and the low-light image detail feature information. Then, the first reference image feature information is deleted from the sequence of reference image feature information to update the sequence of reference image feature information. Then, in response to determining that the updated sequence of reference image feature information is not empty, the enhancement processing is performed again based on the updated sequence of reference image feature information and the updated first low-light image feature information. Then, in response to determining that the updated sequence of reference image feature information is empty, the updated first low-light image feature information is determined as the enhanced low-light image feature information. In this way, the enhanced low-light image feature information containing both the feature information of the reference image (i.e., at least one reference image feature information) and the low-light image detail feature information can be obtained through the above enhancement processing. Based on the enhanced low-light image feature information, the low-light image is enhanced to obtain an enhanced low-light image consistent with the image style of the reference image. In this way, the low-light image can be enhanced through the enhanced low-light image feature information containing both the feature information of the reference image (i.e., at least one reference image feature information) and the low-light image detail feature information, and the color information of the reference image and the features of the low-light image are combined to enhance the brightness and contrast while maintaining the accuracy and naturalness of the color of the low-light image, reducing the degree of color distortion of the low-light image. At the same time, the low-light image detail feature information is considered in the process of enhancing the low-light image, reducing the possibility of overexposure in high-brightness areas and insufficient enhancement in low-brightness areas, and improving the enhancement effect of the low-light image.

[0091] Further reference Figure 2 As an implementation of the methods shown in the various figures, the present disclosure provides some embodiments of low-light image enhancement devices, which device embodiments correspond to those method embodiments shown in Figure 1 The devices can be applied in various electronic devices.

[0092] As Figure 2As shown, the low-light image enhancement apparatus 200 of some embodiments includes an acquisition unit 201, an input unit 202, and an enhancement processing unit 203. The acquisition unit 201 is configured to acquire a low-light image and a first preset number of reference images. The input unit 202 is configured to input the low-light image into a first convolutional layer and a second convolutional layer included in a pre-trained image enhancement model to obtain first low-light image feature information and second low-light image feature information. The enhancement processing unit 203 is configured to, for each reference image in the first preset number of reference images, perform the following image enhancement processing on the low-light image based on the reference image: input the reference image into a second preset number of reference image feature extraction layers included in the image enhancement model to obtain a sequence of reference image feature information corresponding to the reference image; based on the sequence of reference image feature information and the first low-light image feature information, perform the following enhancement processing: input the second low-light image feature information and a first reference image feature information in the sequence of reference image feature information into a conditional information generation module included in the image enhancement model to obtain fused image feature information; input the fused image feature information and the first low-light image feature information into a feature enhancement module included in the image enhancement model to obtain enhanced low-light image feature information; update the first low-light image feature information based on the enhanced low-light image feature information; delete the first reference image feature information from the sequence of reference image feature information to update the sequence of reference image feature information; in response to determining that the updated sequence of reference image feature information is not empty, based on the updated sequence of reference image feature information and the updated first low-light image feature information, perform the enhancement processing again; in response to determining that the updated sequence of reference image feature information is empty, determine the updated first low-light image feature information as the enhanced low-light image feature information; and perform enhancement processing on the low-light image based on the enhanced low-light image feature information to obtain an enhanced low-light image consistent with the image style of the reference image.

[0093] It can be understood that the units described in the apparatus 200 correspond to the respective steps in the method described above. Therefore, the operations, features, and advantages described above for the method also apply to the apparatus 200 and the units included therein, which will not be described here again. Figure 1

[0094] The following refers to Figure 3 which shows a structural schematic diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0095] As Figure 3 ​As shown, the electronic device 300 can include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301 that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 302 or loaded into a random access memory (RAM) 303 from a storage device 308. Various programs and data required for the operation of the electronic device 300 are also stored in the RAM 303. The processing device 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0096] Generally, the following devices can be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 can allow the electronic device 300 to communicate wirelessly or wired with other devices to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that all of the illustrated devices are not required, and more or fewer devices can alternatively be implemented. Figure 3 Each block shown in the flowcharts can represent a device, or multiple devices, as necessary.

[0097] In particular, processes described above with reference to the flowcharts can be implemented as a computer software program according to some embodiments of the present disclosure. For example, some embodiments of the present disclosure include a computer program product including a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In some such embodiments, the computer program can be downloaded and installed from a network through the communication devices 309, or installed from the storage devices 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the functions defined in the methods of some embodiments of the present disclosure are performed.

[0098] Note that the computer-readable medium in some embodiments of the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may, for example and without limitation, be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, the computer-readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave in a propagated data signal, in which the computer-readable program code is contained. Such a propagated data signal can take a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium that is not a storage medium and that can communicate, propagate, or transport program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained in the computer-readable medium can be transmitted using any suitable medium, including but not limited to wire, cable, RF, etc., or any suitable combination of the foregoing.

[0099] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0100] The computer readable medium can be included in the electronic device or exist separately from the electronic device. The computer readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire a low-light image and a first preset number of reference images; input the low-light image into a first convolutional layer and a second convolutional layer included in a pre-trained image enhancement model to obtain first low-light image feature information and second low-light image feature information; for each reference image in the first preset number of reference images, based on the reference image, perform the following image enhancement processing on the low-light image: input the reference image into a second preset number of reference image feature extraction layers included in the image enhancement model to obtain a reference image feature information sequence corresponding to the reference image; based on the reference image feature information sequence and the first low-light image feature information, perform the following enhancement processing: input the second low-light image feature information and a first reference image feature information in the reference image feature information sequence into a conditional information generation module included in the image enhancement model to obtain fusion image feature information; input the fusion image feature information and the first low-light image feature information into a feature enhancement module included in the image enhancement model to obtain enhanced low-light image feature information; update the first low-light image feature information based on the enhanced low-light image feature information; delete the first reference image feature information from the reference image feature information sequence to update the reference image feature information sequence; in response to determining that the updated reference image feature information sequence is not empty, based on the updated reference image feature information sequence and the updated first low-light image feature information, perform the enhancement processing again; in response to determining that the updated reference image feature information sequence is empty, determine the updated first low-light image feature information as the enhanced low-light image feature information; and based on the enhanced low-light image feature information, perform enhancement processing on the low-light image to obtain an enhanced low-light image consistent with the image style of the reference image.

[0101] Computer program code for carrying out operations of some embodiments of the present disclosure can be written in any one or combination of one or more programming languages or combinations thereof, including an object-oriented programming language such as Java, Smalltalk, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0102] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0103] The units described in some embodiments of the present disclosure can be implemented by software, or can be implemented by hardware. The described units can also be arranged in a processor, for example, can be described as: a processor includes an acquisition unit, an input unit, and an enhancement processing unit. Among them, the names of these units do not constitute a limitation to the units themselves in some cases, for example, the acquisition unit can also be described as "a unit for acquiring the low-light image and the first preset number of reference images".

[0104] The functions described above in the specification of the present disclosure can be performed at least in part by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0105] The above description is merely exemplary of some preferred embodiments of the present disclosure and technical principles thereof. It should be understood by those skilled in the art that the scope of the application involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combinations of the technical features, and should also cover other technical solutions formed by the combinations of the technical features or their equivalent features without departing from the inventive concept. For example, technical solutions formed by the mutual replacement of features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) are also included.

Claims

1. A low-light image enhancement method, comprising: obtaining a low-light image and a first preset number of reference images; inputting the low-light image into a first convolutional layer and a second convolutional layer included in a pre-trained image enhancement model to obtain first low-light image feature information and second low-light image feature information; for each reference image in the first preset number of reference images, based on the reference image, performing the following image enhancement processing on the low-light image: inputting the reference image into a second preset number of reference image feature extraction layers included in the image enhancement model to obtain a reference image feature information sequence corresponding to the reference image; based on the reference image feature information sequence and the first low-light image feature information, performing the following enhancement processing: inputting the second low-light image feature information and a first reference image feature information in the reference image feature information sequence into a conditional information generation module included in the image enhancement model to obtain fusion image feature information; inputting the fusion image feature information and the first low-light image feature information into a feature enhancement module included in the image enhancement model to obtain enhanced low-light image feature information; updating the first low-light image feature information based on the enhanced low-light image feature information; deleting the first reference image feature information from the reference image feature information sequence to update the reference image feature information sequence; in response to determining that the updated reference image feature information sequence is not empty, based on the updated reference image feature information sequence and the updated first low-light image feature information, performing the enhancement processing again; in response to determining that the updated reference image feature information sequence is empty, determining the updated first low-light image feature information as the enhanced low-light image feature information; based on the enhanced low-light image feature information, performing enhancement processing on the low-light image to obtain an enhanced low-light image consistent with the image style of the reference image.

2. The method of claim 1, wherein, The feature enhancement module is connected with the convolutional layer, and the updating of the first low-light image feature information based on the enhanced low-light image feature information comprises: inputting the enhanced low-light image feature information into the convolutional layer connected with the feature enhancement module to obtain updated low-light image feature information; updating the updated low-light image feature information as the first low-light image feature information to update the first low-light image feature information.

3. The method of claim 1, wherein, The conditional information generation module includes a low-light image feature pooling layer, a reference image feature pooling layer, a feature fusion layer, a fully connected layer, and the inputting of the second low-light image feature information and the first reference image feature information in the reference image feature information sequence into the conditional information generation module included in the image enhancement model to obtain the fusion image feature information comprises: inputting the second low-light image feature information into the low-light image feature pooling layer to obtain pooled low-light image feature information corresponding to the second low-light image feature information; determining the first reference image feature information in the reference image feature information sequence as target reference image feature information; Input the target reference image feature information into the reference image feature pooling layer to obtain pooled reference image feature information corresponding to the target reference image feature information; Input the pooled low-light image feature information and the pooled reference image feature information into the feature fusion layer to obtain fused image feature information; Input the fused image feature information into the full connection layer to obtain image enhancement guidance information; Generate fused image feature information based on the image enhancement guidance information and the pooled low-light image feature information.

4. The method of claim 3, wherein, The generating of the fused image feature information based on the image enhancement guidance information and the pooled low-light image feature information comprises: Fusing the image enhancement guidance information and the pooled low-light image feature information to obtain initial fused image feature information; Performing element-level addition processing on the initial fused image feature information and the image enhancement guidance information to obtain fused image feature information.

5. The method of claim 1, wherein, The enhancing of the low-light image based on the enhanced low-light image feature information to obtain an enhanced low-light image consistent with the image style of the reference image comprises: Generating an enhanced residual image based on the enhanced low-light image feature information; Enhancing the low-light image based on the enhanced residual image to obtain an enhanced low-light image consistent with the image style of the reference image.

6. The method of claim 1, wherein, The feature enhancement module comprises a first feature enhancement layer, a second feature enhancement layer and a third feature enhancement layer, and the input of the fused image feature information and the first low-light image feature information into the feature enhancement module of the image enhancement model to obtain enhanced low-light image feature information comprises: According to the first preset down-sampling rate information, the second preset down-sampling rate information and the third preset down-sampling rate information, performing down-sampling processing on the first low-light image feature information to obtain a first feature map, a second feature map and a third feature map; Input the first feature map into the enhancement layer one first residual context module of the first feature enhancement layer to obtain a first enhanced feature map, wherein the first feature enhancement layer comprises the enhancement layer one first residual context module, an enhancement layer one second residual context module and an enhancement layer one conditional aggregation module, the enhancement layer one second residual context module receives the first enhanced feature map, the enhancement layer one conditional aggregation module receives information output by the enhancement layer one second residual context module, and the enhancement layer one conditional aggregation module fuses the fused image feature information and the received information; inputting the second feature map into a second feature enhancement layer, to obtain a second enhanced feature map, wherein the second feature enhancement layer comprises a second residual context module, a second feature aggregation module, a third residual context module, a third feature aggregation module, and a conditional aggregation module, the second feature aggregation module receives up-sampling information of the second enhanced feature map and the first enhanced feature map, and aggregates the received information, the third residual context module receives output information of the second feature aggregation module, the third feature aggregation module receives up-sampling information of output information of the third residual context module and output information of the conditional aggregation module, and aggregates the received information, and the conditional aggregation module receives output information of the third feature aggregation module, and fuses the output information of the third feature aggregation module and the received output information of the second feature aggregation module; inputting the third feature map into a third feature enhancement layer, to obtain a third enhanced feature map, wherein the third feature enhancement layer comprises a third residual context module, a third feature aggregation module, a fourth residual context module, and a fourth feature aggregation module, the third feature aggregation module receives output information of the third residual context module and up-sampling information of output information of the second feature aggregation module, and aggregates the received information, the fourth residual context module receives output information of the third feature aggregation module, the fourth feature aggregation module receives up-sampling information of output information of the fourth residual context module and output information of the conditional aggregation module, and aggregates the received information, to obtain enhanced low-light image feature information.

7. A low-light image enhancement device, comprising: an acquisition unit configured to acquire a low-light image and a first preset number of reference images; an input unit configured to input the low-light image into a first convolutional layer and a second convolutional layer included in a pre-trained image enhancement model, to obtain first low-light image feature information and second low-light image feature information; an enhancement processing unit configured to, for each reference image in the first preset number of reference images, based on the reference image, perform the following image enhancement processing on the low-light image: inputting the reference image into a second preset number of reference image feature extraction layers included in the image enhancement model, to obtain a reference image feature information sequence corresponding to the reference image; based on the first low-light image feature information and the reference image feature information sequence, performing the following enhancement processing: inputting the second low-light image feature information and first reference image feature information in the reference image feature information sequence into a conditional information generation module included in the image enhancement model to obtain fused image feature information; inputting the fused image feature information and the first low-light image feature information into a feature enhancement module included in the image enhancement model to obtain enhanced low-light image feature information; updating the first low-light image feature information based on the enhanced low-light image feature information; deleting the first reference image feature information from the reference image feature information sequence to update the reference image feature information sequence; in response to determining that the updated reference image feature information sequence is not empty, based on the updated reference image feature information sequence and the updated first low-light image feature information, performing the enhancement processing again; in response to determining that the updated reference image feature information sequence is empty, determining the updated first low-light image feature information as the enhanced low-light image feature information; based on the enhanced low-light image feature information, performing enhancement processing on the low-light image to obtain an enhanced low-light image consistent with the image style of the reference image.

8. An electronic device, comprising: one or more processors; storage having one or more programs stored thereon; when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-6.

9. A computer readable medium having stored thereon a computer program, wherein, The program is executed by the processor to implement the method of any one of claims 1-6.

Images