Weak image low light enhancement network model, image processing method and device, and medium

Abstract

The invention belongs to the technical field of image processing network models, and provides a weak image low light enhancement network model, an image processing method and device and a medium, and the method comprises the steps: carrying out the decomposition processing of an input original image through an image decomposition module, so as to generate an illumination layer and two reflection layers; the RGB image enhancement module takes the obtained illumination layer and one of the reflection layers as input, and obtains a first output feature through truth value guide training of an RGB image; the RAW image enhancement module takes the obtained illumination layer and another reflection layer as input, and obtains a second output feature through truth value guide training of the RAW image; the first output feature and the second output feature are used to reconstruct the image. The method has the advantages that a double-domain collaborative enhancement framework is constructed, and the model realizes high-fidelity restoration and efficient enhancement of the image in the low-light environment by modeling illumination consistency and reflectivity details in different domains respectively.

Description

Technical Field

[0001] This invention relates to the field of image processing network model technology, and in particular to a weak image low-light enhancement network model, image processing method, apparatus and medium. Background Technology

[0002] In recent years, more and more people have realized that weak images contain higher bit depths and richer texture information, which can provide more detailed information to guide subsequent tasks. Therefore, low-light enhancement of weak images has seen a large amount of research work centered around deep learning, especially in the weak image domain (RAW) and color image domain (RGB), where significant progress has been made in processing methods, model architecture design, and task domain adaptation.

[0003] The core logic of low-light enhancement revolves around the imbalance in the joint and coordinated processing of illumination and noise. With the development of deep learning, some have proposed using CNNs and Transformers to address different aspects of the model's receptive field, but they have neglected the balance between computational complexity and receptive field. Others have proposed using a novel network architecture like Mamba to achieve a balance between complexity and global receptive field, improving the ability to capture information from different pixel domains in weak images, and gradually applying it to the field of low-light enhancement for weak images. However, most of these different architectures are based on Retinex decomposition theory, simply decomposing the image into illumination and reflection components. This simplistic modeling greatly limits the expansion of the task and the network's learning ability, guiding the network to focus only on these two parts of information in the learning path, failing to realize how to utilize additional supervisory information. Summary of the Invention

[0004] The purpose of this invention is to provide a weak image low-light enhancement network model, image processing method, apparatus and medium to solve the above-mentioned problems.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A weak-map low-light enhancement network model includes: The image decomposition module is configured to decompose the input raw image to generate an illumination layer that represents the image brightness information and two reflection layers that record the inherent color and texture details of the object. An RGB image enhancement module is configured to take the obtained illumination layer and one of the reflection layers as input, and obtain the first output feature through ground truth training of the RGB image. The ground truth training of the RGB image involves convolving the input illumination layer and reflection layer features and adding the fused features to the residual to obtain the first salient illumination feature. The first salient illumination feature and the illumination layer are used to dynamically select the optimal receptive field region and range through depthwise separable convolution. The input features are convolved using convolution kernels with different receptive field ranges and then weighted and fused to form the first output feature. The RAW image enhancement module is configured to take the obtained illumination layer and another reflection layer as input, and obtain a second output feature through ground truth training of the RAW image; the ground truth training of the RAW image involves convolving the input illumination layer and reflection layer features and adding them to the residual after fusion to obtain a second saliency illumination feature; and fusing the receptive field features obtained by scanning in multiple scanning directions with the second saliency illumination feature to obtain a second output feature. The first and second output features were used to reconstruct the original image.

[0006] Furthermore, when decomposing the original image, the illumination layer and reflection layer of the image are obtained based on the multi-level Retinex theory.

[0007] Furthermore, the decomposition process of the original image includes: The initial brightness features are obtained by taking the average value of the original input image in the channel dimension; The initial brightness features are subjected to multiple convolutions and activation functions to obtain the first output; The first output is convolved with a 1×1 layer to obtain the illumination layer.

[0008] Furthermore, the RGB image enhancement module includes a first fusion submodule and a denoising submodule; The first fusion submodule is configured to perform depthwise convolution and 1×1 convolution on the input illumination layer and the first reflection layer features respectively, and then concatenate the two to obtain the fused illumination features. The fused illumination features are used to obtain the first saliency illumination features by adding the convolution and the residual. The denoising submodule is configured to dynamically select the region and extent of the optimal receptive field by taking the first salient illumination features of the input and the illumination layer through a depthwise separable convolution with an increasing convolution kernel and a dilation rate.

[0009] Furthermore, the RGB image enhancement module also includes: Multi-scale feature mappings are obtained by convolving the input features with convolution kernels of different receptive field ranges. The feature maps of convolutional kernels at each scale are concatenated along the channel dimension and then processed by the Sigmoid function to obtain spatial selection labels; The features decomposed by convolutional kernels of different scales are weighted and fused to form the first output feature.

[0010] Furthermore, the RAW image enhancement module includes: a second fusion submodule and a multi-level long / short selection space submodule; The second fusion submodule is configured to perform depthwise convolution and 1×1 convolution on the input illumination layer and the second reflection layer features respectively, and then concatenate the two to obtain the fused illumination features. The fused illumination features are used to obtain the second saliency illumination features by adding the convolution and the residual. The multi-level length-short selection space submodule is configured to construct a spatial perception map by scanning the input second saliency illumination feature through multiple scanning directions to obtain local and global receptive field features, and then fuse the local and global receptive field features to obtain the second output feature.

[0011] Furthermore, in the multi-level long-short selection space module, local and global receptive field features are used to perform spatially variable gating fusion to obtain the second output feature.

[0012] The present invention also provides a method for processing weak-image low-light images, comprising the following steps: The original image is input into the aforementioned weak image low-light enhancement network model to obtain the first output feature and the second output feature; The first output feature and the second output feature are input into the UNet decoding network to reconstruct the image and obtain the reconstructed image features. The restored image is output after the features of the restored image are passed through the convolutional layer.

[0013] The present invention also provides a weak image low-light image processing apparatus, the apparatus including a processor, the processor being used to execute a computer program stored in a memory to implement the steps of the weak image low-light image processing method described above.

[0014] The present invention also provides a computer-readable storage medium that, when the instructions in the storage medium are executed by a processor within a device, enables the device to perform the aforementioned weak image low-light image processing method.

[0015] Compared with the prior art, the present invention has at least the following beneficial effects: (1) A dual-domain collaborative enhancement framework consisting of an image decomposition module, an RGB image enhancement module, and a RAW image enhancement module was constructed. This model achieves high-fidelity restoration and efficient enhancement of images in low-light environments by modeling illumination consistency and reflectivity details in different domains respectively; (2) The image decomposition module constructs a dual-domain decomposition model based on Retinex theory, which explicitly decouples the illumination and reflection components, effectively improving the denoising performance and brightness correction capability, and providing a clean illumination feature representation for subsequent enhancement; (3) The RGB image enhancement module introduces an adaptive region fusion and adaptive receptive field denoising mechanism. Through dynamic feature fusion and noise self-suppression strategy, it achieves texture detail preservation and color consistency optimization in complex low-light and non-uniform noise scenes, and outputs high-quality enhanced images. (4) The RAW image enhancement module integrates a multi-level long-short selection spatial structure with an eight-directional receptive field scanning strategy, and introduces a nested RWKV model to achieve collaborative modeling of global illumination consistency and local reflectivity features. This design maintains spatial continuity and effectively suppresses artifact generation while ensuring linear complexity; (5) The overall network is based on dual-domain coding enhancement and designed with UNet encoding and decoding structure to achieve coordinated enhancement and efficient restoration of RAW and RGB images, while taking into account brightness, noise and color consistency. It significantly improves the brightness balance, detail fidelity and color reproduction performance of images in low-light scenes. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the weak map low-light enhancement network model provided in this embodiment; Figure 2 This is a flowchart of the steps of the weak image low-light image processing method provided in this embodiment. Detailed Implementation

[0018] The following are specific embodiments of the present invention, and the technical solutions of the present invention will be further described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.

[0019] like Figure 1 As shown, this embodiment provides a weak-map low-light enhancement network model, which includes: The image decomposition module is configured to decompose the input raw image to generate an illumination layer that represents the image brightness information and two reflection layers that record the inherent color and texture details of the object.

[0020] An RGB image enhancement module is configured to take the obtained illumination layer and one of the reflection layers as input, and obtain the first output feature by training guided by the ground truth of the RGB image.

[0021] The RAW image enhancement module is configured to take the obtained illumination layer and another reflection layer as input, and obtain a second output feature by training guided by the ground truth of the RAW image.

[0022] The first and second output features were used to reconstruct the original image.

[0023] In this model, the original image (which appears black due to low illumination) is input to the image decomposition module for illumination and reflection separation processing, in order to generate illumination layers that respectively represent the brightness information of the image. And two reflective layers that record the object's inherent color and texture details. , where H, W and C represent the height, width and dimension of the image, respectively.

[0024] This module first retrieves the input image. The mean value along the channel dimension is used to obtain the initial brightness feature. Then, after multiple convolutions and the GELU activation function, the first output is obtained. Then, after 1×1 convolution, another illumination layer L is obtained, whose output is a representation of image brightness information.

[0025] Then, the obtained illumination layer L is used as the input branch of the RGB image enhancement module to guide the brightness and color reconstruction process of the RGB image. This branch uses the real labels of the RGB image as supervision signals and achieves joint optimization of illumination consistency and color restoration through end-to-end training. This module mainly includes a fusion submodule and a noise reduction submodule.

[0026] The fusion submodule is configured to perform depthwise convolution and 1×1 convolution on the input illumination layer and the first reflection layer features respectively, and then concatenate the two to obtain the fused illumination features. The fused illumination features are used to obtain the first salient illumination features by adding the convolution and the residual.

[0027] The denoising submodule is configured to dynamically select the region and extent of the optimal receptive field by taking the first salient illumination features of the input and the illumination layer through a depthwise separable convolution with an increasing convolution kernel and a dilation rate.

[0028] The obtained image brightness information, specifically the illumination layer L, is fed into the fusion submodule to further optimize the spatial consistency and regional contrast representation of the illumination features. This submodule first extracts salient illumination features from different scales and regions, including local salient illumination features from historical periods. Significant illumination characteristics at the current stage Depthwise convolutions are performed on these two types of features to capture local spatial structure information. The results are multiplied by a 1×1 convolution and the GELU function, then added to the residuals via convolution, and finally convolved to obtain the first saliency illumination feature. .

[0029] ; .

[0030] Where DWConv represents depthwise convolution, Conv1 represents 1×1 convolution, and T represents residual features.

[0031] Then, the first salience of illumination features Input denoising submodule. This submodule dynamically selects the optimal receptive field region and range through adaptive receptive field convolution and receptive field spatial selection mechanism to balance noise suppression and detail preservation.

[0032] This submodule first takes the illumination layer L as input and uses a set of depthwise separable convolution sequences with increasing kernel size and dilation rate to gradually expand the receptive field and construct an equivalent large convolution kernel structure. While ensuring linear computational efficiency, it captures rich contextual information, realizes the synergy between local detail extraction and global illumination modeling, and improves the adaptability of multi-scale feature aggregation of the model.

[0033] Specifically, the kernel size C of the i-th depthwise convolution in the adaptive receptive field convolution (ReConv) is... i Expansion rate D i and the corresponding receptive field RC i The recurrence relation can be defined as: ; ; .

[0034] Meanwhile, to avoid discontinuities in illumination features or spatial artifacts caused by excessive expansion rate, this embodiment sets an upper limit constraint on the expansion rate to ensure the spatial consistency and brightness smoothness of illumination features during the expansion of the receptive field, so as to achieve robust and high-precision illumination denoising enhancement.

[0035] Furthermore, to enhance the network's ability to focus on weak image content regions and their weak image content spatial context, a multi-scale spatial selection mechanism is introduced in this embodiment. This mechanism aims to enable the network to adaptively select the spatial regions most relevant to the detected target from large convolutional kernel features of different scales, thereby achieving more accurate feature response and localization in complex scenes. Specifically, it includes: Multi-scale feature maps are obtained by convolving the input features with convolution kernels of different receptive field ranges; the feature maps of each scale convolution kernel are concatenated by channel dimension and processed by the Sigmoid function to obtain spatial selection labels; the features decomposed by convolution kernels of different scales are weighted and fused to form the first output feature.

[0036] Specifically, the illumination features of the input image are first convolved using large convolutional kernels with different receptive fields to obtain multi-scale contextual feature representations. Each convolutional branch has a different focus in capturing local details and global structural information.

[0037] Subsequently, the features from convolutional kernels at various scales are mapped and concatenated along the channel dimension to form a comprehensive feature representation containing rich multi-scale contextual information: ; Among them, F Cn The features are represented by convolution kernels of different scales, and Concat() represents the concatenation operation.

[0038] Convolution generates multi-scale spatial feature regions Spatial selection labels are obtained through the Sigmoid function, and finally, the features decomposed by convolutional kernels of different scales are weighted and fused to form the output. .

[0039] .

[0040] The image is then restored using the UNet decoding network, ultimately yielding... Output characteristics.

[0041] Finally, the image features output by the RGB image enhancement module The input is fed into a Conv_3 convolution to obtain the final output result. By using ground truth values ​​of RGB images to guide training, the enhancement and efficient restoration of RGB images were achieved.

[0042] Furthermore, the illumination layer L obtained from the image brightness information is used as the input to the RAW image enhancement module. This image branch is mainly trained using the ground truth of the RAW image. The RAW image enhancement module mainly consists of two key substructures: a second fusion submodule and a multi-level long / short selection space submodule.

[0043] The second fusion submodule is configured to perform depthwise convolution and 1×1 convolution on the input illumination layer and the second reflection layer features respectively, and then concatenate the two to obtain the fused illumination features. The fused illumination features are used to obtain the second salient illumination features by adding the convolution and the residual.

[0044] The multi-level length-short selection space submodule is configured to construct a spatial perception map by scanning the input second saliency illumination feature through multiple scanning directions to obtain local and global receptive field features, and then fuse the local and global receptive field features to obtain the second output feature.

[0045] The obtained image brightness information, specifically the illumination layer L, is input into the second fusion submodule, representing the local salient illumination features from historical stages. And the current significant lighting characteristics After performing depthwise convolution and 1×1 convolution respectively, the images are stitched together to obtain the fused illumination feature A2, so as to fully explore the potential information in the original low-light image and improve the overall imaging quality.

[0046] ; ; Where DWConv represents depthwise convolution, Conv1 represents 1×1 convolution, and T represents residual features.

[0047] Furthermore, in the multi-level length selection space submodule, this embodiment proposes a multi-directional scanning strategy for wide-area receptive field perception, which introduces eight different scanning directions in a zigzag pattern. (Horizontal, vertical, diagonal, and anti-diagonal paths) are used to construct spatially perceptual mappings. This embodiment incorporates the concept of the RWKV (Receptance-Weighted Key-Value Model) in this module, designing a nested multi-level RWKV structure to enhance the collaborative modeling capability of global illumination consistency and local texture details.

[0048] Specifically, the fused illumination feature A2 is input into a nested multi-level RWKV module, which first projects the input under different receptive fields, corresponding to the inner layer s (local) and the outer layer b (global).

[0049] Inner layer s (smaller receptive field, expansion rate) ): ; ; .

[0050] Outer layer b (larger receptive field, expansion rate) ): ; ; .

[0051] Where W, K, and V are the model elements in the RWKV model, It can be represented as a convolution with dilation r, and the projection P is a pixel-wise mapping.

[0052] Furthermore, K and V are analyzed. Serialization in 8 directions: ; ; Where n can be 1, 2, ..., 8, etc., and represents different directions. K and V can be replaced with... or .

[0053] Projecting onto smaller receptive fields : .

[0054] Projecting a larger receptive field : .

[0055] in, It is represented as a nested, multi-level RWKV structure enhancement model.

[0056] Then, the local and global receptive field features are spatially variable-gated and fused. For both low-light enhancement and preservation of local details, spatially variable gating typically yields the best results. The specific formula is as follows: ; ; ; in, Here, ⊙ represents the Sigmoid function, ⊙ indicates channel concatenation, and Norm represents a normalization layer.

[0057] ; The image is then restored using the UNet decoding network, and the final result is... Output characteristics.

[0058] Finally, the image features output by the RAW image enhancement module are... The input is fed into a Conv_3 convolution to obtain the final output result. .like Figure 1 As shown, since it outputs global and local image features from the original image, the final output result is still a black low-light image.

[0059] By using ground truth training of RAW images, we achieved the enhancement and efficient restoration of RAW images, significantly improving the brightness balance, detail fidelity, and color reproduction performance of images in low-light scenes.

[0060] This structure maintains linear time complexity while globally modeling illumination consistency and locally enhancing reflectivity and texture details. In this way, the network achieves collaborative modeling of global and local features, significantly improving the brightness distribution, detail fidelity, and spatial consistency of the generated RGB and RAW images, while effectively suppressing artifacts and color shift problems caused by discontinuous modeling.

[0061] This embodiment proposes a bidirectional guided multi-level Retinex decomposition-based low-light enhancement network model for weak images. It constructs a dual-domain collaborative enhancement framework consisting of a multi-level Retinex decomposition module, an RGB image enhancement module, and a RAW image enhancement module. By modeling illumination consistency and reflectivity details in different domains, this model achieves high-fidelity restoration and efficient enhancement of images in low-light environments.

[0062] This embodiment also provides a method for processing weak-image low-light images, such as... Figure 2 As shown, the steps include: S1. Input the original image into the weak image low light enhancement network model to obtain the first output feature and the second output feature; S2. Input the first output feature and the second output feature into the UNet decoding network to restore the image and obtain the restored image features; S3. The restored image is output after the features of the restored image are passed through the convolutional layer.

[0063] This embodiment also provides a weak image low-light image processing apparatus, which includes a processor. The processor executes a computer program stored in a memory to implement the steps of the weak image low-light image processing method described above.

[0064] This embodiment also provides a computer-readable storage medium that, when the instructions in the storage medium are executed by a processor within the device, enables the device to perform the aforementioned weak image low-light image processing method.

[0065] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

[0066] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a specific posture. If the specific posture changes, the directional indication will also change accordingly.

[0067] Furthermore, in this invention, descriptions involving terms such as "first," "second," and "a" are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0068] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0069] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

Claims

1. A network model for enhancing low-light performance in weak graphs, characterized in that, include: The image decomposition module is configured to decompose the input raw image to generate an illumination layer that represents the image brightness information and two reflection layers that record the inherent color and texture details of the object. An RGB image enhancement module is configured to take the obtained illumination layer and one of the reflection layers as input, and obtain a first output feature through ground truth training of the RGB image; the ground truth training of the RGB image involves convolving the input illumination layer and reflection layer features and adding the fused features to the residual to obtain a first salient illumination feature; dynamically selecting the optimal receptive field region and range by using depthwise separable convolution of the input first salient illumination feature and the illumination layer; and convolving the input features using convolution kernels with different receptive field ranges and then weighted and fused to form the first output feature; The RAW image enhancement module is configured to take the obtained illumination layer and another reflection layer as input, and obtain a second output feature through ground truth training of the RAW image; the ground truth training of the RAW image involves convolving the input illumination layer and reflection layer features and adding them to the residual after fusion to obtain the second saliency illumination feature. And the receptive field features obtained by scanning the second saliency illumination features through multiple scanning directions are fused to obtain the second output features; The first output feature and the second output feature are used to reconstruct the original image.

2. The weak-graph low-light enhancement network model according to claim 1, characterized in that, When decomposing the original image, the illumination layer and reflection layer of the image are obtained based on the multi-level Retinex theory.

3. The weak-graph low-light enhancement network model according to claim 2, characterized in that, The decomposition process of the original image includes: The initial brightness features are obtained by taking the average value of the original input image in the channel dimension; The initial brightness features are subjected to multiple convolutions and activation functions to obtain the first output; The first output is convolved with a 1×1 vector to obtain the illumination layer.

4. The weak-graph low-light enhancement network model according to claim 1, characterized in that, The RGB image enhancement module includes a first fusion submodule and a noise reduction submodule; The first fusion submodule is configured to perform depthwise convolution and 1×1 convolution on the input illumination layer and the first reflection layer features respectively, and then concatenate the two to obtain fused illumination features. The fused illumination features are used to obtain the first saliency illumination features by adding the convolution and the residual. The denoising submodule is configured to determine the region and extent of the receptive field by performing a depthwise separable convolution with an increasing kernel and a dilation rate on the input first salient illumination feature and the illumination layer.

5. A weak-graph low-light enhancement network model according to claim 1 or 4, characterized in that, The RGB image enhancement module also includes: Multi-scale feature mappings are obtained by convolving the input features with convolution kernels of different receptive field ranges. The feature maps of convolutional kernels at each scale are concatenated along the channel dimension and then processed by the Sigmoid function to obtain spatial selection labels. The features decomposed by convolutional kernels of different scales are weighted and fused to form the first output feature.

6. The weak-graph low-light enhancement network model according to claim 1, characterized in that, The RAW image enhancement module includes: a second fusion submodule and a multi-level long / short selection space submodule; The second fusion submodule is configured to perform depthwise convolution and 1×1 convolution on the input illumination layer and the second reflection layer features respectively, and then concatenate the two to obtain fused illumination features. The fused illumination features are used to obtain second saliency illumination features by adding the convolution and the residual. The multi-level length selection space submodule is configured to construct a spatial perception map by scanning the input second saliency illumination feature in multiple scanning directions to obtain local and global receptive field features, and then fuse the local and global receptive field features to obtain the second output feature.

7. The weak-map low-light enhancement network model according to claim 6, characterized in that, In the multi-level long and short selection space module, the local and global receptive field features are used to perform spatially variable gating fusion to obtain the second output feature.

8. A method for processing weak-image low-light conditions, characterized in that, Including the following steps: The original image is input into the weak image low-light enhancement network model as described in claim 1 to obtain the first output feature and the second output feature; The first output feature and the second output feature are input into the UNet decoding network to perform image reconstruction and obtain the reconstructed image features. The restored image is output after the features of the restored image are passed through a convolutional layer.

9. A weak image low-light image processing device, characterized in that, The device includes a processor that executes a computer program stored in a memory to implement the steps of the weak image low-light image processing method as described in claim 8.

10. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor within the device, the device is able to perform the weak image low-light image processing method of claim 8.

Images