Low-illumination image enhancement method and device based on unsupervised feature decoupling
By employing an unsupervised feature decoupling method, wavelet transform and bidirectional cross-attention mechanism are used to decouple and fuse illumination-independent content and illumination-related representations, thus solving the problem of image quality degradation in low illumination and achieving efficient image enhancement and detection in building construction.
Patent Information
- Application Number
- CN202511159381.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-19
- Publication Date
- 2025-11-28
AI Technical Summary
In building construction, the quality of images captured under low-light conditions degrades, leading to a decrease in the accuracy and reliability of visual detection. Existing methods require paired "low-light-normal-light" image data, which is difficult to obtain and costly. Furthermore, unsupervised methods perform poorly in complex scenes.
An unsupervised feature decoupling method is adopted, which decouples low-light images and normal-light images into illumination-independent content and illumination-dependent representation through wavelet transform and bidirectional cross-attention mechanism. Cross-attention operation is then performed to generate pseudo-low-light and pseudo-normal-light images, and finally restores them to high-quality images.
It can significantly improve the quality of low-light images without the need for paired image data, thereby enhancing the accuracy and reliability of wall surface defect detection in construction scenarios and improving construction efficiency and safety.
Smart Images
Figure CN121032879A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a low-light image enhancement method and apparatus based on unsupervised feature decoupling. Background Technology
[0002] In the construction industry, visual inspection technology is widely used in various quality monitoring and safety assessment scenarios. Specifically, when identifying and locating details such as wall cracks, hollow areas, and leaks, visual inspection technology heavily relies on high-resolution, high-contrast image information as the basis for analysis.
[0003] However, during construction, work surfaces often face insufficient lighting. Images captured under such low-light conditions tend to suffer from significantly reduced quality, manifested as overall low brightness, blurred key details, low signal-to-noise ratio, and color distortion. This image quality degradation severely limits the accuracy and reliability of computer vision-based automatic detection algorithms. To ensure the accuracy of visual detection results, low-light images typically require enhancement. Existing supervised low-light image enhancement methods require paired "low-light-normal-light" images. However, construction site environments are complex and variable, and lighting conditions are difficult to control precisely, making it extremely difficult and costly to acquire a large number of paired "low-light-normal-light" construction scene images. Furthermore, existing unsupervised methods relying on unpaired data and manual prior knowledge often encounter problems such as structural blurring, color distortion, and even unpredictable and ineffective enhancement in complex scenes. Summary of the Invention
[0004] This application provides a low-light image enhancement method and apparatus based on unsupervised feature decoupling, which significantly improves the quality of low-light images. This enables tasks such as detecting surface defects on walls in construction scenarios to be performed in high-quality images without the need for paired image data for training, thereby effectively improving construction efficiency and safety.
[0005] In a first aspect, embodiments of this application provide a low-light image enhancement method based on unsupervised feature decoupling. The method includes: performing wavelet transform on a normal illumination image and a low-light image to obtain a low-frequency sub-image and multiple high-frequency sub-images for each image, and performing inverse wavelet transform after a bidirectional cross-attention operation to obtain a second low-light image and a second normal illumination image; decoupling both the second low-light image and the second normal illumination image into illumination-independent content and illumination-dependent representation; performing a cross-attention operation on the illumination-independent content decoupled from the second low-light image and the illumination-dependent representation decoupled from the second normal illumination image, so that the illumination-independent content of the second low-light image is fused with the first... The illumination information of the second normal illumination image is used; a cross-attention operation is performed on the illumination-independent content decoupled from the second normal illumination image and the illumination-related representation decoupled from the second low illumination image, so that the illumination-independent content of the second normal illumination image is fused with the illumination information of the second low illumination image; the pseudo low illumination image is generated based on the illumination-independent content of the second normal illumination image fused with the illumination information of the second low illumination image, and the pseudo normal illumination image is generated based on the illumination-independent content of the second low illumination image fused with the illumination information of the second normal illumination image; the pseudo low illumination image is restored back to the normal illumination image, and the pseudo normal illumination image is restored back to the low illumination image.
[0006] In conjunction with the first aspect, in one possible implementation, performing wavelet transform on the normal illumination image and the low illumination image to obtain a low-frequency sub-image and multiple high-frequency sub-images for each image includes: performing wavelet transform on the normal illumination image and the low illumination image based on the Haar wavelet basis function to obtain the low-frequency sub-image and the multiple high-frequency sub-images; wherein, the multiple high-frequency sub-images include sub-images of the horizontal edge frequency band, sub-images of the vertical edge frequency band, and sub-images of the diagonal edge frequency band.
[0007] In conjunction with the first aspect, in one possible implementation, decoupling both the second low-light image and the second normal-light image into illumination-independent content and illumination-dependent representation includes: extracting shallow image features from the second low-light image and the second normal-light image; separating illumination-dependent features and content-dependent features from the shallow image features; and mapping the illumination-dependent features and the content-dependent features into illumination-dependent representation and illumination-independent content.
[0008] In conjunction with the first aspect, in one possible implementation, the cross-attention operation performed on the light-independent content decoupled from the second low-light image and the light-related representation decoupled from the second normal-light image, so that the light-independent content of the second low-light image is fused with the light information of the second normal-light image; the cross-attention operation performed on the light-independent content decoupled from the second normal-light image and the light-related representation decoupled from the second low-light image, so that the light-independent content of the second normal-light image is fused with the light information of the second low-light image, includes: mapping the light-related representation and the light-independent content to an embedding space and attaching positional encoding; using the light-related representation of the second normal-light image as a query and the light-independent content of the second low-light image as a key and value, so that the light-independent content of the second low-light image is fused with the light information of the second normal-light image; using the light-related representation of the second low-light image as a query and the light-independent content of the normal-light image as a key and value, so that the light-independent content of the second normal-light image is fused with the light information of the second low-light image.
[0009] In conjunction with the first aspect, in one possible implementation, restoring the pseudo-low-light image back to a normal-light image and restoring the pseudo-normal-light image back to a low-light image includes: performing a wavelet transform-based cross-attention operation on the pseudo-normal-light image and the pseudo-low-light image to obtain a third normal-light image and a third low-light image, respectively; decoupling the third normal-light image and the third low-light image to obtain illumination-independent content and illumination-related representations; performing a cross-attention operation on the illumination-independent content of the third normal-light image and the illumination-related representations of the third low-light image, and performing a cross-attention operation on the illumination-independent content of the third low-light image and the illumination-related representations of the third normal-light image; generating a restored low-light image using the illumination-related representations of the third low-light image and the illumination-independent content of the third normal-light image, and generating a normal-light image using the illumination-independent content of the third low-light image that incorporates the relevant representation information of the third normal-light image.
[0010] Secondly, embodiments of this application provide a low-light image enhancement device based on unsupervised feature decoupling. The device includes: a bidirectional cross-attention module based on wavelet transform, used to perform wavelet transform on a normal illumination image and a low-light image to obtain a low-frequency sub-image and multiple high-frequency sub-images for each image, and then perform inverse wavelet transform after bidirectional cross-attention operation to obtain a second low-light image and a second normal illumination image; an image decoupling module, used to decouple both the second low-light image and the second normal illumination image into illumination-independent content and illumination-dependent representation; and a cooperative cross-attention module, used to decouple the light from the second low-light image... A cross-attention operation is performed on the light-related representation of the second normal illumination image decoupled from the irrelevant content and the second low illumination image; a cross-attention operation is also performed on the light-related representation of the second normal illumination image decoupled from the second low illumination image; an image generation module is used to generate the pseudo low illumination image based on the light-related content of the second normal illumination image fused with the light information of the second low illumination image, and to generate a pseudo normal illumination image based on the light-related content of the second low illumination image fused with the light information of the second normal illumination image; a restoration module is used to restore the pseudo low illumination image back to the normal illumination image, and to restore the pseudo normal illumination image back to the low illumination image.
[0011] Thirdly, embodiments of this application provide an electronic device, the electronic device comprising: one or more processors; and a memory storing computer-executable instructions, which, when executed by the one or more processors, cause the one or more processors to perform the low-light image enhancement method based on unsupervised feature decoupling as described in the first aspect or any possible implementation of the first aspect.
[0012] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-readable instructions, which, when executed by a computer, implement the low-light image enhancement method based on unsupervised feature decoupling as described in the first aspect or any possible implementation thereof.
[0013] The technical solutions provided in this application embodiment have at least the following technical effects:
[0014] The low-light image enhancement method based on unsupervised feature decoupling provided in this application can enhance low-light images without requiring paired "low-light-normal-light" training data during the enhancement process, thus solving the problem of difficulty in obtaining paired "low-light-normal-light" data in building construction scenarios. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 A flowchart of a low-light image enhancement method based on unsupervised feature decoupling provided in an embodiment of this application;
[0017] Figure 2 A schematic diagram of the structure of a low-light image enhancement device based on unsupervised feature decoupling provided in an embodiment of this application;
[0018] Figure 3 This is a schematic diagram of the structure of the restoration module provided in an embodiment of this application. Detailed Implementation
[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0020] To facilitate understanding of the technical solutions provided in the embodiments of this application, the relevant terms involved in the embodiments of this application will be introduced first.
[0021] Wavelet transform is a multi-scale time-frequency analysis tool that decomposes an image into sub-bands of different frequencies and locations to achieve localized feature extraction and multi-resolution representation. Its core idea is to replace the infinitely long sine wave of Fourier transform with a set of wavelet basis functions (i.e., wavelet t), thereby simultaneously capturing high-frequency details (such as edges and textures) and low-frequency structures (such as overall contours) of the image.
[0022] Bidirectional cross-attention is a symmetrical cross-modal or cross-scale attention mechanism that aims to achieve complementary enhancement of two different feature spaces through bidirectional information interaction. Its core idea is to allow attention in both directions to occur simultaneously, rather than a unidirectional "query → queried" pattern.
[0023] Lighting-independent content refers to image information that does not change with lighting conditions, such as the edges, shapes, and outlines of objects, as well as the inherent color and texture of objects.
[0024] Lighting-related features refer to image features that change with lighting conditions. They directly encode the lighting effects in the scene, rather than the inherent properties of objects, such as the brightness and contrast of the image, the orientation of shadows, and the position of highlights.
[0025] This application provides a low-light image enhancement method based on unsupervised feature decoupling, such as... Figure 1 As shown, the method includes steps S101 to S105.
[0026] Step S101: Perform wavelet transform on the normal illumination image and the low illumination image to obtain the low-frequency sub-image and multiple high-frequency sub-images of each image, and perform inverse wavelet transform after bidirectional cross-attention operation to obtain the second low illumination image and the second normal illumination image.
[0027] The normal illumination images and low illumination images are acquired by industrial cameras at the construction site under normal illumination conditions and low illumination conditions, respectively, forming a first dataset containing multiple normal illumination images and a second dataset containing multiple low illumination images. Before proceeding to step S101, one normal illumination image is randomly selected from the first dataset, and one low illumination image to be enhanced is selected from the second dataset.
[0028] During step S101, the normal illumination image undergoes wavelet transform to obtain a corresponding low-frequency sub-image and multiple high-frequency sub-images; the low illumination image undergoes wavelet transform to obtain a corresponding low-frequency sub-image and multiple frequency sub-images. The low-frequency sub-images retain the main subject of the image, while the high-frequency sub-images retain details such as edges and textures.
[0029] In step S101, when performing bidirectional cross-attention operation on the low-frequency subgraph and each high-frequency subgraph, each high-frequency subgraph is fused with the corresponding low-frequency subgraph, realizing the interaction between low-frequency information and high-frequency information, and obtaining low-frequency features fused with high-frequency information, so that the low-frequency subgraph contains high-frequency details, and obtains high-frequency features fused with corresponding low-frequency information, so that high-frequency noise in the high-frequency subgraph is suppressed, and low-frequency structural constraints are added to the high-frequency subgraph, adaptively emphasizing high-frequency details.
[0030] Step S101 performs an inverse wavelet transform to convert each image from the frequency domain to the spatial domain, forming a second low-light image and a second normal-light image. Compared to the low-light image before processing in step S101, and compared to the normal-light image before processing in step S101, the second low-light image has a more balanced global brightness distribution, significantly enhanced edge and texture details, suppressed high-frequency noise, improved signal-to-noise ratio in dark areas, adaptively enhanced local contrast, increased visibility in shadow areas, and avoids loss of detail in highlight areas.
[0031] Step S102: Decouple both the second low-light image and the second normal-light image into light-independent content and light-dependent representation.
[0032] By executing step S102, the illumination-independent content and illumination-dependent representation of the second low-illuminance image are obtained, as well as the illumination-independent content and illumination-dependent representation of the second normal-illuminance image, providing a basis for subsequent cross-attention operations.
[0033] Step S103: Perform a cross-attention operation on the light-independent content decoupled from the second low-light image and the light-dependent representation decoupled from the second normal-light image, so that the light-independent content of the second low-light image is integrated with the light information of the second normal-light image; perform a cross-attention operation on the light-independent content decoupled from the second normal-light image and the light-dependent representation decoupled from the second low-light image, so that the light-independent content of the second normal-light image is integrated with the light information of the second low-light image.
[0034] Step S103 transfers the illumination-dependent representation of the second normal illumination image to the illumination-independent content of the second low illumination image, so that the illumination-independent content of the second low illumination image can be displayed under the illumination-dependent representation of the second normal illumination image; similarly, step S103 transfers the illumination-dependent representation of the second low illumination image to the illumination-independent content of the second normal illumination image, so that the illumination-independent content of the second normal illumination image can be displayed under the illumination-dependent representation of the second low illumination image.
[0035] Step S104: Generate a pseudo-low-light image based on the illumination-independent content of the second normal-light image that incorporates the illumination information of the second low-light image, and generate a pseudo-normal-light image based on the illumination-independent content of the second low-light image that incorporates the illumination information of the second normal-light image.
[0036] Step S105: Restore the pseudo low-light image to the normal light image, and restore the pseudo normal light image to the low-light image.
[0037] Compared to the original low-light image, the low-light image reconstructed from the pseudo-normal illumination image has higher image quality and can significantly improve the accuracy and reliability of visual detection when used as input.
[0038] The low-light image enhancement method based on unsupervised feature decoupling provided in this application can enhance low-light images without requiring "low-light-normal-light" training data during the enhancement process, thus solving the problem of difficulty in obtaining paired "low-light-normal-light" data in building construction scenarios.
[0039] In some embodiments of this application, step S101 involves performing wavelet transform on the normal illumination image and the low illumination image to obtain a low-frequency sub-image and multiple high-frequency sub-images for each image. Specifically, this includes performing wavelet transform on the normal illumination image and the low illumination image based on the Haar wavelet basis function to obtain a low-frequency sub-image and multiple high-frequency sub-images. The multiple high-frequency sub-images include sub-images of the horizontal edge frequency band, sub-images of the vertical edge frequency band, and sub-images of the diagonal edge frequency band.
[0040] Wavelet transform based on Haar wavelet basis functions requires less computation, thus allowing wavelet transforms of normal illumination images and low illumination images to be completed with relatively small computational resources.
[0041] The low-frequency sub-image is used to describe the main structure of the image, the horizontal edge frequency band sub-image is used to describe the vertical variation details of the image, the vertical edge frequency band sub-image is used to describe the horizontal variation details of the image, and the diagonal edge frequency band sub-image is used to describe the diagonal variation and texture details of the image.
[0042] In other embodiments of this application, other wavelet basis functions may also be used to perform wavelet transform on normal illumination images and low illumination images, such as the db4 wavelet basis function.
[0043] In some embodiments of this application, step S102 specifically includes steps S201 to S203.
[0044] Step S201: Extract shallow image features from the second low-light image and the second normal-light image.
[0045] The shallow features of the image include texture, edges, brightness, etc., which preserve local texture while avoiding premature loss of high-frequency details.
[0046] Step S202: Separate illumination-related features and content-related features from the shallow features of the image.
[0047] The shallow image features obtained in step S201 are still coupled. In step S202, the light-related features and light-independent features in the shallow image features are separated to obtain light-related features and content-related features.
[0048] Step S203: Map the illumination-related features and content-related features to illumination-related representations and illumination-independent content.
[0049] The illumination-related features and content-related features separated in step S202 are mapped to obtain illumination-related representations and illumination-independent content.
[0050] For example, the low-light image enhancement method based on unsupervised feature decoupling employs an encoder to implement steps S201 to S203, wherein the encoder includes multiple first convolutional layers, a convolutional block, and a third convolutional layer with a sigmoid activation function; the multiple first convolutional layers are used to perform step S201, extracting shallow image features from the second low-light image and the second normal-light image; the convolutional block includes multiple second convolutional layers with a ReLU activation function, used to perform step S202, separating illumination-related representations and illumination-independent content from the shallow image features; the third convolutional layer is used to perform step S203, mapping the illumination-related representations and illumination-independent content to illumination feature maps and content feature maps, respectively.
[0051] In some embodiments of this application, step S103 specifically includes steps S301 to S303.
[0052] Step S301: Map the illumination-related representation and illumination-independent content to the embedding space and add position encoding respectively.
[0053] Step S301 maps the illumination-related representation and illumination-independent content to the embedding space respectively, bringing them to the same semantic-geometric coordinate system, thus preparing data for subsequent calculations. Furthermore, it preserves their spatial relationships by adding positional encoding to the mapping of the illumination-related representation and illumination-independent content to the embedding space.
[0054] Step S302: Using the illumination-related representation of the second normal illumination image as the query and the illumination-independent content of the second low illumination image as the key and value, the illumination-independent content of the second low illumination image is fused with the illumination information of the second normal illumination image.
[0055] After executing step S302, the illumination-independent content of the second low-light image is fused with the illumination information of the second normal-light image, eliminating the grayscale phenomenon of the second low-light image and restoring the natural brightness levels.
[0056] Step S303: Using the illumination-related representation of the second low-illuminance image as the query and the illumination-independent content of the normal illumination image as the key and value, the illumination-independent content of the second normal illumination image is fused with the illumination information of the second low-illuminance image.
[0057] After executing step S303, the illumination-independent content of the second normal illumination image is fused with the illumination information of the second low illumination image, reducing the overexposure phenomenon of the second normal illumination image and maintaining the detail integrity of the highlight area.
[0058] In some embodiments of this application, step S105 specifically includes steps S401 to S404.
[0059] Step S401: Perform a wavelet transform-based cross-attention operation on the pseudo-normal illumination image and the pseudo-low illumination image to obtain the third normal illumination image and the third low illumination image, respectively.
[0060] Specifically, in step S401, wavelet transform is first performed on both the pseudo-normal illumination image and the pseudo-low illumination image to obtain the low-frequency sub-image and multiple high-frequency sub-images of the pseudo-normal illumination image, as well as the low-frequency sub-image and multiple high-frequency sub-images of the pseudo-low illumination image; then, a bidirectional cross-attention operation is performed on each low-frequency sub-image and its corresponding high-frequency sub-image to obtain the third normal illumination image and the third low illumination image.
[0061] Step S402: Decouple the third normal illumination image and the third low illumination image to obtain illumination-independent content and illumination-related representation.
[0062] By executing step S402, the illumination-independent content and illumination-dependent representation of the third low-illuminance image are obtained, as well as the illumination-independent content and illumination-dependent representation of the third normal-illuminance image, providing a basis for subsequent cross-attention operations.
[0063] Step S403: Perform a cross-attention operation on the illumination-independent content of the third normal illumination image and the illumination-dependent representation of the low illumination image, and perform a cross-attention operation on the illumination-independent content of the third low illumination image and the illumination-dependent representation of the normal illumination image.
[0064] Step S403 transfers the illumination-related representation of the third normal illumination image to the illumination-independent content of the third low illumination image, so that the illumination-independent content of the third low illumination image is integrated with the illumination information of the third normal illumination image; so that the illumination-independent content of the third low illumination image can be displayed under the illumination-related representation of the third normal illumination image; similarly, step S403 transfers the illumination-related representation of the third low illumination image to the illumination-independent content of the third normal illumination image, so that the illumination-independent content of the third normal illumination image can be displayed under the illumination-related representation of the third low illumination image.
[0065] Step S404: Generate a restored low-light image using the illumination-related representations of the third low-light image and the illumination-independent content of the third normal-light image; generate a normal-light image using the illumination-independent content of the third low-light image using the illumination-related representation information of the third normal-light image.
[0066] It should be noted that a composite loss function is designed to train the low-light image enhancement method based on unsupervised feature decoupling provided in this application embodiment. The composite loss function includes cycle consistency loss, perceptual loss, structural loss, and color constancy loss. Specifically, cycle consistency loss ensures that the restored image maintains pixel-level consistency with the original real image; perceptual loss aligns the features learned by the model with human perception; structural loss smooths noise reduction while maintaining edge clarity, effectively enhancing image sharpness; and color constancy loss ensures that the restored image has natural and accurate colors. The formula for calculating the composite loss function is as follows:
[0067] L total =L cyc +L percep +L struct +L cc ;
[0068] Among them, L total L represents the composite loss function. cyc L represents the cycle consistency loss. percep L represents perceived loss. struct L represents structural loss. cc This indicates a loss of color constancy.
[0069] The formula for the cycle consistency loss is as follows:
[0070]
[0071] in, This represents the normal illumination image restored in step S105. This represents the original normal illumination image input in step S101. This represents the low-light image restored in step S105. This represents the original low-light image input in step S101. Cyclic consistency loss ensures that the reconstructed image remains consistent with the original image.
[0072] The formula for perceived loss is as follows:
[0073]
[0074] in, This represents a pseudo-normal illumination image. This represents the pseudo-low-light image. φ(·) represents the feature representation obtained by concatenating the feature maps extracted from conv3_3 and conv4_3 of the VGG19 model. Perceptual loss aligns the generated pseudo-low-light image with the original image in semantic space and human perception.
[0075] The formula for structural loss is as follows:
[0076]
[0077] in, and These are the illumination-independent contents of the original illuminance image and the generated pseudo illuminance image input in step S101, respectively. and These are the illumination-related representations of the original illumination image and the generated pseudo illumination image input in step S101, respectively. When the gradient value of a certain region of the image changes significantly... The size will be relatively small, thus relaxing the lighting constraints in this region and avoiding over-smoothing of lighting-related representations. Structural loss maintains smooth global illumination while preserving local structural details, enhancing image sharpness.
[0078] The formula for color constancy loss is as follows:
[0079]
[0080] in, and The illumination-related characterizations of the original normal illumination image and the pseudo-normal illumination image input in step S101 are: and The illumination-related representations of the original low-light image and the pseudo low-light image are input in step S101. Color constancy loss causes... and and Maintaining a consistent light distribution throughout the space ensures that the restored image exhibits natural colors.
[0081] This application also provides a low-light image enhancement device based on unsupervised feature decoupling, such as... Figure 2 As shown, the low-light image enhancement device based on unsupervised feature decoupling includes: a two-way cross-attention module 201 based on wavelet transform, an image decoupling module 202, a cooperative cross-attention module 203, an image generation module 204, and a restoration module 205.
[0082] The wavelet-based bidirectional cross-attention module 201 is used to perform wavelet transform on normal illumination images and low illumination images to obtain low-frequency sub-images and multiple high-frequency sub-images for each image. After bidirectional cross-attention operation, inverse wavelet transform is performed to obtain a second low illumination image and a second normal illumination image.
[0083] The image decoupling module 202 is used to decouple both the second low-light image and the second normal-light image into light-independent content and light-dependent representation.
[0084] The collaborative cross-attention module 203 is used to perform cross-attention operations on the illumination-independent content decoupled from the second low-illuminance image and the illumination-related representation decoupled from the second normal-illuminance image; and to perform cross-attention operations on the illumination-independent content decoupled from the second normal-illuminance image and the illumination-related representation decoupled from the second low-illuminance image.
[0085] The image generation module 204 is used to generate a pseudo low-light image based on the illumination-independent content of the second normal illumination image that incorporates the illumination information of the second low-light image, and to generate a pseudo normal illumination image based on the illumination-independent content of the second low-light image that incorporates the illumination information of the second normal illumination image.
[0086] The restoration module 205 is used to restore a pseudo low-light image back to a normal-light image and a pseudo normal-light image back to a low-light image.
[0087] When the bidirectional cross-attention module 201 performs wavelet transform on normal illumination images and low illumination images to obtain low-frequency sub-images and multiple high-frequency sub-images for each image: it performs wavelet transform on normal illumination images and low illumination images based on Haar wavelet basis functions to obtain low-frequency sub-images and multiple high-frequency sub-images; among them, the multiple high-frequency sub-images include sub-images of horizontal edge frequency bands, sub-images of vertical edge frequency bands, and sub-images of diagonal edge frequency bands.
[0088] The image decoupling module 202 specifically includes multiple first convolutional layers, convolutional blocks, and a third convolutional layer with a sigmoid activation function. The first convolutional layers are used to extract shallow image features from the second low-light image and the second normal-light image; the convolutional block includes multiple second convolutional layers with ReLU activation functions, and the convolutional block is used to separate illumination-related features and content-related features from the shallow image features; the third convolutional layer is used to map the illumination-related features and content-related features into illumination-related representations and illumination-independent content.
[0089] The collaborative cross-attention module 203 is specifically used to: map the illumination-related representation and illumination-independent content to the embedding space and add position encoding respectively; use the illumination-related representation of the second normal illumination image as the query and the illumination-independent content of the second low illumination image as the key and value, so that the illumination-independent content of the second low illumination image is fused with the illumination information of the second normal illumination image; use the illumination-related representation of the second low illumination image as the query and the illumination-independent content of the normal illumination image as the key and value, so that the illumination-independent content of the second normal illumination image is fused with the illumination information of the second low illumination image.
[0090] like Figure 3 As shown, the restoration module 205 includes a wavelet transform-based bidirectional cross-attention module 2051 for restoration, an image decoupling module 2052 for restoration, a cooperative cross-attention module 2053 for restoration, and an image generation module 2054 for restoration.
[0091] The bidirectional cross-attention module 2051 for restoration performs wavelet transform-based cross-attention operations on the pseudo-normal illumination image and the pseudo-low illumination image to obtain the third normal illumination image and the third low illumination image, respectively. The image decoupling module 2052 for restoration decouples the third normal illumination image and the third low illumination image to obtain illumination-independent content and illumination-related representations. The cooperative cross-attention module 2053 for restoration performs cross-attention operations on the illumination-independent content of the third normal illumination image and the illumination-related representations of the third low illumination image, and also performs cross-attention operations on the illumination-independent content of the third low illumination image and the illumination-related representations of the third normal illumination image. The image generation module 2054 for restoration generates a restored low illumination image using the fused illumination-related representations of the third low illumination image and the illumination-independent content of the third normal illumination image, and generates a normal illumination image using the illumination-independent content of the third low illumination image, which incorporates the relevant representation information of the third normal illumination image.
[0092] exist Figure 2 and Figure 3 middle, This represents the normal illumination image input to the bidirectional cross-attention module 201. Input the low-light image of the bidirectional cross-attention module 201. This represents the second normal illumination image. This represents the second low-light image. and These represent the illumination-independent content and illumination-dependent characterization obtained by the image decoupling module 202 after decoupling the second normal illumination image, respectively. and These represent the illumination-independent content and illumination-dependent representation obtained by the image decoupling module 202 after decoupling the second low-light image, respectively. This represents a pseudo-normal illumination image. This represents a pseudo-low-light image. This represents the normal illumination image restored by the restoration module 205. This represents the low-light image restored by the restoration module 205. This represents the third normal illumination image. This represents the third low-light image. and These represent the illumination-independent content and illumination-dependent representation obtained by the image decoupling module 2052 after decoupling the third normal illumination image, respectively. and These represent the illumination-independent content and illumination-related characterization obtained by the image decoupling module 2052 after decoupling the third normal illumination image, respectively.
[0093] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. For ease of description, the above apparatus is described by dividing it into various modules according to their functions. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.
[0094] This application provides an electronic device, which includes: one or more processors; and a memory storing computer-executable instructions, which, when executed by the one or more processors, cause the one or more processors to perform the aforementioned low-light image enhancement method based on unsupervised feature decoupling.
[0095] This application provides a computer-readable storage medium storing computer-readable instructions. When executed by a computer, the computer-readable instructions implement the aforementioned low-light image enhancement method based on unsupervised feature decoupling.
[0096] The aforementioned storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), cache, hard disk drive (HDD), or memory card.
[0097] The various embodiments in this specification are described in a progressive manner. For the same or similar parts between the various embodiments, please refer to each other. Each embodiment focuses on describing the differences from other embodiments.
[0098] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.
Claims
1. A low-light image enhancement method based on unsupervised feature decoupling, characterized in that, include: Wavelet transform is performed on normal illumination images and low illumination images to obtain low-frequency sub-images and multiple high-frequency sub-images for each image. After bidirectional cross-attention operation, inverse wavelet transform is performed to obtain the second low illumination image and the second normal illumination image. Both the second low-light image and the second normal-light image are decoupled into light-independent content and light-dependent representation; A cross-attention operation is performed on the light-independent content decoupled from the second low-light image and the light-related representation decoupled from the second normal-light image, so that the light-independent content of the second low-light image is fused with the light information of the second normal-light image; A cross-attention operation is performed on the light-independent content decoupled from the second normal illumination image and the light-related representation decoupled from the second low illumination image, so that the light-independent content of the second normal illumination image is fused with the illumination information of the second low illumination image; The pseudo-low-light image is generated based on the illumination-independent content of the second normal-light image, which incorporates the illumination information of the second low-light image; and the pseudo-normal-light image is generated based on the illumination-independent content of the second low-light image, which incorporates the illumination information of the second normal-light image. The pseudo-low illumination image is restored to a normal illumination image, and the pseudo-normal illumination image is restored to a low illumination image.
2. The method according to claim 1, characterized in that, The step of performing wavelet transform on normal illumination images and low illumination images to obtain low-frequency sub-images and multiple high-frequency sub-images for each image includes: Based on the Haar wavelet basis function, wavelet transform is performed on the normal illumination image and the low illumination image to obtain the low-frequency sub-image and the plurality of high-frequency sub-images; wherein, the plurality of high-frequency sub-images include sub-images of horizontal edge frequency bands, sub-images of vertical edge frequency bands and sub-images of diagonal edge frequency bands.
3. The method according to claim 1, characterized in that, The step of decoupling both the second low-light image and the second normal-light image into light-independent content and light-dependent representation includes: Extract shallow image features from the second low-light image and the second normal-light image; Separate illumination-related features and content-related features from the shallow features of the image; The illumination-related features and the content-related features are mapped to illumination-related representations and illumination-independent content.
4. The method according to claim 1, characterized in that, The cross-attention operation is performed on the light-independent content decoupled from the second low-light image and the light-related representation decoupled from the second normal-light image, so that the light-independent content of the second low-light image is fused with the light information of the second normal-light image; A cross-attention operation is performed on the light-independent content decoupled from the second normal illumination image and the light-dependent representation decoupled from the second low illumination image, so that the light-independent content of the second normal illumination image is fused with the illumination information of the second low illumination image, including: The illumination-related representation and the illumination-independent content are respectively mapped to the embedding space and positional encoding is added; Using the illumination-related representation of the second normal illumination image as the query and the illumination-independent content of the second low illumination image as the key and value, the illumination-independent content of the second low illumination image is fused with the illumination information of the second normal illumination image; using the illumination-related representation of the second low illumination image as the query and the illumination-independent content of the normal illumination image as the key and value, the illumination-independent content of the second normal illumination image is fused with the illumination information of the second low illumination image.
5. The method according to claim 1, characterized in that, The step of restoring the pseudo-low-light image back to a normal-light image and restoring the pseudo-normal-light image back to a low-light image includes: A wavelet transform-based cross-attention operation is performed on the pseudo-normal illumination image and the pseudo-low illumination image to obtain a third normal illumination image and a third low illumination image, respectively. Decouple the third normal illumination image and the third low illumination image to obtain illumination-independent content and illumination-related characterization; A cross-attention operation is performed on the illumination-independent content of the third normal illumination image and the illumination-dependent representation of the third low illumination image; a cross-attention operation is also performed on the illumination-independent content of the third low illumination image and the illumination-dependent representation of the third normal illumination image. A restored low-light image is generated by fusing the illumination-related representation of the third low-light image and the illumination-independent content of the third normal-light image, and a normal-light image is generated by fusing the illumination-independent content of the third low-light image with the relevant representation information of the third normal-light image.
6. A low-light image enhancement device based on unsupervised feature decoupling, characterized in that, include: The bidirectional cross-attention module based on wavelet transform is used to perform wavelet transform on normal illumination images and low illumination images to obtain low-frequency sub-images and multiple high-frequency sub-images of each image. After bidirectional cross-attention operation, inverse wavelet transform is performed to obtain the second low illumination image and the second normal illumination image. An image decoupling module is used to decouple the second low-light image and the second normal-light image into light-independent content and light-dependent representation; A collaborative cross-attention module is used to perform cross-attention operations on the illumination-independent content decoupled from the second low-light image and the illumination-related representation decoupled from the second normal-light image. Perform a cross-attention operation on the illumination-independent content decoupled from the second normal illumination image and the illumination-related representation decoupled from the second low illumination image; An image generation module is used to generate the pseudo low-light image based on the illumination-independent content of the second normal illumination image that incorporates the illumination information of the second low-light image, and to generate a pseudo normal illumination image based on the illumination-independent content of the second low-light image that incorporates the illumination information of the second normal illumination image. The restoration module is used to restore the pseudo low-light image back to a normal light image, and the pseudo normal light image back to a low-light image.
7. An electronic device, characterized in that, include: One or more processors; as well as A memory storing computer-executable instructions, which, when executed by the one or more processors, cause the one or more processors to perform the low-light image enhancement method based on unsupervised feature decoupling as described in any one of claims 1 to 6.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a computer, implement the low-light image enhancement method based on unsupervised feature decoupling as described in any one of claims 1 to 5.