Low-rank diffusion and double-U compensation attention-based low-light image enhancement method

Abstract

The invention discloses a low-rank factor diffusion and double-U compensation attention-based low-light image enhancement method and system, belongs to the field of image processing and computer vision, and aims to solve the problems of image detail missing and structure distortion in a low-light scene. The method comprises the following steps: carrying out reduced-rank decomposition on an input image, modeling an image structure principal component through a low-rank diffusion model, and realizing detail reconstruction in combination with an inverse diffusion process of controllable noise; a double-branch nested U-shaped structure is adopted, one branch keeps the global structure consistency of the image, and the other branch enhances detail representation through a double-flow attention compensation mechanism; and fusing two branch results to generate a high-quality enhanced image. According to the method, redundancy calculation is reduced through low-rank modeling, the structure modeling capacity is enhanced through the diffusion process, collaborative optimization of the structure and details is achieved in combination with multi-scale attention, high-quality and low-complexity image enhancement in a weak light scene is supported, and the method is suitable for low-light environment perception application such as night monitoring, automatic driving and robot vision.

Description

Technical Field

[0001] This invention relates to a low-light image enhancement method and system based on low-rank diffusion modeling and a double U-shaped compensated attention mechanism. This technology can be widely applied in low-light scenarios such as nighttime video surveillance, autonomous driving, industrial image quality inspection, remote sensing image processing, medical image preprocessing, and mobile device image optimization. It aims to improve the visual quality and structural detail representation of images in low-light environments, providing a high-quality prior image foundation for downstream visual tasks. Background Technology

[0002] With the continuous development of computer vision and image enhancement technologies, low-light image enhancement has demonstrated broad value in various practical application fields. Especially in nighttime video surveillance, autonomous driving assistance systems, security, industrial quality inspection, and medical image preprocessing, the quality of low-light images directly impacts the system's recognition accuracy and user experience. In intelligent monitoring scenarios, improving the clarity and detail restoration of nighttime images helps enhance the reliability of anomaly detection and identity recognition. In autonomous driving, facing visual input under complex lighting conditions, enhancement algorithms help ensure the system's stable detection of key targets such as pedestrians and signs. In industrial inspection and quality control, effective image enhancement techniques can effectively identify subtle defects in low-light images, thereby improving production line efficiency and product quality. Simultaneously, in fields such as medical imaging, detail restoration of low-contrast images helps improve the accuracy and visualization level of clinical diagnosis.

[0003] However, existing low-light image enhancement methods still face several challenges in practical applications. First, traditional enhancement algorithms typically focus on brightness improvement, failing to effectively model the coupling relationship between image structure and texture, leading to artifacts, blurred details, or over-smoothing in the enhanced image. Second, while deep learning methods have achieved some success in low-light enhancement, most network structures have large parameters and high inference costs, making them difficult to deploy on mobile terminals or edge devices. Furthermore, current diffusion models still suffer from redundant diffusion interference and insufficient texture representation in image reconstruction, lacking a structure-aware mechanism to guide a more reasonable image restoration path. Simultaneously, the modeling of structure and detail is often mixed, lacking branch modeling and fusion strategies for backbone and texture information, resulting in uneven model performance across different image regions.

[0004] Therefore, there is an urgent need for a low-light image enhancement method that integrates low-rank structure modeling, controllable diffusion reconstruction, and multi-scale attention mechanisms. This method should be able to accurately represent the main structure of the image and meticulously restore texture details. While improving the subjective visual quality, it should also be lightweight, deployable, and have strong generalization capabilities to adapt to image enhancement tasks in multiple scenes under complex lighting conditions. Summary of the Invention

[0005] To address the technical challenges in low-light image enhancement, such as missing structural information, insufficient texture restoration, uncontrolled diffusion processes, and high model deployment complexity, this invention proposes a low-light image enhancement method and system based on low-rank diffusion and a double-U compensated attention mechanism. The aim is to construct an image enhancement scheme that is structure-aware, detail-oriented, inference-efficient, and flexibly deployable. This method integrates low-rank modeling and a controllable diffusion mechanism, effectively preserving main structural information while enhancing local texture representation. Furthermore, it achieves multi-scale feature fusion and attention guidance through a double-U nested network structure, making it particularly suitable for low-light applications such as nighttime visual enhancement, security monitoring, industrial image quality inspection, and edge computing terminals.

[0006] In a first aspect, the present invention provides a low-light image enhancement method based on low-rank diffusion and double-U compensated attention, comprising the following steps:

[0007] S1, acquire the low-light image input, divide the image into non-overlapping image blocks, and extract the low-rank structure representation using principal component decomposition;

[0008] S2, inject controllable Gaussian noise into the principal component features to construct a structure-aware positive degradation process;

[0009] S3, based on the inverse diffusion model, restores the structure and details of the image and generates a structure-aware degraded image representation;

[0010] S4 extracts structural features and texture details through structural convolution branches and detail convolution branches, respectively modeling the image backbone and local information;

[0011] S5, the above structural features and texture details are input into a double U-shaped nested network, and the inner and outer branches interact and fuse through a compensation attention mechanism to generate an enhanced image output.

[0012] Secondly, the present invention provides a low-light image enhancement system based on low-rank diffusion and double-U compensated attention, characterized in that it includes:

[0013] The image patch segmentation and low-rank modeling module is configured to receive low-light image input, divide it into several non-overlapping image patches, extract low-rank structural features using principal component analysis, retain the main structure information of the image and reduce redundant calculations.

[0014] The controllable noise injection module is configured to inject Gaussian noise with zero mean and adjustable variance into the aforementioned low-rank features to construct a controllable structure-aware diffusion degradation process, thereby improving the model's ability to model image structural perturbations.

[0015] The diffusion reconstruction module is configured to reverse recover the features of degraded images based on the inverse diffusion modeling framework, reconstruct the backbone structure and texture information, and generate structure-aware intermediate image representations.

[0016] The structure and detail feature extraction module is configured to extract the structural features and texture detail features of the image by using a structure convolution branch and a detail convolution branch, respectively. The two branches have different receptive fields and channel settings to achieve joint modeling of global structure and local details.

[0017] The double-U nested enhancement module is configured to input structural and texture features into a double-U nested network. Through the collaborative processing of the inner and outer U networks and the interaction and fusion of the compensation attention mechanism, it achieves deep integration of multi-scale semantic information and detailed texture, and finally outputs an enhanced image. It is suitable for low-light application scenarios such as night vision enhancement, industrial quality inspection and security image enhancement.

[0018] Thirdly, the present invention provides an image processing apparatus, characterized in that it includes one or more processors and a storage device communicatively connected thereto. The storage device pre-stores a computer program, which, when executed by the processor, enables the processor to implement the low-light image enhancement method as described in the first aspect of the present invention, including image block segmentation, low-rank feature extraction, controllable degradation modeling, structural reconstruction, and multi-scale fusion enhancement. The apparatus can be a single computer or server, possessing independent operating capabilities, and is suitable for image processing tasks in low-light environments.

[0019] Fourthly, the present invention also provides a computer-readable storage medium, characterized in that the storage medium stores a computer program, which, when executed by one or more processors, is capable of implementing all the steps of the low-light image enhancement method described in the first aspect of the present invention, and supports the complete execution flow of model loading and image enhancement functions.

[0020] This invention has the following technical advantages:

[0021] (1) Low-rank modeling and dimension reduction compression: The low-rank structure of the image is extracted by principal component decomposition, which effectively reduces redundant information and computation, enhances the ability to model the main structure of the image, and provides a stable structural expression for subsequent processing.

[0022] (2) Controllable diffusion modeling mechanism: Introducing adjustable Gaussian noise to construct a structure-aware degradation process, which makes the image enhancement process controllable and flexible in explicit modeling, and enhances the structural fidelity of the generated image;

[0023] (3) Structure-texture dual-branch design: Extract the backbone and texture features in parallel through structure convolution and detail convolution branches, effectively adapting to the challenges of edge blurring and texture degradation in low-light images;

[0024] (4) Double U-shaped nested compensation attention mechanism: Construct a double U-shaped network and introduce a compensation attention mechanism to achieve synergistic enhancement of global semantics and local details, thereby improving the final image's performance in terms of structural consistency and detail clarity;

[0025] (5) Lightweight and deployment-friendly: The entire enhancement process is designed to be modular and lightweight, making it suitable for deployment needs in resource-constrained scenarios, such as edge computing devices, industrial terminals and mobile image enhancement systems. Attached Figure Description

[0026] Figure 1 This is a flowchart of a low-light image enhancement method based on low-rank diffusion and double-U compensated attention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the following describes in detail, with reference to specific embodiments, a low-light image enhancement method and system based on low-rank diffusion and double-U compensated attention.

[0028] Example 1:

[0029] This embodiment provides an image enhancement method suitable for low-light environments, aiming to enhance detail information and texture quality while maintaining image structural consistency. The overall process includes the following steps:

[0030] S1: Image block segmentation and low-rank feature modeling; S2: Principal component perturbation and forward degradation construction; S3: Inverse diffusion restoration and structure reconstruction; S4: Structural and detail feature extraction; S5: Double U nested fusion and enhanced image generation.

[0031] Furthermore, the specific method of S1 is as follows:

[0032] S11: Acquire raw low-light image input, which may be from night shooting equipment, low-light monitoring or industrial quality inspection system.

[0033] S12: Divide the input image into several non-overlapping image blocks to improve the local modeling capability of subsequent processing.

[0034] S13: Perform principal component decomposition on each image patch, extract the main content, form a compact low-rank structure representation, compress redundant information, and retain the global skeleton structure.

[0035] Furthermore, the specific method of S2 is as follows:

[0036] S21: Injecting Gaussian noise with zero mean and controllable variance into low-rank features to construct a structure-aware positive degradation model.

[0037] S22: The noise amplitude can be dynamically set by the adjustment factor to simulate the degradation process of the image under complex low light conditions and improve the model's ability to perceive natural degradation trends.

[0038] Furthermore, the specific method of S3 is as follows:

[0039] S31: Input the degraded low-rank features into the inverse diffusion restoration module, perform inverse modeling step by step, denoise and reconstruct the structure and content of the image.

[0040] S32: Combine structure restoration strategy and texture completion strategy to generate intermediate image representation, providing structural prior for subsequent feature extraction and enhancement generation.

[0041] Furthermore, the specific method of S4 is as follows:

[0042] S41: Construct a structural convolution branch to extract global structural features of the image, such as edges, contours, and region consistency.

[0043] S42: Constructs a detail convolution branch, focusing on extracting high-frequency texture features, such as texture noise, reflection points, and complex detail areas.

[0044] S43: The two branches use different receptive field configurations and channel numbers to adapt to global representation and local enhancement tasks respectively, and finally output structural feature and detail feature vectors.

[0045] Furthermore, the specific method of S5 is as follows:

[0046] S51: Input structural and detailed features into a double-U nested augmented network model. The model contains two paths: an outer U structure and an inner U structure.

[0047] S52: The outer U structure is responsible for maintaining the overall structural consistency of the image, while the inner U structure introduces a compensation attention mechanism to enhance the texture recovery capability of local areas.

[0048] S53: The two achieve information fusion through the dual-stream compensation attention module, realizing deep collaboration between the semantic layer and the detail layer.

[0049] S54: The final output image is enhanced, with high clarity, contrast and texture performance, and is suitable for a variety of low-light application scenarios such as night monitoring, industrial quality inspection and low-light shooting.

[0050] Example 2:

[0051] To further illustrate the low-light image enhancement method based on low-rank diffusion and double-U compensated attention proposed in this invention, the following combines mathematical expressions and... Figure 1 The steps described in Example 1 will be explained in detail.

[0052] Step 201:

[0053] Let the input image be a three-channel low-light image X∈R H×W×3 Here, H and W represent the height and width of the image, respectively. To enhance the local structure modeling capability of the image, the input image is divided into several non-overlapping image blocks of size p×p, where p is the side length of the image block. After division, a total of M image blocks {X1, X2, ..., X...} are obtained. i ,…,X M Each image patch X i ∈R p×p×3 Flattened into vector form x i ∈R K Where K = 3 × p 2 This represents the feature dimension after flattening. All image patches form the sample matrix X. mat ∈R M×K First, the sample matrix is ​​centered, and its covariance matrix is ​​calculated, as follows:

[0054]

[0055] Where, μ∈R 1×K Let C represent the sample mean vector, where C ∈ R. K×K Let C be the covariance matrix of the image patch features. Then, eigenvalue decomposition is performed on the covariance matrix C to obtain the eigenvector matrix V∈R. K×K The first k principal components are selected to form the feature projection matrix P∈R. K×k This is used to map high-dimensional image patches to a low-dimensional structured subspace. The mapping process is as follows:

[0056] z i =x i P, i = 1, 2, ..., M

[0057] Among them, z i ∈R k Let z be the low-rank structure representation of the i-th image patch. i The set {z1, z2, ..., z} constitutes i ,…,z M This is used to characterize the backbone structure features of an image, reduce redundant information, and improve the expressive stability of subsequent degradation modeling.

[0058] Step 202: For the low-rank principal component features z obtained in step S1 i A structure-aware positive degradation process is constructed by injecting Gaussian noise with zero mean and adjustable standard deviation to simulate the gradual degradation trend of image structural information under low-light conditions. The mathematical form of this process is shown below:

[0059]

[0060] in, ε represents the degraded structure-aware feature representation. σ Let represent a random disturbance term that follows a Gaussian distribution with a mean of zero and a variance of σ. 2 The noise covariance matrix is ​​σ 2 I, where I is the identity matrix and σ is a controllable noise intensity parameter used to adjust the degree of perturbation. This degradation method performs explicit perturbation modeling on the low-rank feature space, which can preserve image skeleton information and simulate its attenuation process under different illumination conditions without introducing redundant information, providing structure-aware noise-guided priors for the subsequent inverse diffusion reconstruction process.

[0061] Step 203: For the perturbation feature sequence obtained in step S2 Using the principal component inverse transform operator P ―1 (·) Map the low-rank latent representation back to the image domain to construct a degenerate image representation with structure-aware properties. The specific form is shown below:

[0062]

[0063] Among them, P ―1 (·) denotes the corresponding inverse principal component projection operation, used to restore low-rank features to the original image representation space. This represents the structure-aware degradation image generated from the i-th image after low-rank perturbation and inverse transform processing. This process generates an intermediate image with global contour preservation and local degradation features by preserving low-rank principal structure information and fusing controllable perturbation signals, providing a balanced image foundation for subsequent structural and detail feature extraction.

[0064] Step 204: For the structure-aware degradation image generated in step S3 Two depthwise separable convolutional branches (DSConv) are constructed to extract structural features and detail texture information from the image, respectively. The two branches have different receptive fields and channel configurations: the structural branch focuses on global contours and backbone structures, while the detail branch models local textures, edges, and high-frequency information. The feature extraction process can be represented as follows:

[0065]

[0066] Among them, DSConv str (·) indicates a structural convolution branch, DSConv det (·) indicates the detail convolution branch. Both branches use depthwise separable convolutional structures to improve computational efficiency and feature discrimination capabilities. Represents the structural feature tensor. This represents a detail feature tensor, used to characterize the main layout and local detail texture features of an image, respectively.

[0067] Step 205: To further enhance the ability to express structural consistency and texture details in low-light images, this invention proposes a dual-branch nested U-shaped enhancement module (DBNUM) to fuse the aforementioned structural features. With detailed features Cross-branch collaborative enhancement is achieved through a compensatory attention mechanism. The final enhanced image is represented as follows:

[0068]

[0069] in, This is the enhanced image, f DBNUM (·) represents a double U-shaped nested augmented network module, consisting of an outer branch and an inner branch, employing a symmetrical encoder-decoder structure, and introducing a dual-stream compensated attention mechanism at each level for interactive fusion. f DBNUM (·) consists of the following three parts:

[0070] (1) Outer U-branch

[0071] This branch As input, semantic structure features are extracted layer by layer. Initially, we set... for Its propagation process is defined as:

[0072]

[0073] in, This represents the structural features of the j-th layer of the outer branch; and These represent the upsampling and downsampling modules, respectively, and multi-scale hierarchical modeling ensures the stable transmission of image backbone structural information.

[0074] (2) Inner U-branch

[0075] This branch As input, the DSCAT module is also introduced to enhance the modeling capability of local textures. Initially, we set... for Its upsampling phase is represented as:

[0076]

[0077] The downsampling phase is represented as:

[0078]

[0079] in, and These are the upsampling and downsampling modules for detailed features, respectively. This represents the output of the detail branch at level j. DSCAT(·) is the DSCAT module, and ReLU(·) is the activation function.

[0080] (3) Compensatory Attention Module (DSCAT)

[0081] DSCAT aims to enhance the interaction and compensation between structural branches and detailed branches. Firstly, it represents the external branches. Intermediate features of inner branches Normalize and project onto the Q, K, V space to obtain and We perform cross-branch modulation by adaptively guiding the value features of one branch's query to the other:

[0082]

[0083] Then, cross-branch self-attention weighting is performed:

[0084]

[0085] Where Norm(·) represents L2 normalization, γ out and γ in As an adjustable scaling factor, the outputs of each header are concatenated and then linearly mapped and positionally encoded to obtain a fused representation:

[0086]

[0087] The final fused output is represented as:

[0088]

[0089] The final output features of the inner branch are generated by connecting the feedforward network with the residual:

[0090]

[0091] Finally, the output of the last layer is obtained. That is, the enhanced image

[0092] Example 3:

[0093] This embodiment provides a low-light image enhancement device based on low-rank diffusion and double-U compensated attention, suitable for image quality improvement tasks in low-light scenarios such as night surveillance, low-light quality inspection, and security image enhancement. The device includes at least one processor and a memory communicatively connected to it. The memory stores computer program instructions executable by the processor to implement the following functional modules:

[0094] Image Patch Segmentation and Low-Rank Modeling Module: This module receives low-light image input and divides the image into several non-overlapping image patches. Principal component analysis is then performed based on the covariance matrix to extract the top K dominant components, generating a compact low-rank structural feature representation. This significantly reduces redundant computation while preserving the main structural information of the image, enhancing the model's ability to express the global structure.

[0095] Controllable noise injection module: This module injects Gaussian noise with zero mean and adjustable variance into the aforementioned low-rank features to simulate the degradation process of images in real low-light environments, constructing a structure-aware forward diffusion model. The noise level can be dynamically set by adjusting parameters, thereby enhancing the model's degradation modeling ability and adaptability to complex low-light image characteristics.

[0096] Diffusion Reconstruction Module: This module is based on an inverse diffusion modeling strategy to achieve progressive reverse modeling of degraded features. Through a multi-step reconstruction process, it restores the backbone structure and texture edges of the image, generating a structure-aware intermediate image representation, providing a stable prior for subsequent feature extraction and fusion enhancement.

[0097] Structure and Detail Feature Extraction Module: This module includes a structure convolution branch and a detail convolution branch, which model the global shape and local texture of the image, respectively. The structure branch focuses on extracting global information such as edges and contours, while the detail branch captures local details such as high-frequency textures and fine noise. The two branches employ heterogeneous receptive fields and channel configurations to enhance the system's ability to analyze feature information at different scales.

[0098] The double-U nested enhancement module: This module fuses the aforementioned structural and detail features, achieving multi-scale and multi-level information enhancement through a double-U nested network. The outer U structure ensures the consistency of image semantics and structure, while the inner U structure introduces a compensating attention mechanism to enhance the expressive power of detail texture. The two structures are collaboratively fused through a dual-stream compensating attention module, ultimately outputting an enhanced image with good structural fidelity and detail clarity.

[0099] Example 4:

[0100] The fourth embodiment provided in this application is a low-light image enhancement device that can be deployed on a single server, industrial edge computing platform, or image processing terminal, supporting real-time or batch image enhancement tasks. Each functional module can operate independently or be integrated and packaged into a unified image processing framework, featuring lightweight, modularity, and scalability. It is suitable for applications such as industrial quality inspection under low light conditions, nighttime visual analysis, and security image preprocessing, significantly improving the system's perception capability and image quality under complex lighting conditions.

Claims

1. A low-light image enhancement method based on low-rank diffusion and double-U compensated attention, characterized in that, Includes the following steps: S1, acquire the low-light image input, divide the image into non-overlapping image blocks, and extract the low-rank structure representation using principal component decomposition; S2, inject controllable Gaussian noise into the principal component features to construct a structure-aware positive degradation process; S3, based on the inverse diffusion model, restores the structure and details of the image and generates a structure-aware degraded image representation; S4 extracts structural features and texture details through structural convolution branches and detail convolution branches, respectively modeling the image backbone and local information; S5, the above structural features and texture details are input into a double U-shaped nested network, and the inner and outer branches interact and fuse through a compensation attention mechanism to generate an enhanced image output.

2. The low-light image enhancement method based on low-rank diffusion and double-U compensated attention as described in claim 1, characterized in that, In step S1, the input image is divided into several non-overlapping image blocks, and the top K dominant structural components are extracted based on principal component analysis of the covariance matrix to construct a low-rank structural feature representation, thereby reducing redundant calculations and preserving the main structural information of the image.

3. The low-light image enhancement method based on low-rank diffusion and double-U compensated attention according to claim 1, characterized in that, In step S2, Gaussian noise with zero mean and controllable variance is added to the principal component features to construct a structure-aware forward diffusion model; the diffusion noise level is controlled by preset adjustment parameters to achieve controllable perturbation degradation modeling of image structure content.

4. The low-light image enhancement method based on low-rank diffusion and double-U compensated attention according to claim 1, characterized in that, In step S3, the inverse diffusion reconstruction module is used to reverse model the features of the degraded image, and the structure restoration and texture completion are combined to generate a structure-aware intermediate image representation, providing prior support for subsequent detail enhancement.

5. The low-light image enhancement method based on low-rank diffusion and double-U compensated attention according to claim 1, characterized in that, In step S4, structural features and texture features are extracted through structural convolutional branches and detail convolutional branches, respectively. These convolutional branches are depthwise separable convolutional structures with different receptive fields and channel configurations to adapt to global and local feature extraction tasks.

6. The low-light image enhancement method based on low-rank diffusion and double-U compensated attention according to claim 1, characterized in that, In step S5, structural features and detail features are input into a double-U nested network. The outer U network is used to maintain the overall structural consistency of the image, while the inner U network introduces a compensation attention mechanism to enhance the expression of local texture details. The two achieve the synergistic fusion of structural and detail information through a dual-stream compensation attention module.

7. A low-light image enhancement system based on low-rank diffusion and double-U compensated attention, characterized in that, This system integrates multiple functional modules to achieve low-light image enhancement processing, suitable for low-light scenarios such as nighttime visual enhancement and industrial image quality inspection. Specifically, it includes: an image block segmentation and low-rank modeling module, a controllable noise injection module, a diffusion reconstruction module, a structural and detail feature extraction module, and a double-U nested enhancement module.

8. An image processing apparatus for low-light image enhancement, characterized in that, The device includes one or more processors and a memory, wherein the memory stores one or more computer programs. When the computer programs are executed by the processor, the processor performs the method steps as described in any one of claims 1 to 6, specifically including: dividing a low-light image into image blocks and extracting low-rank features of principal components; injecting controllable Gaussian noise into the principal components to construct a degradation process; reconstructing structural information using an inverse diffusion network; extracting structural and texture features through structural convolution and detail convolution branches; and finally inputting the extracted features into a double-U nested network, fusing them through a compensated attention mechanism to generate an enhanced image. The device supports operation on a single computer or server, has low deployment costs and high processing efficiency, and is suitable for image enhancement tasks in low-light scenes.

Images