A low-light image enhancement method and system based on edge extraction and feature fusion

Abstract

The application discloses a low-light image enhancement method and system based on edge extraction and feature fusion, aiming at improving image quality under low-light conditions, the method comprises differential convolution edge extraction, feature fusion and image enhancement, the differential convolution kernel effectively extracts image edge information, the feature fusion module combines global and local features, and the image enhancement module uses a deep learning network to improve image brightness and suppress noise, the application is suitable for high-performance computing environments and resource-limited embedded devices, and experiments prove that the application performs excellently on multiple data sets, significantly improves image definition and details, and is suitable for security monitoring, automatic driving and other fields, and has a wide application prospect.

Description

Technical Field

[0001] This invention relates to a low-light image enhancement method and system based on edge extraction and feature fusion, and more specifically, to a low-light image enhancement method and system based on an edge extraction module, a feature fusion module, and an image enhancement module, belonging to the field of low-light image enhancement. Background Technology

[0002] In low-light environments, image quality deteriorates significantly due to insufficient illumination, manifesting as loss of detail, increased noise, and color distortion. This degradation poses a serious challenge to computer vision tasks such as object detection, image segmentation, and scene understanding, particularly in critical fields like autonomous driving, security monitoring, and medical imaging, where high-quality image input is crucial for system performance. Therefore, developing a low-light image enhancement technique that can effectively restore image details, improve contrast, and suppress noise is an urgent practical need.

[0003] Traditional methods, such as histogram equalization and gamma correction, are easy to implement, but they are not good at restoring details and may cause problems such as over-enhancement or noise amplification. At the same time, their generalization ability is limited and they are difficult to adapt to diverse low-light scenes.

[0004] With the rise of deep learning technology, low-light image enhancement methods based on convolutional neural networks (CNNs) have gradually become mainstream. These methods, with their end-to-end learning mechanisms, can automatically learn the mapping from low-light to normal-light images from massive amounts of data, significantly improving the enhancement effect. However, current deep learning methods often improve performance by increasing network size and complexity, leading to a sharp increase in the number of model parameters and computational complexity, making it difficult to deploy and process in real time on resource-constrained edge devices. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, this invention proposes a low-light image enhancement method based on an edge extraction module, a feature fusion module, and an image enhancement module. By introducing an edge extraction module, edge and texture information of the image is effectively captured, providing crucial structural features for subsequent enhancement. The feature fusion module further enhances feature representation capabilities by combining low-level details and high-level semantic information through a multi-scale feature fusion strategy. The image enhancement module utilizes the fused features for nonlinear mapping to generate a high-quality enhanced image. This invention significantly reduces the computational complexity of the model while maintaining enhancement effectiveness, making it suitable for efficient operation on cloud servers and embedded edge devices. It also meets the demands for computational efficiency and deployment flexibility in practical applications while improving image quality.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0007] This invention discloses a low-light image enhancement method based on edge extraction and feature fusion, the method comprising the following steps:

[0008] Step 1) Extract edge and gradient information from the image using multi-directional differential convolution kernels to generate a multi-dimensional edge feature map;

[0009] Step 2) The edge feature map is fused with the original low-light image features at multiple levels, combining global semantics and local details to provide a richer feature representation for image enhancement;

[0010] Step 3) Perform non-linear mapping on the image using a deep learning network to output the enhanced image;

[0011] Step 4) Real-time processing on embedded devices is achieved through model quantization and hardware acceleration. Step 4) includes: deploying on NVIDIA Jetson Nano edge devices, converting PyTorch models to ONNX format, and then optimizing them into an FP16 precision execution engine using TensorRT; the real-time processing flow is: camera input; Jetson Nano enhanced computation; USB interface output to LED screen, with a latency of less than 50ms@720p; hardware acceleration methods include: parallelizing differential convolution calculations using CUDA kernel functions; and accelerating sparse attention matrix operations using TensorCore.

[0012] Preferably, in step 3), a lightweight Transformer network (a deep learning model based on a self-attention mechanism) is used to perform nonlinear mapping on the fused features.

[0013] Preferably, step 1) specifically includes:

[0014] Three types of differential convolution kernels are used: circular, radial, and cross-shaped. The formula for calculating the circular differential convolution is as follows:

[0015] ;

[0016] In the above formula, W and X represent the learnable parameters of the convolution kernel and the feature map being convolved, respectively, y represents the output of the differential convolution, and the subscripts indicate the row and column in the 3×3 region, where x 11 This represents the first row and first column of a 3×3 region; the extracted gradient information is subjected to multi-scale enhancement processing, including a compact dilated convolution module and a compact spatial attention module. The compact dilated convolution module uses convolution kernels with dilation rates of 1, 2, and 3, and the compact spatial attention module generates a dynamic weight mask through the Sigmoid activation function.

[0017] Preferably, step 2) includes: a dual-branch structure is used for feature fusion, the first branch performs 3×3 convolution on the original image to extract low-level features F1, and the second branch performs 2-layer convolution on the edge map to extract high-level features F2;

[0018] The fusion formula is: F fuse =α·F1+(1-α)·F2, where α is dynamically calculated from the variances of F1 and F2. The calculation formula is: α=Var(F1) / (Var(F1)+Var(F2)). The feature distribution is dynamically adjusted through spatial adaptive normalization, and the input semantic segmentation map generates normalization parameters μ and σ.

[0019] Preferably, step 3) specifically involves: adopting a four-layer UNet structure, with the encoder using stride 2 convolutions to downsample to 1 / 16 resolution; each layer contains a Transformer module based on a sparse attention mechanism, which divides the feature map into an 8×8 window for local self-attention calculation; the decoder gradually restores the resolution through transposed convolutions and injects edge feature maps via skip connections.

[0020] This invention discloses a system for low-light image enhancement based on edge extraction and feature fusion, comprising a data-connected edge extraction module based on differential convolution, a feature fusion module, and an image enhancement module. In step 1), the edge extraction module based on differential convolution effectively extracts edge information from the image by utilizing multiple differential convolution kernels in different directions. In step 2), the feature fusion module fuses the edge features output by the edge extraction module with the features of the original image. Through a multi-level fusion strategy, it effectively combines global information and local details of the image, thereby providing richer feature representations for image enhancement. In step 3), the image enhancement module performs image enhancement processing using the fused features. By applying a non-linear mapping to the image through a deep learning network, it can enhance image brightness while effectively suppressing noise and artifacts, thus improving the visual quality of the image.

[0021] Preferably, the operation mode of the edge extraction module based on differential convolution includes the following:

[0022] S1) Multi-dimensional edge information extraction branch: Differential convolution is used to extract edge information of different dimensions. Differential convolution explicitly encodes the prior information contained in traditional edge detection operators into the convolutional layer through a carefully designed pixel gradient calculation strategy, which can extract complex image gradient information more flexibly. Differential convolution first calculates the gradient corresponding to each pixel according to the manually pre-designed gradient calculation strategy, and then uses a learnable convolutional kernel to convolve the calculated gradient. The gradient information extracted by a single differential convolution according to its corresponding pixel gradient calculation strategy is limited. In order to simultaneously pay attention to local edge information and global edge information, this invention uses differential convolution to extract edge information of different dimensions and designs different differential convolutions to extract multiple gradient information.

[0023] Furthermore, taking a 3×3 circular difference convolution as an example, it first calculates the gradient between adjacent pixels in a clockwise direction, and then convolves the calculated gradient information through a learnable convolutional kernel. The corresponding calculation formula is as follows:

[0024] ;

[0025] In the above formula, W and X represent the learnable parameters of the convolution kernel and the feature map being convolved, respectively; y represents the output of the differential convolution; the subscripts indicate the row and column in the 3×3 region; and x... 11 This represents the first row and first column of a 3x3 area;

[0026] S2) Edge Information Processing Branch: This branch further processes the obtained edge information to better enhance the texture and detail features of the image. In this invention, the edge information processing branch further processes the extracted edge information by introducing a compact dilated convolution module and a compact spatial attention module. The compact dilated convolution module uses dilated convolution to extract and fuse multi-scale edge features, enhancing edge expressiveness; the compact spatial attention module dynamically adjusts feature weights through a spatial attention mechanism to highlight key edge regions. The combination of these two significantly improves the texture and detail features of the image, providing high-quality edge information support for subsequent image enhancement.

[0027] S3) Multi-dimensional edge information fusion: The multi-dimensional edge information extracted from the edge information processing branch is processed and fused. The fused feature map is output as an image gradient map after passing through the Sigmoid activation function.

[0028] Preferably, the operation mode of the feature fusion module includes the following:

[0029] S1) Feature preprocessing branch: Features are extracted and processed from the edge map and the input image through a combination of conventional convolutional layers and the Sigmoid activation function;

[0030] S2) Feature Fusion Attention Mechanism: The feature fusion attention mechanism effectively fuses gradient images (such as edge information) with input features through convolution and normalization operations, thereby enhancing the image's feature representation capabilities. Specifically, this module first performs convolution operations on the gradient image and input features respectively to extract their respective multi-level features. Next, normalization is applied to the features to ensure the stability of the feature distribution. Finally, the processed gradient features are weighted and fused with the input features to generate a fused feature map. This fusion method not only preserves the global information of the input image but also incorporates detailed and texture features from the gradient image, thus providing richer feature representations for subsequent image enhancement.

[0031] S3) Feature Normalization: Image features are normalized through spatial adaptive normalization. By introducing a semantic layout map as input, the parameters of the normalization layer (such as mean and variance) are dynamically adjusted to better preserve the input semantic information when generating the image. Unlike traditional normalization methods, spatial adaptive normalization can adaptively adjust the normalization parameters of the feature map according to the spatial location of the input semantic map, thereby generating higher quality images.

[0032] Preferably, the network structure in the image enhancement module adopts a UNet-style network structure, which is divided into three parts: encoder, feature transformation module and decoder; the network has a total of four levels, each level representing a different resolution, and each level is composed of stacked basic modules.

[0033] We will reduce computational complexity by introducing sparsity improvements to the Transformer (a deep learning model based on self-attention mechanism) while maintaining the model's high performance. The sparse attention mechanism only calculates the correlation between local regions or key locations, which significantly reduces the amount of computation. We will limit the attention range by using fixed patterns (such as local windows or strip regions) or dynamic learning (such as content-based sparsification), thereby reducing computational overhead while maintaining performance.

[0034] The present invention discloses an edge computing device based on a low-light image enhancement method using edge extraction and feature fusion, comprising an edge computing module, a video input module, and a result output module connected by communication;

[0035] Edge computing module: Utilizes NVIDIA Jetson Nano for real-time low-light image enhancement calculations; specifically: Jetson Nano employs an NVIDIA Maxwell architecture with a quad-core Cortex A57 MP Core processor and 128 CUDA cores, enabling complex neural network calculations to be performed using only 5 watts of power;

[0036] Video input module: Uses a camera to capture images as input for low-light images;

[0037] Results output module: A portable LED screen powered by USB for outputting and displaying results;

[0038] The following steps will be used to deploy a deep learning model trained on a server cluster on an edge computing device:

[0039] Convert neural network models trained using the deep learning framework PyTorch into the general deep learning format ONNX; convert ONNX format neural network models into TensorRT format models. TensorRT is a C++ inference framework for NVIDIA GPUs that enables high-performance inference on GPUs.

[0040] TensorRT is used to optimize the pre-trained model into an execution engine through quantization, operator fusion, automatic kernel adjustment, dynamic tensor memory, and multi-stream execution. Then, the Python interface is used to call the execution engine TensorRT to perform low-light image enhancement calculations for the neural network.

[0041] Beneficial effects: This invention has the technical characteristics of significantly reducing the computational complexity of the model while ensuring the enhancement effect, making it suitable for efficient operation on cloud servers and embedded edge devices, and meeting the needs of practical applications for computational efficiency and deployment flexibility while improving image quality; This invention has significant performance advantages in low-light image enhancement tasks, effectively improving image clarity and detail, and is applicable to multiple fields such as security monitoring and autonomous driving. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the entire invention.

[0043] Figure 2 These are diagrams illustrating different types of dilated convolution patterns of this invention.

[0044] Figure 3 This is a diagram of the dilated convolution module of the present invention.

[0045] Figure 4 This is a diagram of the channel and spatial attention module of the present invention.

[0046] Figure 5 This is a diagram of the multi-void convolution module of the present invention.

[0047] Figure 6 This is one of the detailed diagrams of the image enhancement module of the present invention.

[0048] Figure 7 This is the second detailed diagram of the image enhancement module of the present invention.

[0049] Figure 8 This is the third detailed diagram of the image enhancement module of the present invention. Detailed Implementation

[0050] The following will refer to the accompanying drawings in the embodiments of the present invention. Figures 1 to 8 The technical solutions in the embodiments of the present invention are clearly and completely described herein. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0051] Figure 1 The algorithm comprises three parts: a, b, and c. Part a presents the overall workflow framework for low-light image enhancement. After inputting an image, it sequentially passes through a "residual module," a "transform block + structure-aware fusion block" (multiple sets in series), and then back to the "residual module," finally outputting the enhanced image. This demonstrates the multi-module sequential processing flow from the original image to the enhanced image, with each structure-aware fusion block and transform block working together to progressively optimize image features. Part b is the edge extraction and preliminary fusion sub-process. After feature extraction by the "pyramid dilated convolution module" (multi-level), the features are processed by the "channel-dimensional convolution module" and the "channel-space attention module." The outputs of these two modules are then "concatenated" and passed through the "Sigmoid activation function" to finally output the processed image. This mainly achieves feature extraction and preliminary fusion based on pyramid dilated convolution and attention mechanisms. Part c is the feature fusion and transformation sub-process. After the "feature map" is input, it undergoes operations such as "convolution, residual block, convolution", and then passes through "fusion block". Combined with processing such as "corrected linear unit + convolution" and "spatial adaptive normalization + corrected linear unit", a new "feature map" is output through "convolution", focusing on the deep fusion and transformation of features to optimize image feature representation.

[0052] Figure 2 This demonstrates the operational logic of three differential convolution kernels. Each group contains the original pixel matrix and the resulting matrix: Group 1: A 3×3 original pixel matrix (x1-x9), which, after processing, becomes a new 3×3 matrix with a center position of 0 and the remaining positions representing the original pixel differences (e.g., x1-x5). This illustrates one differential calculation method used to extract specific gradient information from the image. Group 2: A 3×3 original pixel matrix (x1-x9, with a blank center x5), after processing, has a center position of 0 and the remaining positions representing the original pixel differences (e.g., x1-x2). This presents another pixel difference logic, capturing edge gradients in different directions. Group 3: A 5×5 original pixel matrix (x1-x25, including blank pixels), after processing, has a center position of 0 and the remaining positions representing the original pixel differences (e.g., x1-x7). This is used to extract image gradients in more complex scenes, adapting to edge extraction of images of different sizes and structures. Figure 3 The processing flow for sub-blocks / functional blocks is as follows: The input is sequentially passed through "sub-block / functional block_1" and "sub-block / functional block_2", then subjected to "3×3 convolution kernel dilated convolution operation", followed by "corrected linear unit" and "1×1" convolution, and added to the branch signal from "sub-block / functional block_2", and then passed through "sub-block / functional block_3" and "sub-block / functional block_4", presenting the feature processing and residual connection logic within the local module, which is used to optimize feature extraction and transmission and enhance the module's ability to process image features.

[0053] Figure 4 The process involves fusion of attention mechanisms and convolutions: the input passes through a "corrected linear unit," a "1×1 convolution," a "3×3 convolution," and a "Sigmoid activation function," and is multiplied by the original input branch before output. This process first reduces the dimensionality of the input features and extracts local features, then generates attention weights through activation functions, which are then fused with the original features to highlight key features, suppress irrelevant information, and improve the specificity of feature representation.

[0054] Figure 5 The multi-scale dilated convolution process is as follows: the input passes through a "corrected linear unit" and then a "1×1 convolution," which branches into four "3×3 convolutions (with dilation rates of 5, 7, 9, and 11)," and finally the results of the four branches are summed for output. By using dilated convolutions with different dilation rates, multi-scale image features are extracted, and the fused features enrich the representation, adapting to the feature extraction needs of targets or scenes of different sizes.

[0055] Figure 6 This demonstrates the feature processing flow related to the Transformer: the input undergoes "layer normalization" and "sparse self-attention with token kernel awareness," and is then added to the original input branch; it then undergoes "layer normalization" and "multi-scale feature network," and is added to the input branch of this stage before output. Layer normalization stabilizes the feature distribution, sparse self-attention reduces computation, and the multi-scale feature network extracts rich features, achieving efficient feature transformation and enhancement, and adapting to deep processing of image features.

[0056] Figure 7 This is an attention-based process based on depthwise separable convolution: the input passes through three parallel branches ("1×1 + 3×3 depthwise separable convolution + dimensionality reshaping") to generate Q, K, and other features. These features are then processed through "masking," "sparsening," "restoring the feature space / channel layout," and "Softmax activation function," multiplied by the original input branch, and finally output through "dimensionality reshaping + 1×1 convolution." By leveraging depthwise separable convolution for dimensionality reduction and combining it with an attention mechanism to focus on key features, this process optimizes feature extraction and fusion, enhancing the ability to capture important image features.

[0057] Figure 8The cross-fusion convolution process works as follows: the input passes through two parallel branches ("1×1 + 3×3 convolution to extract local features + corrected linear units"), the branch results are cross-fused (two "C" nodes), then through two more parallel "3×3 convolutions to extract local features + corrected linear units", and cross-fused again ("C" nodes), finally outputting through a "1×1" process. By repeatedly cross-fusioning the local features extracted from different branches, feature interaction and integration are strengthened, feature richness and expressive power are improved, and image feature processing effects are optimized.

[0058] The technical solution / principle of this invention addresses the problem of image quality degradation under low-light conditions affecting computer vision tasks. It proposes a low-light image enhancement method based on edge extraction, feature fusion, and image enhancement modules. The low-light image enhancement method mainly includes four key steps: "edge extraction → feature fusion → nonlinear mapping → device deployment." Through the coordinated operation of different modules in each step, it achieves real-time processing of low-light images from edge extraction and feature processing to the final enhanced output, as well as on embedded devices.

[0059] Edge Extraction: Multi-directional differential convolution kernels and multi-scale enhancement modules are used to extract and optimize image edge gradient information. Feature Fusion: Edge features and original image features are integrated using a dual-branch structure, attention mechanism, and spatial normalization. Image Enhancement: Based on lightweight U-Net and sparse Transformer, non-linear feature mapping and image quality improvement are achieved. Embedded Deployment: Through model format conversion and optimization, it is adapted for real-time processing on NVIDIA Jetson Nano. Modules achieve low-light image enhancement through data flow and functional integration.

[0060] A specific implementation method:

[0061] (I) Step 1: Extraction of multi-dimensional edge information

[0062] Differential convolution kernel operations: Three types of differential convolution kernels are used: circular, radial, and cross-shaped, to encode image gradient information. Taking circular differential convolution as an example, for a 3×3 pixel region (such as building outlines and vehicle edges in a nighttime road surveillance image), the gradient of adjacent pixels is calculated according to the formula: ;

[0063] Prior edge detection logic is encoded into convolutional layers. In practice, three types of convolutional kernels process low-light images in parallel: circular convolutional kernels: capture the curved contours of buildings and the edges of vehicle taillights; radial convolutional kernels: extract the gradient of road markings and streetlight beam extension directions; and cross convolutional kernels: enhance the horizontal and vertical edges of utility poles and traffic light poles, outputting multi-dimensional gradient feature maps.

[0064] Multi-scale enhancement processing. Compact dilated convolution: Convolutional kernels with dilation rates of 1, 2, and 3 are deployed in parallel. When processing distant, blurred vehicles in low-light images, the dilation rate 1 convolutional kernel focuses on vehicle edge details (such as tire outlines). The dilation rate 2 convolutional kernel associates vehicles with environmental features such as road shadows. The dilation rate 3 convolutional kernel covers the global location of the vehicle (such as lane distribution) and integrates multi-scale features to enhance edge representation. Compact spatial attention: Dynamic weight masks are generated through ReLU (Revised Linear Unit, which introduces non-linearity into the network, improving its ability to fit complex functions and accelerating network training), 1×1 convolution, 3×3 convolution, and Sigmoid activation. In low-light road scenes, key edges such as road markings and pedestrian outlines are given high weights, while weakly correlated areas such as background vegetation and sky are weighted less, highlighting effective edge information.

[0065] (ii) Step 2: Feature fusion - dual-branch feature processing.

[0066] Feature preprocessing branches: The first branch extracts low-level features of the original image (such as low-light road surface texture and vegetation outline) using 3×3 convolution; the second branch performs 2 layers of convolution on the edge map (output of step 1) to extract high-level features (such as building outline and vehicle edge abstract features).

[0067] Dynamic weighted fusion: according to formula F fuse =α·F1 + (1-α)·F2 The fusion feature is α = Var(F1) / (Var(F1) + Var(F2)). Taking low-light road conditions for autonomous driving as an example: if the original image has blurred texture due to poor lighting (small variance of F1) and clear road markings in the edge map (large variance of F2), then α is relatively small, and the fusion focuses on the high-level edge features of F2 to highlight the key edges of the road.

[0068] Spatial adaptive normalization: Semantic segmentation maps (such as pedestrian areas, building areas, and road areas in urban street images) are introduced, and normalization parameters μ and σ are generated to dynamically adjust the feature distribution. For building area features, the adjusted building edges are more prominent and stable in the fused features, preserving the input semantic information and providing high-quality fused features for subsequent enhancement.

[0069] (III) Step 3: Nonlinear mapping based on lightweight networks

[0070] A lightweight U-Net architecture is employed. A four-layer U-Net structure is used, where the encoder downsamples to 1 / 16 resolution (e.g., lesion regions in low-light medical images) through stride-2 convolutions, progressively compressing feature dimensions to extract high-level semantics. Each layer embeds a sparse attention Transformer module: dividing the feature map into 8×8 windows (e.g., local lesion regions in medical images), calculating local self-attention, and focusing on lesion edges and internal texture relationships. By limiting the attention range through a fixed window (e.g., calculating only pixel relationships within the window), the computational cost is reduced, making it suitable for embedded devices.

[0071] The decoder and skip connections. The decoder restores resolution through transposed convolution (gradually restoring from 1 / 16), and skip connections inject the edge feature map from step 1. Taking medical imaging as an example: during the decoder's restoration stage, lesion edge features supplement fracture lines and tumor boundary details, helping to clearly restore the lesion morphology and improve the diagnostic value of the image. For example, in low-light X-ray images, after edge feature injection, the lesion outline is clearer, optimizing visual quality and diagnostic accuracy.

[0072] (iv) Step 4: Deployment of Embedded Devices

[0073] Edge device architecture. Computing module: The NVIDIA Jetson Nano features a quad-core Cortex-A57 processor with 128 CUDA cores, operating at 5W power. When processing low-light images from surveillance cameras, it rapidly performs convolution and attention calculations (such as pedestrian and vehicle feature extraction from street images) to meet real-time requirements.

[0074] Input / output module: The camera captures low-light images (such as nighttime park monitoring), and the enhanced results are output to the USB-powered LED screen, realizing a closed loop of "acquisition → enhancement → display".

[0075] Model Deployment Process: Server-side: PyTorch-trained models are converted to ONNX format (cross-framework compatible), then to TensorRT format (adapted to NVIDIA GPUs). Edge-side: TensorRT optimizes the model through quantization (e.g., converting 32-bit floating-point parameters to 8-bit integers) and operator fusion (merging consecutive convolutions and activation operations) to generate the execution engine. Inference Call: The execution engine is called via a Python interface to quickly complete augmentation calculations when processing low-light images (such as images from autonomous vehicle camera cameras), outputting augmented images in real time to assist driving decisions.

[0076] A specific scenario-based implementation example: Taking a security monitoring scenario as an example, nighttime monitoring images are blurry due to insufficient lighting, making it difficult to identify details of faces and vehicles. The implementation process is as follows:

[0077] Edge extraction: Three differential convolution kernels extract facial contours and vehicle edge gradient information; a multi-scale enhancement module (dilated convolution + spatial attention) strengthens the edges and generates clear edge maps (such as vehicle taillights and facial contours).

[0078] Feature fusion: The dual-branch fusion of features from the original image and edge map dynamically weights and highlights key information about faces and vehicles (such as facial features and vehicle brand); spatial adaptive normalization adapts to scene semantics (road and building areas) and optimizes feature distribution.

[0079] Nonlinear mapping: The lightweight U-Net network restores image details, and sparse attention focuses on facial features and vehicle characteristics to output enhanced images (such as clearly distinguishable faces and recognizable vehicle brands).

[0080] Edge deployment: Jetson Nano processes and displays enhanced images in real time, assisting security personnel in accurate identification and demonstrating the application value of the method in real-world scenarios.

[0081] This invention effectively solves the challenge of low-light image enhancement through multi-dimensional edge extraction, dynamic feature fusion, lightweight network mapping, and embedded deployment, covering fields such as security, autonomous driving, and medical imaging. This invention provides a complete solution for the engineering application of low-light image enhancement technology, facilitating the efficient execution of computer vision tasks in related fields.

[0082] Finally, it should be noted that the present invention is not limited to the above embodiments, and many variations are possible. All variations that can be directly derived or conceived by those skilled in the art from the disclosure of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A low-light image enhancement method based on edge extraction and feature fusion, characterized in that The method comprises the following steps: Step 1) edge and gradient information of the image is extracted by a multi-directional differential convolution kernel to generate a multi-dimensional edge feature map; Step 2) the edge feature map is fused with the original low-light image features in multiple levels to combine global semantics and local details, thereby providing more abundant feature representation for image enhancement; Step 3) the image is nonlinearly mapped by a deep learning network to output an enhanced image; Step 4) real-time processing on an embedded device is realized through model quantization and hardware acceleration; In step 3), a lightweight Transformer network is used to nonlinearly map the fused features; Step 1) specifically comprises: Three differential convolution kernels, namely, ring, radial and cross, are used, wherein the calculation formula of the ring differential convolution is: ； In the above formula, W and X respectively represent a learnable parameter of a convolution kernel and a feature map to be convolved, y represents an output of the differential convolution, and the subscript represents a row and a column in a 3*3 region, wherein x 11 The multi-scale enhancement processing on the extracted gradient information comprises a compact dilated convolution module and a compact spatial attention module, the compact dilated convolution module uses convolution kernels with dilated rates of 1, 2 and 3, and the compact spatial attention module generates a dynamic weight mask through a Sigmoid activation function. 2.The low-light image enhancement method based on edge extraction and feature fusion according to claim 1, characterized in that: Step 2) comprises: a double-branch structure is used for feature fusion, a first branch is used to extract bottom-level features F1 by 3x3 convolution on the original image, and a second branch is used to extract high-level features F2 by 2-layer convolution on the edge map; The fusion formula is: F fuse = α·F1 + (1-α)·F2, wherein alpha is dynamically calculated from the variance of F1 and F2, and the calculation formula is: alpha = Var(F1) / (Var(F1) + Var(F2)), the feature distribution is dynamically adjusted through spatial adaptive normalization, and the normalized parameters mu and sigma are generated by input semantic segmentation map. 3.The low-light image enhancement method based on edge extraction and feature fusion of claim 1, characterized in that: Step 3) specifically comprises: a four-level U-Net structure is used, the encoder uses step 2 convolution to downsample to 1 / 16 resolution; each level contains a Transformer module based on sparse attention mechanism, which divides the feature map into 8x8 windows for local self-attention calculation; the decoder gradually restores the resolution by transposed convolution, and the edge feature map is injected through the jump connection.

4. The system of low-light image enhancement method based on edge extraction and feature fusion according to any one of claims 1-3, characterized in that, The method comprises a differential convolution-based edge extraction module, a feature fusion module and an image enhancement module connected by data; the differential convolution-based edge extraction module in step 1) can effectively extract edge information in the image by using differential convolution kernels in multiple directions; the feature fusion module in step 2) fuses the edge features output by the edge extraction module with the features of the original image, and can effectively combine global information and local details of the image through a multi-level fusion strategy, thereby providing more abundant feature representation for image enhancement; the image enhancement module in step 3) uses the fused features for image enhancement processing, and can effectively suppress noise and artifacts while enhancing image brightness through a deep learning network for nonlinear mapping of the image, thereby improving the visual quality of the image.

5. The system of claim 4, wherein, The operation mode of the differential convolution-based edge extraction module comprises the following: S1) multi-dimensional edge information extraction branch: differential convolution is used to extract edge information of features in different dimensions; differential convolution explicitly encodes prior information contained in traditional edge detection operators into convolution layers, which can more flexibly extract complex image gradient information; differential convolution first calculates the gradient corresponding to each pixel according to a hand-pre-designed gradient calculation strategy, and then convolves the calculated gradient by using a learnable convolution kernel; S2) Edge information processing branch: the edge information processing branch further processes the extracted edge information through the introduction of a compact hollow convolution module and a compact spatial attention module, the compact hollow convolution module extracts multi-scale edge features and fuses them using a hollow convolution to enhance edge expression ability; the compact spatial attention module dynamically adjusts feature weights through a spatial attention mechanism to highlight key edge regions; S3) Multi-dimensional edge information fusion: the multi-dimensional edge information extracted by the edge information processing branch is processed and fused, and the feature map obtained by fusion is output as an image gradient map through an activation function.

6. The system of claim 4, wherein, The operation mode of the feature fusion module includes the following: S1) Feature preprocessing branch: the edge map and the input image are processed and extracted through the combination of a conventional convolution layer and a Sigmoid activation function; S2) Feature fusion attention mechanism: the gradient image and the input feature are effectively fused through convolution and standardization operations, thereby enhancing the feature expression ability of the image; S3) Feature normalization: the image features are normalized through spatial adaptive normalization, and the parameters of the normalization layer are dynamically adjusted by introducing a semantic layout map as input, so that the input semantic information is better preserved when generating an image.

7. The system of claim 4, wherein, The network structure in the image enhancement module adopts a U-Net type network structure, which is divided into an encoder, a feature conversion module, and a decoder; The improved Transformer is introduced to reduce the computational complexity while maintaining the high performance of the model, the sparse attention mechanism only calculates the correlation between local regions or key positions, significantly reducing the computational load; the attention range is limited through fixed mode or dynamic learning, thereby reducing the computational overhead while maintaining performance.

8. The edge computing device of a low-light image enhancement method based on edge extraction and feature fusion according to any one of claims 1-3, characterized in that, An edge computing module, a video input module, and a result output module are connected; The edge computing module: NVIDIA Jetson Nano is used for real-time low-light image enhancement calculation; specifically, Jetson Nano uses a quad-core Cortex-A57 MPCore processor and 128 CUDA cores of NVIDIA Maxwell architecture, which can complete complex neural network calculations using only 5 watts of power; The video input module: a camera is used to capture images as input for low-light images; The result output module: a portable LED screen powered by USB is used for result output and display; The following steps are used to deploy the deep learning model trained on the server cluster on the edge computing device: the neural network model trained using the deep learning framework pytorch is converted into the general deep learning format ONNX; the neural network model in ONNX format is converted into a model in Tensor RT format, and then the Python interface is called to execute the engine Tensor RT for low-light image enhancement calculation of the neural network.

Images