A low-light image enhancement method and system based on parallel hybrid attention progressive fusion

Abstract

The application discloses a kind of parallel mixed attention progressive fusion low-illumination image enhancement method and system, specific steps are as follows: constructing parallel mixed attention progressive fusion image enhancement network, the network includes multi-scale weighted aggregation module MWA, the progressive feature fusion module PFM after being connected in multi-scale weighted aggregation module MWA and multiple parallel mixed attention modules PHA embedded in progressive feature fusion module PFM;Parallel mixed attention progressive fusion image enhancement network is trained, and parallel mixed attention progressive fusion image enhancement network model is obtained;The data set to be detected is input into parallel mixed attention progressive fusion image enhancement network model, and the low-illumination image after image enhancement is obtained.The method of the application can effectively enhance the overall brightness of image, adjust image color imbalance and remove noise to avoid detail information loss, facilitate the development of subsequent required task and work.

Description

Technical Field

[0001] The present invention belongs to the technical field of low-light image enhancement, and in particular to a low-light image enhancement method and system with parallel mixed attention progressive fusion. Background Art

[0002] In many real-life scenarios, capturing high-quality images is one of the key factors for achieving high accuracy in tasks such as target detection, image segmentation, autonomous driving, and medical surgery. However, images and videos captured by electronic devices are easily affected by various environmental factors such as poor lighting, resulting in low image brightness, color distortion, high noise, and loss of effective detail and texture information. These problems pose many difficulties for subsequent tasks and work.

[0003] Low-light image enhancement typically involves increasing brightness, removing noise, and restoring color through methods like background fusion and adaptive mapping functions to restore image clarity and provide high-quality images for a variety of practical scenarios. In recent years, deep neural networks, due to their powerful nonlinear fitting capabilities, have achieved promising results in low-light enhancement and other fields. However, in some cases, color imbalance can occur when increasing image brightness and contrast, and the effects of noise can be easily overlooked, creating significant challenges for subsequent tasks and work.

[0004] To address the loss of detail after image enhancement, methods that extract and fuse multi-scale image features can effectively enhance the detail representation of low-light images after brightness enhancement. However, this process assigns equal importance to each feature being fused, which can easily overlook the complex nature of multi-scale features. This can lead to varying degrees of color desaturation, artifacts, and color loss in the fused image. Attention mechanisms can identify semantic correlations within the original low-light image data, highlighting key features of the object of interest while ignoring irrelevant image information. However, as the network deepens, most models only perform aggregation at certain locations within the network. This not only prevents the effective utilization of the semantic information of the fused multi-level features, resulting in redundancy, but also creates significant semantic discrepancies between the information, resulting in unnecessary computational resource consumption. However, the multi-branch fusion modules rely solely on simple information flow, and there is no effective feature correlation between non-adjacent modules. This results in good brightness improvement in the initial stages of image enhancement, but as the network deepens and time passes, the image suffers from varying degrees of brightness unevenness and loss of detail. Summary of the Invention

[0005] In order to solve the problems existing in the prior art, the present invention provides a low-light image enhancement method and system with parallel mixed attention progressive fusion, which can effectively enhance the overall brightness of the image, adjust the image color imbalance and remove noise to avoid the loss of detail information, and facilitate the implementation of subsequent tasks and work.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a low-light image enhancement method with parallel hybrid attention progressive fusion, the specific steps of which are as follows:

[0007] S1 constructs a parallel hybrid attention progressive fusion image enhancement network, which includes a multi-scale weighted aggregation module MWA, a progressive feature fusion module PFM connected after the multi-scale weighted aggregation module MWA, and multiple parallel hybrid attention modules PHA embedded in the progressive feature fusion module PFM;

[0008] S2 trains the parallel mixed attention progressive fusion image enhancement network to obtain a parallel mixed attention progressive fusion image enhancement network model;

[0009] S3 inputs the dataset to be detected into the parallel mixed attention progressive fusion image enhancement network model to obtain the enhanced low-light image.

[0010] Furthermore, the multi-scale weighted aggregation module MWA includes three branch parallel branches, a concat function, a 3×3 convolution, and a sigmoid function. The three branch parallel branches are used to extract the shallow color local multi-scale feature information in the original low-light image, and the concat function is used to stack and aggregate the shallow color local multi-scale feature information extracted by different branch branches.

[0011] The 3×3 convolution and sigmoid function are used to assign different weights to the shallow color local multi-scale feature information extracted from different branches, so that the shallow color local multi-scale feature information can be effectively fused under the guidance of different weights, and the global representation information containing the shallow color local multi-scale features can be obtained.

[0012] Furthermore, each branch parallel branch is composed of a different number of 3×3 convolutions. From branch1 to branch3, the number of 3×3 convolutions in each branch is 1, 3, and 5, respectively.

[0013] Furthermore, the progressive feature fusion module PMF includes three stages. The input of stage 1 is the global representation information including the shallow color local multi-scale features output by the multi-scale weighted aggregation module MWA and the shallow color local multi-scale features output by branch 1.

[0014] The input of the stage2 stage is the output of the stage1 stage and the output of the branch2 branch, which contains the local multi-scale features of shallow color; the input of the stage3 stage is the output of the stage2 stage and the output of the branch3 branch, which contains the local multi-scale features of shallow color.

[0015] Furthermore, each stage includes a global average pooling module GAP and a parallel hybrid attention module PHA. The global average pooling module GAP is used to calculate the average value of all pixels in the input channel feature map to obtain the aggregated spatial information; the parallel hybrid attention module PHA is used to refine the input channel feature map to obtain the attention refinement feature.

[0016] In each stage, the spatial information and attention refinement features of the corresponding stage are multiplied and fused with the local multi-scale features containing shallow colors of the corresponding branch to obtain the global attention features.

[0017] Furthermore, the parallel hybrid attention module PHA includes two parallel hybrid attention branches, each branch includes a channel attention module CAM and a pixel attention module PAM, the arrangement order of the channel attention module CAM and the pixel attention module PAM in the two branches is different, and the attention modules in the two branches are connected alternately in parallel front and back.

[0018] Furthermore, after the attention modules in the two branches are connected alternately in parallel front and back, the input processing of the first mixed attention branch and the second mixed attention branch are converged and input into the first branch channel attention module CAM, so that the refined features output by the first branch channel attention module CAM flow to the second branch pixel attention module PAM, and the refined features output by the second branch pixel attention module PAM flow to the first branch pixel attention module PAM.

[0019] The present invention also provides a system for enhancing low-light images by parallel hybrid attention progressive fusion, comprising:

[0020] A network construction module is used to construct a parallel hybrid attention progressive fusion image enhancement network, wherein the network includes a multi-scale weighted aggregation module MWA, a progressive feature fusion module PFM connected after the multi-scale weighted aggregation module MWA, and multiple parallel hybrid attention modules PHA embedded in the progressive feature fusion module PFM;

[0021] A network model training module is used to train the parallel mixed attention progressive fusion image enhancement network to obtain a parallel mixed attention progressive fusion image enhancement network model;

[0022] The image enhancement module is used to input the data set to be detected into the parallel mixed attention progressive fusion image enhancement network model to obtain the enhanced low-light image.

[0023] The present invention also provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0024] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0025] Compared with the prior art, the present invention has at least the following beneficial effects:

[0026] The present invention proposes a low-light image enhancement method with parallel hybrid attention and progressive fusion, constructs a parallel hybrid attention progressive fusion image enhancement network, and uses the multi-scale weighted aggregation module MWA to weight the limited correlation between local features extracted by different multi-scale branches to achieve mutual complementation of local image detail features under multiple receptive fields. The parallel hybrid attention module PHA is used to simultaneously focus on color information and illumination features, effectively improving the network's detail expression and weakening noise. Finally, the progressive feature fusion module PFM is used to fuse shallow feature information in multiple stages to alleviate the weakening of color information expression due to increased network depth and the model confusion caused by single-stage feature stacking. The method of the present invention can effectively enhance the overall brightness of the image, adjust image color imbalance and remove noise. Using this method, it can increase the visibility of night-time monitoring for multiple scenarios in real life, such as car driving, surveillance cameras and Internet social scenes.

[0027] Furthermore, the present invention adopts different numbers of 3×3 convolutions in each branch of the multi-scale weighted aggregation module MWA to extract multi-scale contextual information. Compared with directly applying 5×5 or 7×7 convolution kernels, this method can reduce parameters while retaining a larger receptive field.

[0028] Furthermore, the present invention utilizes a parallel hybrid attention module (PHA) to prevent the situation where single brightness enhancement and image denoising cannot effectively improve image quality, and the reverse order of brightness enhancement and image denoising will lead to blurred details in the enhanced image or the introduction of noise.

[0029] Furthermore, the present invention uses a progressive feature fusion operation to not only enrich the functions of the next stage and make the network optimization process more stable, but also makes full use of non-overlapping features to effectively compensate for the missing contextual information, and obtains clear edge information and accurate abstract information in a coarse-to-fine manner using a multi-level refinement strategy. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is the overall structure diagram of the network of the present application;

[0031] Figure 2 is the network diagram of the multi-scale weighted aggregation module MWA of the present application.

[0032] Figure 3 is the network diagram of the parallel hybrid attention module PHA of the present application.

[0033] Figure 4 is the network diagram of the progressive feature fusion module PFM of the present application.

[0034] Figure 5 is the execution result diagram of the present application. DETAILED DESCRIPTION

[0035] The present application will be further described below in combination with the drawings and specific embodiments.

[0036] Reference Figure 1 The present application discloses a parallel hybrid attention progressive fusion low-illumination image enhancement method, comprising the following steps:

[0037] S1, a progressive fusion image enhancement network of parallel hybrid attention is constructed, including a multi-scale weighted aggregation module MWA, a parallel hybrid attention module PHA and a progressive feature fusion module PFM, the multi-scale weighted aggregation module MWA is used to obtain multi-scale local characteristics by aggregating multi-scale features learned under different receptive fields, promote the global representation of local features, strengthen the preservation of detail information of the original low-illumination image to be enhanced, and obtain global representation information containing multi-scale local characteristics; the progressive feature fusion module PFM is used to reprocess the global representation information and the multi-scale local characteristics from coarse to fine in three stages, supplement the loss of shallow features caused by the increase of network depth, avoid information redundancy caused by single-stage feature stacking, realize re-attention of important areas of the image from coarse to fine, form a more robust feature representation, and each stage of the progressive feature fusion module PFM contains a parallel hybrid attention module PHA, the parallel hybrid attention module PHA is used to refine the input channel feature map by using the pixel attention module PAM and the channel attention module CAM arranged in parallel combination, relieve the color difference caused by the lag of different branch attention distribution, and effectively improve the color expressiveness of the global representation information and weaken the noise by mutual complementation of information between adjacent attentions, to obtain attention refined features.

[0038] Specifically:

[0039] 1. Multi-scale weighted aggregation module MWA

[0040] The fixed receptive field method tends to ignore the distribution of low illumination in the image, and the enhancement result often worsens the illumination, and even leads to problems such as instability of the enhancement effect and poor generalization performance of the model. The reason for weighted aggregation of the different feature information provided by each branch is that all scale features extracted from multiple receptive fields on the low-illumination image are directly aggregated by simple element addition or connection, which will cause the results of these simple summations to neither show more important features nor suppress irrelevant features, and may cause useful information branches to be weakened or even overwhelmed by useless information branches. In response to the above problems, the present invention has designed the following Figure 2 The multi-scale weighted aggregation module MWA shown in the figure consists of three parallel branches, a concat function for feature stacking, and a 3×3 convolution and sigmoid function for generating different weights. The execution process of the MWA module is as follows: Figure 2 :

[0041] First, three parallel branches are used to extract local multi-scale feature information of shallow colors in the original low-light image. Each parallel branch consists of a different number of 3×3 convolutions. From branch1 to branch3, the number of 3×3 convolutions in each branch is 1, 2, and 3, respectively. Compared with directly applying 5×5 or 7×7 convolution kernels, this approach effectively reduces the number of parameters while retaining a larger receptive field.

[0042] Secondly, the concat function is used to stack the shallow color local multi-scale feature information extracted by different branches, so that the shallow color local multi-scale feature information extracted by different branches is aggregated under the action of the concat function, which can better learn the long-term relationship between adjacent pixels.

[0043] Finally, the sigmoid function is used to assign different weights to the shallow color local multi-scale feature information extracted by each branch, so that the shallow color local multi-scale feature information can be effectively fused under the guidance of different weights, and global representation information containing shallow color local multi-scale features is obtained. This process can not only retain the complex characteristics of the multi-scale itself, but also, as the network continues to deepen, shallower feature mappings can also achieve good learning performance, fully considering the complementarity and differences between features from both global and local aspects, and effectively restoring the image lighting details.

[0044] 2. Progressive Feature Fusion Module PMF

[0045] Fusing features of different levels is necessary for improving image effect, and shallow features in the convolution process not only have higher resolution, but also usually contain more location and illumination details, etc., but the semantic information contained is reduced and the noise is increased after a small amount of convolution. The way of fusing features of different levels through hierarchical stacked modules ignores the fusion efficiency between deep and shallow features, which not only easily leads to confusion, feature redundancy and local feature under-fusion of the obtained model, but also usually specifically shows that the features of the same target are offset in the image.

[0046] In view of the above problems, the present application designs a progressive feature fusion module PMF, as shown in the figure, Figure 4 The PFM module includes stage1 stage, stage2 stage and stage3 stage from bottom to top, each stage contains a global average pooling module GAP and a parallel hybrid attention module PHA, wherein the global average pooling module GAP is used to calculate the average value of all pixels of the input channel feature map to obtain the collected spatial information, and the parallel hybrid attention module PHA is used to refine the input channel feature map to obtain attention refined features; from bottom to top, stage1 stage to stage3 stage:

[0047] In each stage, the spatial information and the attention refined features are multiplied to obtain the attention feature map highlighting the important regions of the image that need to be paid attention to, and the attention feature map is fused with the shallow color local multi-scale feature information extracted under different receptive fields in the multi-scale weighted aggregation module MWA to form the feature fusion result global attention feature, and the global attention feature is taken as the input of the global average pooling module GAP and the parallel hybrid attention module PHA of the next stage;

[0048] In the first stage1 stage, the input of the global average pooling module GAP and the parallel hybrid attention module PHA is the global representation information in the multi-scale weighted aggregation module MWA, the spatial information obtained by the global average pooling module GAP is multiplied by the attention refined features obtained by the parallel hybrid attention module PHA, the important regions of the image are focused, the attention feature map is obtained, and the attention feature map is fused with the shallow color local multi-scale feature information extracted by the branch1 branch in the multi-scale weighted aggregation module MWA to obtain the global attention feature of the first stage1 stage;

[0049] The global attention features of the stage 1 are then passed to the global average pooling GAP module and the parallel hybrid attention PHA of the second stage 2. Based on stage 1, the important areas of the image are re-focused to obtain the attention feature map. The attention feature map is fused with the shallow color local multi-scale feature information extracted by the branch 2 branch in the multi-scale weighted aggregation module MWA to further refine the features and obtain the global attention features of the second stage 2.

[0050] Finally, the global attention features of the second stage2 are passed to the global average pooling GAP module and parallel hybrid attention PHA of the third stage3 respectively. Based on stage2, further attention is paid to the important areas of the image to obtain an attention feature map. The attention feature map is fused with the shallow color local multi-scale feature information extracted by branch3 in the multi-scale weighted aggregation module MWA to obtain a deep global attention feature map, which can achieve precise attention to the key areas of low-light images.

[0051] The progressive feature fusion module PMF of the present invention adopts a progressive fusion approach to study the deep features of the image, which can extend the existing single-stage fusion model to multiple stages to gradually refine the significant features, ensure the increase of useful information in the process and reduce the information difference caused by the fusion of features at different stages.

[0052] Furthermore, the present invention redistributes the network's computational power through a multi-level refinement strategy, not only reducing the difficulty of the original task but also fully utilizing the shallow local multi-scale feature information of color extracted by the multi-scale weighted aggregation module (MWA) to obtain clear edge information and accurate abstract information. This operation not only enriches the functionality of the next stage, but also makes the network optimization process more stable and easier to train.

[0053] 3. Parallel Hybrid Attention Module PHA

[0054] See also Figure 3 , the module includes two parallel hybrid attention branches, each branch includes a channel attention module CAM and a pixel attention module PAM, but the arrangement order of the channel attention module CAM and the pixel attention module PAM in the two branches is different, and the attention modules in the two branches are connected in parallel alternately, as shown in Figure 2. Figure 3As shown, the refined features output by the first-branch channel attention module CAM are directed to the second-branch pixel attention module PAM, and the refined features output by the second-branch pixel attention module PAM are directed to the first-branch pixel attention module PAM. This allows the output features of different attention modules to complement each other, alleviating the feature differences caused by the sequential arrangement of the channel attention module CAM and the pixel attention module PAM in the mixed attention branch, thereby simultaneously extracting the brightened and denoised deep features of the mixed attention. Compared with existing low-light enhancement methods, this structure can effectively avoid the blurred image details and noise amplification caused by the reversal of the order of brightness enhancement and image denoising during the image enhancement process.

[0055] Channel Attention Module CAM

[0056] The output feature maps of different channel dimensions have different effects on low-light image enhancement, and the features of some channels are more important than others. Specifically, if the correlation with adjacent channels is stronger, the network needs to pay more attention to the adjacent channels and assign them greater weight.

[0057] The channel attention module CAM first compresses the global information into a channel descriptor through global average pooling GAP, and obtains a 1×1×C vector by shrinking the feature space dimension, where each component represents the average of all feature pixels in each channel and has a global receptive field.

[0058] The weighted global attention representation obtained on this branch is multiplied with the input feature branch on the channel. The importance of each feature channel is adaptively determined by rescaling the weight information of each channel, and the nonlinear relationship between channels is learned to improve the sensitivity of the network model to task-related information, thereby obtaining a channel attention feature map.

[0059] Pixel Attention Module (PAM)

[0060] Pixels at different positions in low-light images have extremely high local dependence on their adjacent pixels. This module improves the local representation strength of pixel information by weighting its adjacent pixels, so that the image generates a pixel-by-pixel attention module graph that helps weaken random noise with the help of the pixel attention module PAM.

[0061] The input feature F of the pixel attention module PAM is processed by convolution and activation function. First, an attention map PA with the same size as the input feature can be generated as shown in formula (1). This attention map can rescale each position of the feature map and be applied to each pixel of the feature map.

[0062] PA=σ(Conv(δ(Conv(F)))) (1)

[0063]

[0064] By learning the relationship between all pixel feature map positions, as shown in formula (2), the output pixel attention feature F of the pixel attention module PAM is obtained by element-by-element multiplication fusion and input feature F. P . Where δ and σ represent ReLu and sigmoid activation functions respectively.

[0065] All pixel-level features are rescaled according to the exposure intensity at different positions in the image. Each pixel in the feature map generated according to the input feature mapping relationship can be assigned a corresponding weight, weakening the dependence on distant pixels, making the model pay more attention to information areas with uneven light distribution, and reducing the impact of random noise.

[0066] Then, the shallow color information extracted by different branches in the multi-scale weighted aggregation module MWA and the deep image features output by the parallel hybrid attention module PHA are fused to reduce the overall color distortion of the image caused by brightness enhancement and color mining. Ultimately, the simultaneous attention of light intensity and color information in the image enhancement process is achieved, and the blurred details or noise in the enhanced image caused by the reversal of the order of brightness enhancement and image denoising are avoided.

[0067] S2 loss function

[0068] The total loss function of the present invention is shown in formula (3):

[0069]

[0070] For low-light images, it is more necessary to consider those areas with insufficient light and introduce the regional loss function L Region :

[0071]

[0072] Where: E L and G L Represent the enhanced image and the low-light area of ​​the original image, E H and G H is the rest of the image, usually w L =4,w H =1.

[0073] Structural loss L SSIM(x,y) It can better measure the difference before and after image enhancement under low illumination conditions and describe the similarity between two images. The calculation process can be described as follows:

[0074]

[0075] Among them, μ X ,μ YRepresents the mean of images X and Y, σ X ,σ Y Represents the variance of images X and Y respectively, C1, C2 are constants σ XY Represents the covariance of image X and Y. The value range of this loss is [0,1]. The larger the value, the smaller the image distortion.

[0076] Attention loss function L Attention It can better guide the network to pay attention to areas with uneven lighting, which can be expressed as:

[0077]

[0078] Where I represents the low-light image, G A (I) represents the predicted illumination distribution map, N represents the number of sample training, represents a normal lighting image, It represents the maximum value of the pixel on the three channels of RGB image.

[0079] This paper uses the PyTorch deep learning framework. The GPU workstation configuration includes an Intel Core i7 CPU, 32GB of RAM, and a 1080Ti graphics card with 12GB of video memory. Training is performed for 300 epochs, with a minimum batch size of 8, an initial decay rate of 0.0005, and a cosine learning rate decay.

[0080] The experimental dataset used is the LOL dataset, a dataset of real-world image pairs used for low-light enhancement. It contains 500 low-light / normal-light image pairs, most of which were collected by varying exposure time and ISO from various scenes, such as houses, campuses, clubs, and streets. Therefore, 485 pairs were selected for training and 15 for testing, resulting in a parallel hybrid attention progressive fusion image enhancement network model.

[0081] S3 inputs the dataset to be tested into the parallel hybrid attention progressive fusion image enhancement network model for detection, and compares the enhanced results of the experimental results on the LOL dataset with different advanced methods. The results are as follows Figure 5As shown in the figure, the row containing "ours" represents the execution results of the proposed method, while (a)-(e) represent the experimental results of different methods in various scenarios. The image obtained by the RetinexNet method improves the illumination to a certain extent, but the enhancement results will cause color shift, and the reconstructed image has varying degrees of color distortion. As can be clearly seen in Figures (a) and (d), the enhanced image produces significant color difference after brightening, and the image environment lacks overall visual authenticity. The image obtained by the KinD method is dark and overly smooth, which easily misses subtle details and causes the overall image edge contour to become blurred. The surface of the ornament in Figure (c) lacks texture and has a whitish color. The image enhanced by the MBLLEN method is darker, with some areas not significantly enhanced and exhibiting poor contrast. As can be seen in Figure (b), it is even impossible to distinguish whether the stacked objects are one layer or multiple layers in dim lighting. The proposed method can effectively enhance brightness and improve color while maintaining good image naturalness in low-light images.

[0082] The method of the present invention is quantitatively compared with advanced image enhancement methods on the LIME, DICM, and MEF datasets. The results are shown in Table 1:

[0083] Table 1 shows the performance comparison of NIQE on LIME, DICM, and MEF datasets.

[0084]

[0085] As shown in Table 1, compared with other advanced algorithms, our method achieves NIQE scores of 3.4281, 2.8054, and 3.5193 on the LIME, DICM, and MEF datasets, respectively. Smaller values ​​indicate better performance. Compared with the suboptimal methods, our method leads by 0.2293, 0.1112, and 0.018, respectively. This leading low-light enhancement performance further validates the effectiveness of our method.

[0086] The method of the present invention can effectively reduce and alleviate problems such as color distortion, noise amplification, and loss of detail information that occur during low-light image enhancement. In addition, through low-light image enhancement, the visibility of nighttime monitoring can be improved in multiple scenarios in real life, such as car driving, surveillance cameras, and Internet social scenarios.

[0087] An embodiment of the present invention discloses a parallel hybrid attention progressive fusion image enhancement system, comprising a network construction module for constructing a parallel hybrid attention progressive fusion image enhancement network, wherein the network comprises a multi-scale weighted aggregation module MWA, a parallel hybrid attention module PHA connected after the multi-scale weighted aggregation module MWA, and multiple progressive feature fusion modules PFM embedded in the parallel hybrid attention module PHA;

[0088] The multi-scale weighted aggregation module MWA is used for feature extraction of the low-illumination image through three branches under different receptive fields, and the multi-scale feature information extracted through branches with different numbers of convolutions is stacked, so that the feature information extracted by different branches is aggregated, and the feature information extracted by the three branches is given different weights, so as to guide the effective fusion of the features.

[0089] The parallel hybrid attention module PHA has a channel attention module CAM and a pixel attention PAM on each of the two branches, forming a front and rear parallel alternating structure to realize regional attention, so that the output features of each attention are complementary to the two adjacent attentions flowing thereto, and then the features output by the different branches of the attention are fused to realize synchronous attention of illumination brightness and color information in the image enhancement process.

[0090] The progressive feature fusion module PFM is composed of three stages, and the multi-scale features extracted by the branches of each stage under different receptive fields are fused to form more robust feature representation, and are used as input in the next stage, so that clear edge information and accurate abstract information are obtained by the multi-level refinement strategy from coarse to fine.

[0091] The network model training module is used for training the progressive fusion image enhancement network with parallel hybrid attention to obtain a progressive fusion image enhancement network model with parallel hybrid attention.

[0092] The image enhancement module is used for inputting the to-be-detected data set into the progressive fusion image enhancement network with parallel hybrid attention to obtain a low-illumination image after image enhancement.

[0093] The present application provides a terminal device, comprising: a processor, a memory and a computer program stored in the memory and executable on the processor. The processor executes the computer program to realize the steps in each of the above method embodiments. Alternatively, the processor executes the computer program to realize the functions of each module / unit in each of the above device embodiments.

[0094] The computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application.

[0095] The terminal device can be a desktop computer, a notebook computer, a palm computer and a cloud server, etc. The terminal device can include, but is not limited to, a processor and a memory.

[0096] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0097] The memory may be used to store the computer programs and / or modules, and the processor implements various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory.

[0098] If the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0099] Based on this understanding, the present invention can implement all or part of the processes in the above-mentioned method embodiments by means of a computer program that instructs the relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned method embodiments can be implemented. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form.

[0100] The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunications signal, and a software distribution medium. It should be noted that the content of the computer-readable medium may be appropriately increased or decreased based on the requirements of legislation and patent practice within a jurisdiction. For example, in some jurisdictions, based on legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunications signals.

[0101] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A low-light image enhancement method based on parallel hybrid attention progressive fusion, characterized in that: The specific steps are as follows: S1 constructs a parallel hybrid attention progressive fusion image enhancement network, which includes a multi-scale weighted aggregation module MWA, a progressive feature fusion module PFM connected after the multi-scale weighted aggregation module MWA, and multiple parallel hybrid attention modules PHA embedded in the progressive feature fusion module PFM; S2 trains the parallel mixed attention progressive fusion image enhancement network to obtain a parallel mixed attention progressive fusion image enhancement network model; S3 inputs the dataset to be detected into the parallel mixed attention progressive fusion image enhancement network model to obtain the enhanced low-light image; The multi-scale weighted aggregation module MWA includes three parallel branches, a concat function, a 3×3 convolution, and a sigmoid function. The three parallel branches are used to extract the local multi-scale feature information of shallow colors in the original low-light image, and the concat function is used to stack and aggregate the local multi-scale feature information of shallow colors extracted by different branches. The 3×3 convolution and sigmoid function are used to assign different weights to the shallow color local multi-scale feature information extracted by different branches, so that the shallow color local multi-scale feature information can be effectively fused under the guidance of different weights, and the global representation information containing the shallow color local multi-scale features can be obtained; The progressive feature fusion module PMF includes three stages. The input of stage 1 is the global representation information including the shallow color local multi-scale features output by the multi-scale weighted aggregation module MWA and the shallow color local multi-scale features output by branch 1. The input of the stage 2 stage is the output of the stage 1 stage and the output of the branch 2 branch, which contains the local multi-scale features of shallow color. The input of the stage 3 stage is the output of the stage 2 stage and the output of the branch 3 branch, which contains the local multi-scale features of shallow color. The parallel hybrid attention module PHA includes two parallel hybrid attention branches, each branch includes a channel attention module CAM and a pixel attention module PAM. The arrangement order of the channel attention module CAM and the pixel attention module PAM in the two branches is different, and the attention modules in the two branches are connected alternately in parallel front and back.

2. The low-light image enhancement method of parallel hybrid attention progressive fusion according to claim 1, characterized in that: Each branch parallel branch is composed of a different number of 3×3 convolutions. From branch1 to branch3, the number of 3×3 convolutions in each branch is 1, 3, and 5, respectively.

3. The low-light image enhancement method of parallel hybrid attention progressive fusion according to claim 1, characterized in that: Each stage consists of a global average pooling module GAP and a parallel hybrid attention module PHA. The global average pooling module GAP is used to calculate the average value of all pixels in the input channel feature map to obtain the aggregated spatial information; the parallel hybrid attention module PHA is used to refine the input channel feature map to obtain the attention refinement feature. In each stage, the spatial information and attention refinement features of the corresponding stage are multiplied and fused with the local multi-scale features containing shallow colors of the corresponding branch to obtain the global attention features.

4. The low-light image enhancement method of parallel hybrid attention progressive fusion according to claim 1, characterized in that: After the attention modules in the two branches are connected alternately in parallel front and back, the input processing of the first mixed attention branch and the second mixed attention branch are converged and input into the first branch channel attention module CAM, so that the refined features output by the first branch channel attention module CAM flow to the second branch pixel attention module PAM, and the refined features output by the second branch pixel attention module PAM flow to the first branch pixel attention module PAM.

5. A system for enhancing low-light images by parallel hybrid attention progressive fusion, characterized in that: The system runs the steps of a parallel hybrid attention progressive fusion low-light image enhancement method according to any one of claims 1 to 4, and the system includes: A network construction module is used to construct a parallel hybrid attention progressive fusion image enhancement network, wherein the network includes a multi-scale weighted aggregation module MWA, a progressive feature fusion module PFM connected after the multi-scale weighted aggregation module MWA, and multiple parallel hybrid attention modules PHA embedded in the progressive feature fusion module PFM; A network model training module is used to train the parallel mixed attention progressive fusion image enhancement network to obtain a parallel mixed attention progressive fusion image enhancement network model; The image enhancement module is used to input the data set to be detected into the parallel mixed attention progressive fusion image enhancement network model to obtain the enhanced low-light image.

6. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images