Multi-scale low-illumination image enhancement network model

Abstract

The invention provides a multi-scale low-illumination image enhancement network model. In recent years, an image enhancement technology has been widely applied to the fields of criminal investigation, medical imaging and the like. However, an image shot in a low-illumination environment has the conditions of low brightness, low contrast, color distortion and the like, which not only can reduce the visual experience of a user, but also can interfere with scene analysis and cause difficulty to follow-up computer vision tasks such as target tracking, image fusion and the like. The problem that information such as multiple scales and channels is not considered in image enhancement is solved. A multi-layer feature extraction module is introduced, multi-scale features are extracted, meanwhile, information interaction of high and low frequency features is achieved, and the calculated amount is saved. And meanwhile, the receptive field is expanded by utilizing cavity convolution. And a residual structure is introduced into the multi-layer feature extraction module and finally, so that the problems of low-layer information loss and the like in the convolution process are solved. The multi-scale low-illumination image enhancement network model provided by the invention has effective technical means such as a multi-layer feature extraction module and cavity convolution receptive field expansion, so that the brightness, the contrast ratio and the color reduction degree of the low-illumination image can be greatly improved, and the problems of the image can be solved. In addition, the innovative technical model has a wide application prospect, can be widely applied to the fields of criminal investigation, medical imaging and the like, possibly and continuously promotes the development of an image enhancement technology, and brings deeper influence and value to research and practice in related fields.

Description

Technical Field

[0001] The invention relates to the field of image enhancement of computer vision, and is a multi-scale low-illumination image enhancement network model. Background Art

[0002] In recent years, image enhancement technology has been widely used in criminal investigation, medical imaging and other fields. However, images taken in low-light environments will have low brightness, low contrast, color distortion and other problems. These phenomena will not only reduce the user's visual experience, but also interfere with scene analysis, and cause difficulties for subsequent computer vision tasks such as target tracking and image fusion. Such images bring varying degrees of challenges to human perception of the real world and a large number of high-order visual tasks based on optical images. Such problems can be alleviated to a certain extent by improving imaging equipment or using more professional shooting techniques, such as using longer exposure time or higher ISO. However, the objects being photographed often have inevitable movement, so using long exposure shooting will cause motion blur; using high ISO settings will also cause high noise problems. Due to the irreversibility of the imaging process, using image enhancement to enhance such degraded images has become a mainstream method. Therefore, research on low-light image enhancement has important application value.

[0003] Therefore, based on the above problems, a multi-scale low-illumination image enhancement network model is proposed. Summary of the invention

[0004] The present invention proposes a multi-scale low-light image enhancement network model, the method comprising:

[0005] Step 1: First, use a 3×3 convolutional layer to extract features, and then downsample the extracted features by 2 and 4 times respectively;

[0006] Step 2: The 3×3 convolution image obtained in step 1 is input into the multi-layer feature extraction module after downsampling by 2 times and 4 times respectively.

[0007] Step 3: Multi-layer feature extraction module processes the input feature map

[0008] Step 3.1: Perform 1×1 convolution on the input feature map;

[0009] Step 3.2: Perform 3×3 convolution on the result of the previous step;

[0010] Step 3.3: Divide the convolution results into 4 groups according to the number of channels;

[0011] Step 3.4: Stack different numbers of groups and perform dilated convolution on the stacked results with dilation rates of 2, 4, and 8;

[0012] Step 3.5: stack the result of step 3.1 and the result of step 3.4;

[0013] Step 3.6: Perform a 3×3 convolution on the output feature map of 3.5 and activate it with ReLU.

[0014] Step 4: Stack the result of step 3 with the result of step 1 without downsampling.

[0015] Step 5: Pass the result of step 4 through two 3×3 convolutions and ReLU activation.

[0016] Step 6: The result of step 5 is connected using a residual connection, that is, it is added to the result of step 4, passes through a 3×3 convolution, and is activated by ReLU.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] (1) Introducing multi-scale information and fully considering feature information at different scales;

[0019] (2) Introduce dilated convolution to make full use of information from different fields of view. Description of the drawings:

[0020] Attached Figure 1 : Network framework diagram of the present invention

[0021] Attached Figure 2 : Multi-layer feature extraction module Specific implementation method:

[0022] The present invention will be further described below in conjunction with the accompanying drawings.

[0023] Step 1: As attached Figure 1 As shown in the figure, a 3×3 convolutional layer is first used to extract features, and then the extracted features are downsampled by 2 times and 4 times respectively.

[0024] Step 2: As attached Figure 1 As shown, the 3×3 convolved image obtained in step 1 and the three feature images downsampled by 2 and 4 times respectively are input into the multi-layer feature extraction module respectively.

[0025] Step 3: As attached Figure 2 As shown, the multi-layer feature extraction module processes the input feature map

[0026] Step 3.1: As attached Figure 2 As shown in the figure, a 1×1 convolution is performed on the input feature map. The obtained feature map is used for subsequent multi-feature extraction and the combination of the final feature extraction results;

[0027] Step 3.2: As attached Figure 2As shown, a 3×3 convolution is performed on the result of the previous step.

[0028] Step 3.3: As attached Figure 2 As shown, the convolution results are divided into 4 groups according to the number of channels, namely ×1, ×2, ×3, and ×4;

[0029] Step 3.4: As attached Figure 2 As shown in the figure, ×1 and ×2 are stacked, and a dilated convolution with a dilation rate of 2 is performed. ×1, ×2 and ×3 are stacked, and a dilated convolution with a dilation rate of 4 is performed. The input features are subjected to a dilated convolution with a dilation rate of 8. The three convolution results are stacked with ×1;

[0030] Step 3.5: As attached Figure 2 As shown, the result of step 3.1 is stacked with the result of step 3.4;

[0031] Step 3.6: Perform a 3×3 convolution on the output feature map of 3.5 and activate it with ReLU.

[0032] Step 4: As attached Figure 1 , stack the results of step 3.

[0033] Step 5: As attached Figure 1 , the result of step 4 is passed through two 3×3 convolutions and activated by ReLU.

[0034] Step 6: As attached Figure 1 , the result of step 5 uses a residual connection, that is, it is added to the result of step 4, passes through a 3×3 convolution, and is activated by ReLU.

Claims

1. A multi-scale low-light image enhancement network model, characterized in that: The following steps are involved: Step 1: First, a 3×3 convolutional layer is used for feature extraction, and then the extracted features are downsampled by 2 and 4 times respectively. Step 2: The 3×3 convolution image obtained in step 1 is input into the multi-layer feature extraction module after downsampling by 2 times and 4 times respectively. Step 3: Multi-layer feature extraction module processes the input feature map Step 3.1: Perform 1×1 convolution on the input feature map; Step 3.2: Perform 3×3 convolution on the result of the previous step; Step 3.3: Divide the convolution results into 4 groups according to the number of channels; Step 3.4: Stack different numbers of groups and perform dilated convolution on the stacked results with dilation rates of 2, 4, and 8; Step 3.5: stack the result of step 3.1 and the result of step 3.4; Step 3.6: Perform a 3×3 convolution on the output feature map of 3.5 and activate it with ReLU. Step 4: Stack the result of step 3 with the result of step 1 without downsampling. Step 5: Pass the result of step 4 through two 3×3 convolutions and ReLU activation. Step 6: The result of step 5 is connected using a residual connection, that is, it is added to the result of step 4, passes through a 3×3 convolution, and is activated by ReLU.

2. The method according to claim 1, characterized in that: In step 2, the 3×3 convolution image obtained in step 1 is input into the multi-layer feature extraction module with 2 times and 4 times downsampling respectively.

3. The method according to claim 1, characterized in that: In step 3, the multi-layer feature extraction module divides the image into 4 groups according to the number of channels, and uses different dilation rates of the dilated convolution for different stacking combinations.

4. The method according to claim 1, characterized in that: In step 4, the result of step 3 is stacked with the result of step 1 without downsampling.

5. The method according to claim 1, characterized in that: A residual connection is introduced at the final output, that is, the result of step 5 is added to the result of step 4 using a residual connection, and then passes through a 3×3 convolution and ReLU activation.

Images