Network frame node detection image preprocessing method based on improved U-Net architecture

Abstract

The invention discloses a net rack node detection image preprocessing method based on an improved U-Net framework. The method comprises the following steps of: 1, constructing a low-illumination net rack node image feature data set by collecting low-illumination net rack node images under different indoor illumination; step 2, fusion is carried out, and an improved U-Net-like network model comprising a three-scale decoder and a three-scale codec is constructed; step 3, training the improved similar U-Net network model to obtain a trained similar U-Net network model; and step 4, preprocessing the node image to be enhanced by using the trained U-Net-like network model to obtain an enhanced net rack node detection picture, thereby providing a solid foundation for subsequent net rack node detection. According to the method, an adaptive feature fusion module SKFF is designed in a decoder by applying a self-attention mechanism to fuse features of different levels so as to restore, reconstruct and enhance an image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and computer vision, and in particular to a grid node detection image preprocessing method based on an improved U-Net architecture. Background Art

[0002] In the acceptance and health inspection of grid steel structure buildings, accurately capturing the image information of grid nodes is the key to achieving accurate positioning and spacing calculation of grid nodes, thereby achieving high-precision detection of grid steel structure buildings. Grid steel structures are widely used in the roofs of key facilities such as airport terminals and large industrial plants. However, since the indoor environment of these buildings is often in a low-light state, the acquired images face severe challenges such as insufficient brightness, low contrast, image blur, and noise interference, which seriously restricts the accuracy and reliability of node positioning. However, when processing low-light grid node images, existing methods still have problems such as high computational complexity, insufficient feature extraction, and limited feature fusion effects. Therefore, how to effectively pre-process and enhance grid node images in low-light environments is an urgent problem that needs to be solved.

[0003] With the development of artificial intelligence technology, many scholars at home and abroad have applied artificial intelligence technology to the field of image processing. Among them, low-light image enhancement has attracted widespread attention in many emerging computer vision fields in recent years, and a large number of low-light image enhancement algorithms have been proposed.

[0004] It can be roughly divided into traditional algorithms, methods based on model representation, and methods based on network. Traditional algorithms are mainly represented by histogram equalization (HE) and gamma correction (GC). HE-based methods can be divided into global HE and local HE. To a certain extent, it can improve contrast and enhance images. However, for input images with relatively complex content, the enhancement effect is not ideal. There are problems such as overexposure in some areas and blurred local details, and the interference of potential noise in low-light images cannot be effectively eliminated, and such noise may even be amplified. Although the GC-based method can effectively enhance the local dark areas in the image, it will also cause the local bright areas of the image to be overexposed. Low-light image enhancement methods based on model representation, such as those based on Retinex theory, have achieved good results in brightness and contrast adjustment, but it is difficult to obtain more ideal enhancement results for complex lighting environments. Methods based on deep neural networks, such as CNN, Transformer, U-Net, etc., have greatly improved the quality of image enhancement to a certain extent.

[0005] In order to obtain a better low-light image enhancement effect, the local feature capture capability of CNN, the global context modeling advantage of Transformer, and the symmetrical encoding and decoding structure of U-Net (facilitating multi-scale feature fusion) are deeply integrated. Although this strategy broadens the performance boundary of image enhancement, it also inevitably leads to an increase in computational complexity, and puts forward an urgent need for computational efficiency and model optimization. At the same time, how to efficiently and harmoniously integrate low-level local image details and high-level global semantic information to cope with the limitation of lack of image information under low-light conditions is also an urgent problem to be solved. Summary of the invention

[0006] In order to overcome the above technical problems, the purpose of the present invention is to provide a grid node detection image preprocessing method based on an improved U-Net architecture, which embeds a small U-Net (SUNet) in the encoder and decoder to provide multi-scale learning and reduce complexity; introduces deep separable convolution in the encoder to reduce the calculation complexity of the model, deploys a lightweight multi-branch feature extraction module LMFE to extract features at different scales, captures a variety of detail information from global to local, and an illumination attention module IAM focuses on important illumination features and optimizes illumination distribution; uses the self-attention mechanism to design an adaptive feature fusion module SKFF in the decoder to fuse features of different levels to restore and reconstruct enhanced images.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] A grid node detection image preprocessing method based on an improved U-Net architecture comprises the following steps:

[0009] Step 1: Collect low-illumination grid node images under different indoor lighting conditions to build a low-illumination grid node image feature dataset;

[0010] Step 2: Fusion and construction to obtain an improved U-Net-like network model including a three-scale decoder and a three-scale encoder-decoder;

[0011] Step 3: Train the improved U-Net-like network model to obtain a trained U-Net-like network model;

[0012] Step 4: Use the trained U-Net-like network model to preprocess the node image to be enhanced to obtain an enhanced grid node detection image, providing a solid foundation for subsequent grid node detection.

[0013] The step 2 specifically includes the following steps:

[0014] Step 21, insert a small U-Net (SUNet) at the first scale of the encoder to process the highest resolution features; provide more multi-scale representation learning, understand the image content from different scales, so as to better restore the brightness and contrast of the image and provide high-quality enhancement effect for the image;

[0015] For the other two scales of the encoder, DResBlock is used to achieve further node image feature extraction and accurate information capture; 3×3 cross-row convolution is used between two adjacent scales to spatially downsample the feature map;

[0016] Step 22: A small U-Net (SUNet) is still used at the first scale of the decoder to integrate information from different scales, including features from the encoder and intermediate features from other scales of the decoder, to restore the up- and down-sampled node image features to their original size, and generate clearer and more detailed images. Based on the idea of ​​self-attention, in order to make full use of the high-quality features extracted by the network, an adaptive feature fusion module SKFF is designed at the other two scales of the decoder to aggregate features of different scales of the encoder and decoder. In the decoding process, 3×3 transposed convolution is combined for spatial upsampling to achieve top-down reconstruction of enhanced up- and down-sampled node image features.

[0017] Furthermore, the step 21 includes the following sub-steps:

[0018] Step 211, a small U-Net (SUNet) is introduced at the first scale of the encoder, whose internal encoder and decoder are composed of multiple LRes residual blocks, which effectively captures rich multi-scale node image features, and performs downsampling and upsampling through deep convolution (stride=2, kernel=2) and bilinear interpolation respectively;

[0019] In step 212, the remaining two scales of the encoder use DResBlock formed by stacking multiple LRes residual blocks to perform node image feature extraction step by step.

[0020] Furthermore, in step 211, SUNet in the encoder processes the highest resolution features, processes and learns the detail information of the input image at an early stage, and provides more multi-scale representation learning by reducing the resolution of the features, thereby improving efficiency; specifically:

[0021] First, four LRes residual blocks are used to extract features. Deep convolution reduces the resolution by half, reducing the amount of calculation and the number of parameters, while capturing more global node image feature information. After two LRes residual blocks, the features are upsampled to the input size for subsequent processing, and feature representations of different scales are learned, thereby providing more multi-scale information. This helps the network better understand the structure and content of the image.

[0022] Furthermore, in step 212, the residual block LRes first uses a 3×3 depthwise separable convolution to perform shallow feature extraction on the basis of reducing the number of parameters and computational complexity. For degradations of different sizes in the low-light grid node image, a lightweight multi-branch feature extraction module LMFE is embedded after the depthwise separable convolution. The receptive field is expanded based on pooling technology to perform processing in the global and local range, and then the 3×3 depthwise separable convolution is used again for feature extraction. In order to reduce redundant feature information and pay more attention to features such as lighting and texture details, an illumination attention module IAM is deployed after the LMFE of the last LRes block.

[0023] Furthermore, the step 22 includes the following sub-steps:

[0024] Step 221, using the high-quality features extracted by the network, breaking through the limitations of traditional feature fusion methods (such as splicing and summation) that are simple to operate but limit the network feature expression ability, and using the adaptive feature fusion module SKFF based on the self-attention mechanism decoder to perform nonlinear fusion and recombination of the extracted features.

[0025] Specifically, the outputs of the last two scales of the encoder are fused as the input of the first SKFF, and the output is fused with the SUNet of the first scale of the encoder through 1×1 convolution as the input, and the output is adjusted through 1×1 convolution as the input of the next scale;

[0026] Step 222, for low-light images of grid nodes, some subtle texture and structural information is difficult to recover through simple convolution operations; a small U-Net (SUNet) with the same structure as the encoder's first scale is introduced at the first scale of the decoder to further improve the recovery of image details and texture information through its internal multi-scale processing mechanism.

[0027] Furthermore, in step 3, a part of the low-light grid node data set is taken as a test set, and another part is taken as a trainval file, and then a part of the trainval file is taken as a training set, and another part is taken as a verification set.

[0028] Beneficial effects of the present invention:

[0029] 1. The present invention introduces a small U-Net (SUNet) at the first scale of the network encoder and decoder to provide multi-scale learning and reduce the complexity of the network. In the encoder, preliminary feature extraction is performed on the low-light grid node image to capture richer and more representative global and local information in the image, so as to better understand the content of the image; in the decoder, the intermediate features from the encoder features and other scales of the decoder are integrated to reduce the semantic differences of features at different levels, better integrate the feature information of each scale, and make the enhanced image better in terms of details.

[0030] 2. Reference the depth-separable convolution in the LRes residual block, and deploy the lightweight multi-branch feature extraction module LMFE and the illumination attention module IAM to reduce the number of model parameters and computational complexity. The multi-branch feature extraction module LMFE is designed to effectively expand the receptive field through global and window-based average pooling technology to better capture the global and local information in low-light images; the illumination attention module IAM is deployed in the last LRes residual block of each scale. Through the synergy of the channel attention module and the spatial attention module, the network pays more attention to important lighting features, and can adaptively adjust the feature weights to better adapt to different low-light images, improve the generalization ability of the network, and effectively improve the image enhancement effect and visual effect.

[0031] 3. The decoder of the present invention uses the adaptive feature fusion module SKFF to achieve better feature fusion and improve the effect of low-light image enhancement through nonlinear fusion features and fine feature aggregation and weighted fusion operations. At the same time, SKFF can reduce the semantic differences of features at different levels, making the model more robust when processing low-light grid node images and better cope with noise and uncertainty in images.

[0032] In summary, the improved U-Net-like network model trained by the present invention is used to enhance the low-light grid node image, and the PSNR and SSIM values ​​are used as performance indicators to evaluate the performance. On the 4080 self-made low-light grid node data set, the performance is better than the original U-Net network. The test results show that compared with the original U-Net algorithm, the PSNR and SSIM on the self-made low-light grid node data set are improved by 4.93dB and 0.027db respectively, with better enhancement effect and better robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is the overall network architecture of the present invention. Among them, SUNet is a small U-Net, and SKFF is an adaptive feature fusion module.

[0034] Figure 2The architecture of SUNet of the present invention includes multiple LRes residual blocks, in which downsampling and upsampling are depth convolution and bilinear interpolation with a step size of 2, respectively.

[0035] Figure 3 This is the architecture diagram of the encoder part DResBlock of the present invention, where LMFE is a lightweight multi-branch feature extraction module.

[0036] Figure 4 This is the structural diagram of the illumination attention module IAM in DResBlock.

[0037] Figure 5 It is a structural diagram of the decoder adaptive feature fusion module SKFF of the present invention.

[0038] Figure 6 This is a picture of the grid node to be enhanced.

[0039] Figure 7 This is the enhanced grid node picture. DETAILED DESCRIPTION

[0040] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0041] like Figure 1 As shown, a grid node detection image preprocessing method based on an improved U-Net architecture of the present invention specifically includes the following steps:

[0042] Step 1: Collect low-light grid node images under different indoor lighting conditions and construct a low-light grid node image feature dataset.

[0043] The low-light grid node dataset in the working environment was obtained by the researchers themselves. In different buildings, such as gymnasiums and airport terminals, a total of 4080 grid node images were taken with different illumination backgrounds, and a one-to-one mapping between data and labels was established to form a low-light grid node dataset.

[0044] Step 2: Fusion and construction of an improved U-Net-like network model as a model for low-illumination grid node image enhancement.

[0045] like Figure 1As shown, it is the improved U-Net network model of the present invention. The backbone network is the U-Net network. The original U-Net network contains a large number of convolution operations and consists of an encoder, a decoder and a jump connection. For the low-light grid node image enhancement in this paper, more rich detail information is needed for restoration and enhancement. Therefore, in order to extract more information of different scales, a small U-Net (SUNet) and DResBlock are used for feature extraction and enhancement; the adaptive feature fusion module SKFF and the small U-Net (SUNet) are used for feature fusion and reconstruction to restore the image.

[0046] Step 21: Input the encoder low-light grid node image and then input it into the first scale small U-Net (SUNet) after 3×3 convolution adjustment channel, such as Figure 2 As shown in Figure 1, based on its multi-scale learning ability, it initially extracts richer global and local features in the image; the output of SUNet is spatially downsampled by 3×3 cross-row convolution and then input into the first DResBlock that contains a depth-wise separable convolution and deploys a lightweight multi-branch feature extraction module LMFE and an illumination attention module IAM, as shown in Figure 1. Figure 3 As shown in the figure, LMFE processes in the global and local range to better capture the global and local information in the low-light image; IAM, through the synergy of the channel attention module and the spatial attention module, enables the network to pay more attention to important features such as important lighting and texture details, better adapt to different low-light images, reduce the interference of redundant feature information, improve the generalization ability of the network, improve the image enhancement effect and visual effect, and make the enhanced image perform better in details and lighting.

[0047] The image depth features are extracted through multiple LRes residual blocks. First, it is adjusted by 3×3 depth separable convolution. Then, the input LMFE is split into two branches, which are respectively subjected to global average pooling and window-based average pooling to expand the receptive field, extract global context information and local features, and then re-stitched and subjected to 3×3 depth separable convolution to obtain the final output of the LRes residual block. The illumination attention module IAM is deployed in the last LRes residual block of DResBlock, as shown in Figure 4As shown in the figure, after the input IAM is adjusted by 3×3 depth-separable convolution, it is subjected to average pooling (smoothing the features, reducing noise and interference in the features, and extracting the global information of the features), maximum pooling (highlighting the local maximum value in the features, emphasizing the significant information in the features, such as edges, textures, etc.), and the average pooling and maximum pooling processing splicing (comprehensively capturing image information, including the overall lighting conditions and local important details, providing richer input for subsequent weight calculation and attention mechanism, and improving the effect and quality of image enhancement) to generate three branches, and then processed by two 1×1 convolutions, element-wise addition, and then the channel feature weight is obtained by the Sigmoid function, which is multiplied with the input feature map to obtain the channel attention output feature map. Its output is spliced ​​after capturing the feature information between channels at different spatial positions by maximum pooling, 3×3 convolution, and average pooling, and then processed by 1×1 convolution and 3×3 convolution, element-wise addition, and the spatial feature weight is generated by the Sigmoid function, which is multiplied with the channel attention output to obtain the spatial attention output feature. Finally, the spatial attention output feature map and the channel attention output feature map are added element by element to obtain the final output of the IAM module;

[0048] The output of the first DResBlock is spatially downsampled by 3×3 cross-row convolution and then input into the second DResBlock in series to further mine the richer deep features of the image, so that the feature information is gradually transmitted and deepened in the encoder.

[0049] Step 22: The outputs of the two DResBlocks of the encoder are used as the input of the first adaptive feature fusion module SKFF of the decoder, as shown in Figure 5 As shown in the figure, SKFF first merges the two input tensors by splicing, reshaping and accumulation operations in turn, then uses global average pooling to aggregate features, and finally uses point-by-point convolution to enhance feature correlation; then the obtained tensor is forward propagated in two ways, and the attention weights are obtained through 1×1 convolution and softmax function, which are multiplied and summed with the SKFF input respectively; the module output is obtained after transposed convolution upsampling and 1×1 convolution adjustment channel, and then input into the next SKFF together with the output of the encoder SUNet for feature fusion, and its output is input into the decoder SUNet after transposed convolution upsampling and 1×1 convolution adjustment channel, and the information from different scales is integrated to restore the feature map to a higher resolution, thereby realizing the recovery and reconstruction of the image feature details and texture information.

[0050] Step 3: Train the improved U-Net-like network model to obtain a trained U-Net-like network model.

[0051] Specifically: during the training and detection process, 40% of the low-light grid node data set is taken as the test file (test set), and 60% is taken as the trainval file; then 90% of the trainval file is taken as the training set, and 10% is taken as the validation set. Among them, the training set is used to train the model and obtain the model weights, and the validation set is used to determine the network structure and adjust the hyperparameters of the model. A total of 2000 epochs are set, and the optimizer used in the training process is the Adam optimizer. After the training is completed, the optimal U-Net-like network model weights are obtained for low-light grid node image enhancement.

[0052] Step 4: Use the trained U-Net-like network model to enhance the node image to obtain an enhanced grid node image.

[0053] as follows Figure 6 This is an image of the grid nodes to be enhanced. The details of the nodes in the image are blurred and the contrast is low, making it difficult to accurately identify the location and status of the nodes.

[0054] Figure 7 The model outputs an enhanced grid node image in which the node details are clearer, the contrast is improved, and the originally blurred parts become clearly discernible, providing a solid foundation for subsequent grid node detection.

[0055] The present invention aims to achieve high-quality enhancement of low-illumination grid node images by constructing an improved U-Net network model including a three-scale encoder and a three-scale decoder.

[0056] Information is transferred between the encoder and the decoder through jump connections; a small U-Net (SUNet) is introduced at the first scale of the encoder. The input low-light grid node image is input into this SUNet after a 3×3 convolution adjustment channel, and richer global and local features in the image are initially extracted based on its multi-scale learning ability. The output of SUNet is downsampled by 3×3 cross-row convolution space. Then it is input into the first DResBlock, and the output is downsampled by 3×3 cross-row convolution space and input into the second serial DResBlock to complete the step-by-step feature extraction; then it enters the decoding part, and the output of the two DResBlocks of the encoder is used as the input of the first SKFF. The output is upsampled by transposed convolution and adjusted by 1×1 convolution, and then input into the next SKFF together with the output of the encoder's SUNet for feature fusion. The output is upsampled by transposed convolution and adjusted by 1×1 convolution, and then input into the decoder's SUNet.

Claims

1. A grid node detection image preprocessing method based on an improved U-Net architecture, characterized in that: The steps include: Step 1: Collect low-illumination grid node images under different indoor lighting conditions to build a low-illumination grid node image feature dataset; Step 2: Fusion and construction to obtain an improved U-Net-like network model including a three-scale decoder and a three-scale encoder-decoder; Step 3: Train the improved U-Net-like network model to obtain a trained U-Net-like network model; Step 4: Use the trained U-Net-like network model to preprocess the node image to be enhanced to obtain an enhanced grid node detection image.

2. According to claim 1, a network node detection image preprocessing method based on an improved U-Net architecture is characterized in that: The step 2 specifically includes the following steps: Step 21, insert a small U-Net (SUNet) at the first scale of the encoder to process the highest resolution features; The other two scales of the encoder use DResBlock to achieve further node image feature extraction and accurate information capture; A 3×3 cross-row convolution is used between two adjacent scales to spatially downsample the feature map; Step 22, using a small U-Net (SUNet) at the first scale of the decoder to integrate information from different scales, including features from the encoder and intermediate features from other scales of the decoder, to restore the up- and down-sampled node image features to their original size, generating clearer and more detailed images; For the other two scales of the decoder, an adaptive feature fusion module SKFF is designed to aggregate features of different scales of the encoder and decoder; In the decoding process, 3×3 transposed convolution is combined for spatial upsampling to achieve top-down reconstruction of node image features after enhanced up and down sampling.

3. According to claim 2, a network node detection image preprocessing method based on an improved U-Net architecture is characterized in that: The step 21 includes the following sub-steps: Step 211, a small U-Net (SUNet) is introduced at the first scale of the encoder, whose internal encoder and decoder are composed of multiple LRes residual blocks, which effectively captures rich multi-scale node image features, and performs downsampling and upsampling through deep convolution (stride=2, kernel=2) and bilinear interpolation respectively; In step 212, the remaining two scales of the encoder use DResBlock formed by stacking multiple LRes residual blocks to perform node image feature extraction step by step.

4. According to claim 3, a network node detection image preprocessing method based on an improved U-Net architecture is characterized in that: In step 211, First, four LRes residual blocks are used to extract features. Deep convolution reduces the resolution by half, reducing the amount of calculation and the number of parameters, while capturing more global node image feature information. After two LRes residual blocks, the features are upsampled to the input size for subsequent processing, and feature representations of different scales are learned, thereby providing more multi-scale information; In step 212, the residual block LRes first uses 3×3 depthwise separable convolution to perform shallow feature extraction on the basis of reducing the number of parameters and computational complexity. For degradations of different sizes in the low-light grid node image, a lightweight multi-branch feature extraction module LMFE is embedded after the depthwise separable convolution. The receptive field is expanded based on pooling technology to perform processing in the global and local range, and then 3×3 depthwise separable convolution is used again for feature extraction. An illumination attention module IAM is deployed after the LMFE of the last LRes block.

5. According to claim 2, a network node detection image preprocessing method based on an improved U-Net architecture is characterized in that: The step 22 includes the following sub-steps: Step 221, the outputs of the last two scales of the encoder are fused as the input of the first SKFF, and the output is fused with the SUNet of the first scale of the encoder as the input through 1×1 convolution, and the output is adjusted through 1×1 convolution as the input of the next scale; Step 222: Introduce a small U-Net (SUNet) with the same structure as the encoder's first scale at the decoder's first scale, and further improve the recovery of image details and texture information through its internal multi-scale processing mechanism.

6. According to claim 1, a network node detection image preprocessing method based on an improved U-Net architecture is characterized in that: In step 3, a part of the low-light grid node data set is taken as a test set, and another part is taken as a trainval file, and then a part of the trainval file is taken as a training set, and another part is taken as a verification set.

Images