Auto parts image super-resolution method based on dual-channel residual attention

Through the combination of a multi-level residual attention network and a lightweight processing module, the problem of limited image detail recovery effect in the prior art is solved, efficient super-resolution reconstruction of automotive parts images is realized, and detection accuracy and characteristics are improved Expression ability.

CN118570065BActive Publication Date: 2024-12-03SHANDONG BLUEBIRD IND INTERNET CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202411017322.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-12-03
Estimated Expiration
2044-07-29

AI Technical Summary

Technical Problem

The existing super-resolution method for image super-resolution of automobile parts fails to effectively combine local and global features, resulting in limited image detail recovery effect, affecting quality control and detection accuracy.

Method used

A multi-level residual attention network is used to process local and global features through dual-channel residual attention blocks, and the image feature expression ability is improved through feature fusion layer and lightweight processing module to achieve full fusion of multi-scale features.

Benefits of technology

It significantly improves image reconstruction quality and detection accuracy, improves the resolution of automotive parts images, and enhances feature expression capabilities and model efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118570065B_ABST
    Figure CN118570065B_ABST
Patent Text Reader

Abstract

The present invention proposes an automobile parts image super-resolution method based on dual-channel residual attention, which relates to the field of computer vision. The multi-level residual attention network proposed in the present invention consists of N dual-channel residual attention blocks and a feature fusion layer. The dual-channel residual attention block contains two parallel branches, which respectively extract local features and global features of the input image. At the same time, residual connections are introduced to improve the stability of image features and reduce the problems of gradient disappearance and gradient explosion. Finally, the initial image features and the outputs of all dual-channel residual attention blocks are fused to enhance the feature expression capability and improve the training effect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to computer vision, and in particular to an automobile parts image super-resolution method based on dual-channel residual attention. Background Art

[0002] In the modern automotive manufacturing industry, quality control and precision inspection are crucial, and high-resolution images are the key to ensuring the quality and reliability of automotive parts. However, due to the limitations of shooting equipment and environment, obtaining high-resolution images often faces technical challenges, especially in images taken by industrial cameras. Low resolution often leads to loss of details and insufficient detection accuracy, which affects the quality control of the manufacturing process. Therefore, it is necessary to improve the resolution of the image in order to show the details of the parts more clearly, detect tiny defects and damage, and thus improve the accuracy of detection.

[0003] In recent years, deep learning has made great progress in image enhancement. Among them, the residual network effectively alleviates the gradient vanishing problem in deep network training by introducing jump connections, enabling the network to perform deeper feature learning; the attention mechanism enables the model to better focus on key parts by giving higher weights to important features, thereby enhancing the feature representation ability. However, the existing methods still have shortcomings in extracting multi-scale features and making full use of the attention mechanism, and fail to effectively combine local features and global features, resulting in limited detail restoration effects in super-resolution images. In order to solve the above problems, the present invention proposes an automotive parts image super-resolution method based on dual-channel residual attention. The method constructs a multi-level residual attention network and uses dual-channel residual attention blocks to process local features and global features respectively, thereby achieving full fusion of multi-scale features and significantly improving feature expression capabilities and image reconstruction quality. Summary of the invention

[0004] The present invention provides an automobile parts image super-resolution method based on dual-channel residual attention, aiming to achieve full fusion of multi-scale features by constructing a multi-level residual attention network and utilizing dual-channel residual attention blocks to process local features and global features respectively.

[0005] The present invention integrates the residual network and the attention mechanism to provide an automobile parts image super-resolution method based on dual-channel residual attention, comprising the following steps:

[0006] S1. Use an industrial camera to capture images of automotive parts to obtain low-resolution images to be processed;

[0007] S2. Build a multi-level residual attention network and input image features To this network, the multi-level residual attention network includes N dual-channel residual attention blocks that retain important image features and a feature fusion layer. The dual-channel residual attention block is divided into an upper branch and a lower branch. The upper branch is mainly used to train and extract local features of the image, while the lower branch focuses on capturing global features and inputting image features. As the input of the first two-channel residual attention block, each subsequent two-channel residual attention block uses the output of the previous block as input to further learn and optimize features. After all feature blocks are calculated, Fuse with the output of all dual-channel residual attention to obtain enhanced important features;

[0008] S3, build a lightweight processing module, input the fused image features into the module, the module compresses the image features and retains important feature information, thereby reducing the computational complexity and the number of parameters, and improving the efficiency and performance of the model;

[0009] S4. Build an image super-resolution model, which consists of a feature extraction module, a multi-level residual attention network, a lightweight processing module and an image reconstruction module;

[0010] S5. Input the automobile parts image into the image super-resolution model, and obtain the super-resolution image after a series of processing.

[0011] Preferably:

[0012] S21, the dual-channel residual attention block is divided into an upper branch and a lower branch. If the input image feature is , the initial features Divided into The two parts are processed by two branches respectively. For the upper branch, the input feature The maximum pooling operation and the average pooling operation are performed respectively. The maximum pooling layer can retain the main features in the feature map and suppress noise and unimportant details. , Represents the maximum pooling operation. The average pooling layer reduces the size of the feature map by calculating the average value of the local area. , Represents the average pooling operation, connecting the pooling results , Represents the connection operation. The connected features are batch normalized by the batch normalization layer. Then, a 3×3 convolution layer and a ReLU activation function are used to capture the details of the local features while avoiding the gradient vanishing. Then, a 3×3 convolution layer and a sigmoid activation function are used to obtain the local spatial features. In order to solve the gradient vanishing and gradient exploding problems in deep networks, residual connections are introduced. This process can be described as , represents the batch normalization operation, represents a 3×3 convolutional layer, represents the ReLU activation function, Represents the sigmoid activation function, uses a 3×3 convolutional layer and a LeakyReLU activation function to enhance the expression of local features and obtain local features of the image , Represents the LeakyReLU activation function;

[0013] S22. For the lower branch, input feature Perform global average pooling operation to obtain the global characteristics of the channel , Represents the global average pooling operation. The pooling result uses a 7×7 convolution layer and a ReLU activation function to capture the global feature information. Then, a 7×7 convolution layer and a sigmoid activation function are used to obtain the global spatial features. The residual connection is introduced to obtain , Represents a 7×7 convolutional layer. The 7×7 convolutional layer and the LeakyReLU activation function are used to enhance the expression ability of global features and obtain the global features of the image. ;

[0014] S23. After the two branches have extracted the features, the outputs of the two branches are first connected to fuse the features of different scales together. The fused features are processed through a 1×1 convolution layer, which not only reduces the channel depth but also further integrates the feature information. Finally, the residual connection is used to connect the initial input features. Add to the output to get the final output of the dual-channel residual attention block .

[0015] Preferably:

[0016] For the lightweight processing module built in S3, input image features The lightweight processing module consists of a bottleneck layer with a convolution kernel of 1×1, a batch normalization layer, and a global average pooling layer. The bottleneck layer is first used to compress high-dimensional features to reduce the feature dimension, and then the compressed features are batch normalized. Finally, global average pooling is used to further compress the features to obtain simplified features. ,in, Represents the output of the lightweight processing module, Represents the bottleneck layer.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] The technical solution provided by the present invention is a multi-level residual attention network composed of N dual-channel residual attention blocks and a feature fusion layer. The dual-channel residual attention block contains two parallel branches. The upper branch uses a smaller convolution kernel to extract local features of the input image, and the lower branch uses a larger convolution kernel to extract global features of the input image. The stability of the features is retained through residual connections, and the problems of gradient disappearance and gradient explosion are reduced. Finally, the initial features and the outputs of all attention blocks are fused to enhance the expression ability of image features and obtain the best effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The present invention provides a flowchart for realizing super-resolution of automobile parts images.

[0020] Figure 2 This is a structural diagram of the multi-level residual attention network provided by the present invention.

[0021] Figure 3 This is an example diagram of the dual-channel residual attention block provided by the present invention.

[0022] Figure 4 This is an example diagram of the lightweight processing module provided by the present invention. DETAILED DESCRIPTION

[0023] The present invention aims to propose an automobile parts image super-resolution method based on dual-channel residual attention. First, a multi-level residual attention network is constructed, which includes N dual-channel residual attention blocks and a feature fusion layer. The dual-channel residual attention block contains two parallel branches. The upper branch uses a smaller convolution kernel to extract local features of the image, and the lower branch uses a larger convolution kernel to extract global features of the image. The stability of the features is retained through residual connections, which reduces the problems of gradient disappearance and gradient explosion. After all dual-channel residual attention blocks are calculated, the initial features and the outputs of all dual-channel residual attention blocks are fused to enhance the expression ability of image features and obtain the best effect.

[0024] See also Figure 1 As shown, the automobile parts image super-resolution method based on dual-channel residual attention in the embodiment of the present application:

[0025] S1. Fix the industrial camera on a stable bracket, ensure that the camera lens is aimed at the automotive parts, start the camera, aim at the automotive parts to shoot, obtain a series of low-resolution images to be processed, and save the captured images to a designated folder for subsequent processing and management;

[0026] S2. Build a multi-level residual attention network and input image features To this network, the multi-level residual attention network includes 12 dual-channel residual attention blocks that retain important image features and a feature fusion layer. The dual-channel residual attention block is divided into an upper branch and a lower branch. The upper branch is mainly used to train and extract local features of the image, while the lower branch focuses on capturing global features and inputting image features. As the input of the first two-channel residual attention block, each subsequent two-channel residual attention block uses the output of the previous block as input to further learn and optimize features. After all feature blocks are calculated, Fuse with the output of 12 dual-channel residual attentions to obtain enhanced important features;

[0027] S3, build a lightweight processing module, input the fused image features into the module, the module compresses the image features and retains important feature information, thereby reducing the computational complexity and the number of parameters, and improving the efficiency and performance of the model;

[0028] S4. Build an image super-resolution model, which consists of a feature extraction module, a multi-level residual attention network, a lightweight processing module and an image reconstruction module;

[0029] S5. Input the automobile parts image into the image super-resolution model, and obtain the super-resolution image after a series of processing.

[0030] Further, if Figure 2 As shown in the figure, after obtaining the vehicle image using the industrial camera, the image is input into the feature extraction module to obtain the initial features of the image , the image features are input into the multi-level residual attention network, which sets N dual-channel residual attention blocks and a feature fusion layer. As the input of the first two-channel residual attention block, each subsequent two-channel residual attention block uses the output of the previous block as input to further learn and optimize features. After all two-channel residual attention blocks are calculated, the initial features The output of each dual-channel residual attention block is input to the feature fusion layer for feature fusion to obtain enhanced image features. This process can be expressed as ,in, They represent the outputs of the 12 dual-channel residual attention blocks respectively. The fused image features are compressed by the lightweight processing module and important feature information is retained. The lightweight processing module consists of a bottleneck layer with a convolution kernel of 1×1, a batch normalization layer, and a global average pooling layer. The bottleneck layer is first used to compress high-dimensional features to reduce the feature dimension, and then the compressed features are batch normalized. Finally, the global average pooling is used to further compress the features to obtain simplified features. ,in, Represents the output of the lightweight processing module, Represents the bottleneck layer. In the image reconstruction module, the compressed image features are upsampled using pixel shuffling. The image resolution is improved by increasing the number of channels and rearranging the pixels in the feature map. Finally, the result is passed to the 3×3 convolutional layer to obtain a super-resolved image.

[0031] Further, if Figure 3 As shown in the figure, the image features are input into the upper branch and the lower branch respectively. The upper branch uses the average pooling layer, the maximum pooling layer and the convolution layer with a kernel of 3×3 to extract the local features in the image. The lower branch uses the global average pooling layer and the convolution layer with a kernel of 7×7 to extract the global features in the image. The residual connection is used in the upper and lower branch processes to improve the stability, and then the outputs of the two branches are connected. Finally, in order to reduce the problems of gradient vanishing and gradient exploding of the entire network, the connected features are connected to the initial features.

[0032] further,

[0033] S21, the dual-channel residual attention block is divided into an upper branch and a lower branch. If the input image feature is , the initial features Divided into The two parts are processed by two branches respectively. For the upper branch, the input feature The maximum pooling operation and the average pooling operation are performed respectively. The maximum pooling layer can retain the main features in the feature map and suppress noise and unimportant details. , Represents the maximum pooling operation. The average pooling layer reduces the size of the feature map by calculating the average value of the local area. , Represents the average pooling operation, connecting the pooling results , Represents the connection operation. The connected features are batch normalized by the batch normalization layer. Then, a 3×3 convolution layer and a ReLU activation function are used to capture the details of the local features while avoiding the gradient vanishing. Then, a 3×3 convolution layer and a sigmoid activation function are used to obtain the local spatial features. In order to solve the gradient vanishing and gradient exploding problems in deep networks, residual connections are introduced. This process can be described as , represents the batch normalization operation, represents a 3×3 convolutional layer, represents the ReLU activation function, Represents the sigmoid activation function, uses a 3×3 convolutional layer and a LeakyReLU activation function to enhance the expression of local features and obtain local features of the image , Represents the LeakyReLU activation function;

[0034] S22. For the lower branch, input feature Perform global average pooling operation to obtain the global characteristics of the channel , Represents the global average pooling operation. The pooling result uses a 7×7 convolution layer and a ReLU activation function to capture the global feature information. Then, a 7×7 convolution layer and a sigmoid activation function are used to obtain the global spatial features. The residual connection is introduced to obtain , Represents a 7×7 convolutional layer. The 7×7 convolutional layer and the LeakyReLU activation function are used to enhance the expression ability of global features and obtain the global features of the image. ;

[0035] S23. After the two branches have extracted the features, the outputs of the two branches are first connected to fuse the features of different scales together. The fused features are processed through a 1×1 convolution layer, which not only reduces the channel depth but also further integrates the feature information. Finally, the residual connection is used to connect the initial input features. Add to the output to get the final output of the dual-channel residual attention block .

[0036] Further, if Figure 4 As shown, the input image features The lightweight processing module consists of a bottleneck layer with a convolution kernel of 1×1, a batch normalization layer, and a global average pooling layer. The bottleneck layer is first used to compress high-dimensional features to reduce the feature dimension, and then the compressed features are batch normalized. Finally, global average pooling is used to further compress the features to obtain simplified features. ,in, Represents the output of the lightweight processing module, Represents the bottleneck layer.

[0037] The above are only preferred embodiments of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A super-resolution method for automobile parts images based on dual-channel residual attention, characterized in that: The following steps are involved: S1. Use an industrial camera to capture images of automotive parts to obtain low-resolution images to be processed; S2. Construct a multi-level residual attention network and input the image feature F0 into the network. The multi-level residual attention network includes N dual-channel residual attention blocks that retain important image features and a feature fusion layer. The dual-channel residual attention block is divided into an upper branch and a lower branch. The upper branch is used to train and extract local features of the image, while the lower branch focuses on capturing global features. The image feature F0 is used as the input of the first dual-channel residual attention block. Each subsequent dual-channel residual attention block uses the output of the previous block as input to further learn and optimize the features. After all feature blocks are calculated, the image feature F0 and the output of all dual-channel residual attentions are fused to obtain enhanced important features. S21, the dual-channel residual attention block is divided into an upper branch and a lower branch. If the input image feature is F in , the initial feature F in Divided into The two parts are processed by two branches respectively. For the upper branch, the input feature The maximum pooling operation and the average pooling operation are performed respectively. The maximum pooling layer can retain the main features in the feature map and suppress noise and unimportant details. MaxPool() represents the maximum pooling operation. The average pooling layer reduces the size of the feature map by calculating the average value of the local area. AvgPool() represents the average pooling operation, which connects the pooling results Concat() represents the connection operation. The connected features are batch normalized by the batch normalization layer, and then a 3×3 convolution layer and a ReLU activation function are used to capture the details of the local features while avoiding the gradient disappearance. Then a 3×3 convolution layer and a sigmoid activation function are used to obtain the local spatial features. In order to solve the gradient disappearance and gradient explosion problems in deep networks, residual connections are introduced. This process can be expressed as LN() represents batch normalization operation, Conv 3×3 Re() represents a 3×3 convolutional layer, Re() represents a ReLU activation function, and Si() represents a sigmoid activation function. The 3×3 convolutional layer and the LeakyReLU activation function are used to enhance the expression of local features and obtain local features of the image. Lr() represents the LeakyReLU activation function; S22. For the lower branch, input feature Perform global average pooling operation to obtain the global characteristics of the channel GloAvgPool() represents the global average pooling operation. The pooling result uses a 7×7 convolution layer and a ReLU activation function to capture the global feature information, and then uses a 7×7 convolution layer and a sigmoid activation function to obtain the global spatial features. The residual connection is introduced to obtain Conv 7×7 Represents a 7×7 convolutional layer. The 7×7 convolutional layer and the LeakyReLU activation function are used to enhance the expression ability of global features and obtain the global features of the image. S23. After the two branches have extracted the features, the outputs of the two branches are first connected to fuse the features of different scales together. The fused features are processed through a 1×1 convolution layer, which not only reduces the channel depth but also further integrates the feature information. Finally, the initial input feature F is connected by residual connection. in Add to the output to get the final output of the dual-channel residual attention block S3, build a lightweight processing module, input the fused image features into the module, the module compresses the image features and retains important feature information, thereby reducing the computational complexity and the number of parameters, and improving the efficiency and performance of the model; S4. Build an image super-resolution model, which consists of a feature extraction module, a multi-level residual attention network, a lightweight processing module and an image reconstruction module; S5. Input the automobile parts image into the image super-resolution model, and obtain the super-resolution image after a series of processing.

2. According to the dual-channel residual attention based automobile parts image super-resolution method of claim 1, the lightweight processing module in S3 is used to simplify image features, characterized in that: Input image feature F1, lightweight processing module consists of a bottleneck layer with a convolution kernel of 1×1, a batch normalization layer and a global average pooling layer. First, the bottleneck layer is used to compress high-dimensional features to reduce the feature dimension, and then the compressed features are batch normalized. Finally, global average pooling is used to further compress the features to obtain the simplified features F2 = GloAvgPool (LN (F B (F1))), where F2 represents the output of the lightweight processing module, F B Represents the bottleneck layer.

Citation Information

Patent Citations

  • Mine image super-resolution reconstruction method and system based on multi-scale residual network

    CN113592718A

  • Image super-resolution reconstruction model and method based on residual mixed attention network

    CN115222601A

  • Pedestrian re-identification method based on residual multi-channel attention multi-feature fusion

    CN115830531A

  • Image magnification method based on depth recursion residual channel attention

    CN116563111A

  • Convolutional neural network-based super-resolution reconstruction method suitable for real scene image

    CN117196940A