A super-resolution method for automobile parts images based on large kernel distillation network

Abstract

The present invention proposes a super-resolution method for automobile parts images based on a large kernel distillation network, which relates to the field of computer vision. The feature distillation block proposed in the present invention includes two branches, each of which can independently obtain important local features, thereby improving the model's acquisition of local features of the image; at the same time, the proposed dual-channel large convolution kernel decomposition block includes two branches, each of which decomposes a large convolution kernel into three parts: channel convolution, spatial local convolution, and spatial long-range convolution, thereby improving the model's acquisition of global features of the image and reducing the computational cost and the number of parameters; the feature distillation block and the dual-channel large convolution kernel decomposition block respectively obtain local features and global features of the image, enhance the feature expression capability, and improve the super-resolution effect of automobile parts images.

Description

Technical Field

[0001] The invention belongs to the field of computer vision, and in particular relates to an automobile parts image super-resolution method based on a large-core distillation network. Background Art

[0002] Modern automobile manufacturing is becoming more and more refined and intelligent. During the production process, low-resolution images make quality inspection difficult, and it is difficult to find tiny flaws, scratches or cracks on the surface of parts. These defects may gradually expand during subsequent use and affect the performance and safety of the car. This requires higher-resolution images to clearly present details. High-resolution images can detect tiny flaws and defects in time to prevent defective products from flowing into the next process, thereby ensuring the quality and reliability of auto parts and improving the quality and safety of cars.

[0003] In recent years, great progress has been made in efficient and lightweight image super-resolution. Lightweight super-resolution networks have achieved remarkable results in resource-constrained scenarios such as mobile devices and embedded systems. In terms of network structure, distillation networks have been proven to reduce computational costs and memory usage while ensuring model capacity and performance. However, existing models still do not meet the requirements of lower computational costs. To solve this problem, the present invention proposes an automotive parts image super-resolution method based on a large-core distillation network. This method decomposes large convolution kernels to improve model performance while reducing computational costs, thereby improving the performance of the image super-resolution model. Summary of the invention

[0004] The present invention proposes a super-resolution method for automobile parts images based on a large kernel distillation network, which aims to improve the image super-resolution effect by constructing a large kernel distillation network, extracting deep features by stacking large kernel distillation blocks, and processing global features by using dual-channel large convolution kernel decomposition blocks and reducing the computational cost and the number of parameters.

[0005] The present invention integrates a distillation network and a large kernel design to provide an automobile parts image super-resolution method based on a large kernel distillation network, comprising the following steps:

[0006] S1, automobile parts image collection, using industrial cameras to shoot automobile parts and collect low-resolution automobile parts images;

[0007] S2. Construct a feature distillation block FRB. The feature distillation block FRB consists of two branches, the upper and lower branches, which obtain feature information of different scales, use convolution operations, deep convolutions and ReLU activation functions to extract features, and use skip connections to enhance residual learning;

[0008] S3. Construct a dual-channel large convolution kernel decomposition block DBKB. The dual-channel large convolution kernel decomposition block DBKB consists of two branches, the upper and lower branches are large convolution kernels of different scales, and the large convolution kernel is decomposed into 1×1 convolution, depth convolution and depth expansion convolution by using convolution decomposition. The skip connection is used to enhance residual learning, and the output of the upper and lower branches is weighted fused to obtain the final output.

[0009] S4, construct a large kernel distillation block BKDB, which includes three feature distillation blocks FRB, 1×1 convolution, connection operation, dual-channel large convolution kernel decomposition block DBKB and pixel normalization module;

[0010] S5. Construct a large core distillation network BKDN, which includes N large core distillation blocks BKDB stacking, feature fusion module and image reconstruction module;

[0011] S6. Build an image super-resolution model, including input, feature extraction, feature distillation, feature fusion, image reconstruction and output;

[0012] S7. Input the low-resolution image of the automobile parts into the image super-resolution model, and obtain the super-resolution image after a series of processing.

[0013] Preferably, the characteristic distillation block in step S2 is characterized by comprising:

[0014] S21, the feature distillation block contains two branches. For the upper branch, input the feature , the initial features It is obtained by 1×1 convolution and skip connection. , 1×1 convolution extracts richer feature information, and skip connections prevent gradient disappearance. , In represents the upper branch in the characteristic distillation process, + represents element-by-element addition, represents a 1×1 convolution operation, which is then obtained by depthwise convolution and skip connection. , , represents a 1×1 depthwise convolution, Represents a 3×3 deep convolution, using 1×1 deep convolution to fuse features, 3×3 deep convolution to enhance feature propagation and jump connection to capture local feature information while avoiding gradient disappearance, and then activates the upper branch through the ReLU activation function , ReLU activation function is used to improve the expressiveness of local features, , Represents the ReLU activation function;

[0015] S22. For the lower branch, input feature , the initial features It is obtained by 3×3 convolution and skip connection. , 3×3 convolution extracts richer local feature information, and skip connections prevent gradient disappearance. , In They represent the lower branches in the characteristic distillation process, + represents element-by-element addition, represents a 3×3 convolution operation, which is then obtained by depthwise convolution and skip connection. , , represents a 3×3 depth convolution, Represents a 5×5 deep convolution, using 3×3 deep convolution to enhance feature propagation, 5×5 deep convolution to avoid deep network degradation and jump connection to capture a wider range of local feature information, while avoiding gradient disappearance, and then activate the output of the lower branch through the ReLU activation function , ReLU activation function is used to improve the expressiveness of local features, , Represents the ReLU activation function;

[0016] S23, feature distillation After the two branches have extracted features, the outputs of the two branches are , Connect and input features Add to the output to avoid gradient disappearance and get the final output of the feature distillation block FRB , , + represents element-by-element addition.

[0017] Preferably, the dual-channel large convolution kernel decomposition block in step S3 is characterized by comprising:

[0018] S31, dual-channel large convolution kernel decomposition block is divided into upper branch and lower branch, input feature , the initial features Divided into There are two parts. The upper branch is a large convolution kernel decomposition block with a size of 17×17. Obtained through 1×1 channel convolution , fuse the features of different channels and enhance the expressiveness of features, , Represents a 1×1 convolution operation, and then extracts the features of the local area of ​​the image through spatial local convolution , , Represents a 5×5 deep convolution, and then extracts the image remote features through spatial remote convolution , , Represents a deep dilated convolution with a kernel size of 5 and a dilation of 3, and then a skip connection is used to avoid gradient vanishing to obtain the output of the upper branch , , stands for element-wise multiplication;

[0019] S32, for the lower branch, decompose the large convolution kernel of size 31×31 into Obtained through 1×1 channel convolution , fuse the features of different channels and enhance the expressiveness of features, , Represents a 1×1 convolution operation, and then extracts the features of the local area of ​​the image through spatial local convolution , , Represents a 7×7 deep convolution, and then extracts the image remote features through spatial remote convolution , , Represents a deep dilated convolution with a kernel size of 7 and a dilation of 4, and then a skip connection is used to avoid gradient vanishing to obtain the output of the lower branch , , stands for element-wise multiplication;

[0020] S33. After the two branches have finished extracting features, the outputs of the two branches are weightedly connected to extract features of different scales, enhance the stability of the model, and obtain the final output of the dual-channel large convolution kernel decomposition block DBKB , , represents the weight, Stands for element-wise addition.

[0021] Preferably, the large core distillation block in step S4 is characterized in that the input image feature , use the feature distillation block to perform three distillations to extract deep features , , , represents the i-th distillation, Represents the output of the i-th distillation, using 1×1 convolution to , , , Distilled , , , , represents the i-th information distillation, Represents the output of the i-th information distillation. The distilled features of the four distillation outputs are connected and fused through 1×1 convolution to obtain the fused feature , , Represents a connection operation, It represents 1×1 convolution fusion, extracts global features through dual-channel large convolution kernel decomposition blocks, uses 1×1 convolution for feature transformation, and adds pixel normalization to ensure the stability of model training to obtain features. , , Represents a dual-channel large convolution kernel decomposition block, represents feature transformation, represents the pixel normalization operation, and finally a long skip connection is used to enhance the residual learning ability of the model to obtain the final output of the large core distillation block , , + represents skip connection.

[0022] Preferably, the large core distillation network in step S5 is characterized in that the input image feature , extract deep features by stacking N large core distillation blocks , represents the nth large core distillation block, Represents the output features of the m-th large core distillation block, all intermediate features are connected and fused and activated through a 1×1 convolution layer and GELU activation , Represents the output feature after all intermediate features are fused. Represents a connection operation, represents 1×1 convolution fusion, represents the GELU activation function, and finally uses skip connections to enhance residual learning, and obtains super-resolution images through the image reconstruction module , Stands for Image Reconstruction.

[0023] Compared with the prior art, the present invention has the following technical effects:

[0024] The large-core distillation network proposed in the technical solution provided by the present invention is composed of N large-core distillation blocks, a feature fusion module and an image reconstruction module; the large-core distillation block BKDB includes a feature distillation block FRB and a dual-channel large convolution kernel decomposition block DBKB; the feature distillation block FRB includes two branches, each of which can independently obtain important local features, thereby improving the model's acquisition of local features of the image; the dual-channel large convolution kernel decomposition block DBKB includes two branches, and the two branches respectively decompose a large convolution kernel into three parts: channel convolution, spatial local convolution and spatial remote convolution, thereby improving the model's acquisition of global features of the image, reducing the computational cost and the number of parameters, enhancing the feature expression capability, and obtaining the best effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flowchart of the automobile parts image super-resolution proposed by the present invention.

[0026] Figure 2 This is a structural diagram of the characteristic distillation block proposed by the present invention.

[0027] Figure 3 This is the dual-channel large convolution kernel decomposition block structure diagram proposed in the present invention.

[0028] Figure 4 This is a structural diagram of the large core distillation block proposed by the present invention.

[0029] Figure 5 This is a structural diagram of the large-core distillation network proposed in the present invention. DETAILED DESCRIPTION

[0030] The present invention aims to propose an automobile parts image super-resolution method based on a large kernel distillation network. First, a large kernel distillation network is constructed, which consists of N large kernel distillation blocks, a feature fusion module and an image reconstruction module; the large kernel distillation block BKDB includes a feature distillation block FRB and a dual-channel large convolution kernel decomposition block DBKB; the feature distillation block FRB includes two branches, each branch can independently obtain important local features, thereby improving the model's acquisition of local features of the image; the dual-channel large convolution kernel decomposition block DBKB includes two branches, and the two branches respectively decompose a large convolution kernel into three parts: channel convolution, spatial local convolution and spatial remote convolution, thereby improving the model's acquisition of global features of the image, reducing the computational cost and the number of parameters, enhancing the feature expression ability, and obtaining the best effect.

[0031] See also Figure 1 As shown, an automobile parts image super-resolution method based on a large core distillation network in an embodiment of the present application.

[0032] S1. Auto parts image collection: Use industrial cameras to shoot auto parts and collect low-resolution auto parts images.

[0033] S2. Construct a feature distillation block FRB. The feature distillation block FRB consists of two branches, the upper and lower branches. The upper and lower branches obtain feature information of different scales, use convolution operations, deep convolutions and ReLU activation functions to extract features, and use jump connections to enhance residual learning.

[0034] Further, if Figure 2 As shown in the figure, a feature distillation block is constructed, and the two branches can separately obtain important local features, thereby improving the model's acquisition of local features of the image:

[0035] S21. For the upper branch, input feature , the initial features It is obtained by 1×1 convolution and skip connection. , 1×1 convolution extracts richer feature information, and skip connections prevent gradient disappearance. , In represents the upper branch in the characteristic distillation process, + represents element-by-element addition, represents a 1×1 convolution operation, which is then obtained by depthwise convolution and skip connection. , , represents a 1×1 depthwise convolution, Represents a 3×3 deep convolution, using 1×1 deep convolution to fuse features, 3×3 deep convolution to enhance feature propagation and jump connection to capture local feature information while avoiding gradient disappearance, and then activates the upper branch through the ReLU activation function , ReLU activation function is used to improve the expressiveness of local features, , Represents the ReLU activation function;

[0036] S22. For the lower branch, input feature , the initial features It is obtained by 3×3 convolution and skip connection. , 3×3 convolution extracts richer local feature information, and skip connections prevent gradient disappearance. , In They represent the lower branches in the characteristic distillation process, + represents element-by-element addition, represents a 3×3 convolution operation, which is then obtained by depthwise convolution and skip connection. , , represents a 3×3 depth convolution, Represents a 5×5 deep convolution, using 3×3 deep convolution to enhance feature propagation, 5×5 deep convolution to avoid deep network degradation and jump connection to capture a wider range of local feature information, while avoiding gradient disappearance, and then activate the output of the lower branch through the ReLU activation function , ReLU activation function is used to improve the expressiveness of local features, , Represents the ReLU activation function;

[0037] S23, feature distillation After the two branches have extracted features, the outputs of the two branches are , Connect and input features Add to the output to avoid gradient disappearance and get the final output of the feature distillation block FRB , , + represents element-by-element addition.

[0038] S3. Construct a dual-channel large convolution kernel decomposition block DBKB. The dual-channel large convolution kernel decomposition block DBKB consists of two branches, the upper and lower branches are large convolution kernels of different scales, and the large convolution kernel is decomposed into 1×1 convolution, depth convolution and depth expansion convolution by convolution decomposition. Skip connection is used to enhance residual learning, and the output of the upper and lower branches is weighted fused to obtain the final output.

[0039] Further, if Figure 3 As shown in the figure, the dual-channel large convolution kernel decomposition block includes two branches, which decompose a large convolution kernel into three parts: channel convolution, spatial local convolution, and spatial remote convolution, thereby improving the model's acquisition of global image features, reducing computational costs and the number of parameters, and improving performance:

[0040] S31, dual-channel large convolution kernel decomposition block is divided into upper branch and lower branch, input feature , the initial features Divided into There are two parts. The upper branch is a large convolution kernel decomposition block with a size of 17×17. Obtained through 1×1 channel convolution , fuse the features of different channels and enhance the expressiveness of features, , Represents a 1×1 convolution operation, and then extracts the features of the local area of ​​the image through spatial local convolution , , Represents a 5×5 deep convolution, and then extracts the image remote features through spatial remote convolution , , Represents a deep dilated convolution with a kernel size of 5 and a dilation of 3, and then a skip connection is used to avoid gradient vanishing to obtain the output of the upper branch , , stands for element-wise multiplication;

[0041] S32, for the lower branch, decompose the large convolution kernel of size 31×31 into Obtained through 1×1 channel convolution , fuse the features of different channels and enhance the expressiveness of features, , Represents a 1×1 convolution operation, and then extracts the features of the local area of ​​the image through spatial local convolution , , Represents a 7×7 deep convolution, and then extracts the image remote features through spatial remote convolution , , Represents a deep dilated convolution with a kernel size of 7 and a dilation of 4, and then a skip connection is used to avoid gradient vanishing to obtain the output of the lower branch , , stands for element-wise multiplication;

[0042] S33. After the two branches have finished extracting features, the outputs of the two branches are weightedly connected to extract features of different scales, enhance the stability of the model, and obtain the final output of the dual-channel large convolution kernel decomposition block DBKB , , represents the weight, Stands for element-wise addition.

[0043] S4. Construct a large kernel distillation block BKDB, which includes three feature distillation blocks FRB, 1×1 convolution, connection operation, dual-channel large convolution kernel decomposition block DBKB and pixel normalization module.

[0044] Further, if Figure 4 As shown, a large core distillation block is constructed and the image features are input , use the feature distillation block to perform three distillations to extract deep features , , , represents the i-th distillation, Represents the output of the i-th distillation, using 1×1 convolution to , , , Distilled , , , , represents the i-th information distillation, Represents the output of the i-th information distillation. The distilled features of the four distillation outputs are connected and fused through 1×1 convolution to obtain the fused feature , , Represents a connection operation, It represents 1×1 convolution fusion, extracts global features through dual-channel large convolution kernel decomposition blocks, uses 1×1 convolution for feature transformation, and adds pixel normalization to ensure the stability of model training to obtain features. , , Represents a dual-channel large convolution kernel decomposition block, represents feature transformation, represents the pixel normalization operation, and finally a long skip connection is used to enhance the residual learning ability of the model to obtain the final output of the large core distillation block. , , + represents skip connection.

[0045] S5. Construct a large core distillation network BKDN, which includes N large core distillation blocks BKDB stacking, a feature fusion module and an image reconstruction module.

[0046] S6. Build an image super-resolution model, including input, feature extraction, feature distillation, feature fusion, image reconstruction and output.

[0047] S7. Input the low-resolution image of automobile parts into the image super-resolution model, and obtain the super-resolution image after a series of processing.

[0048] Further, an image super-resolution model is constructed, such as Figure 5 As shown in the figure, after collecting low-resolution images of automotive parts using an industrial camera, the images are input into the feature extraction module to obtain shallow features of the image. , the image features Input into the large core distillation network, and extract deep features through N stacked large core distillation blocks. This process can be expressed as , represents the nth large core distillation block, Represents the output features of the m-th large core distillation block. All intermediate features are connected and fused and activated through 1×1 convolutional layers and GELU activation. This process can be expressed as , Represents the output feature after all intermediate features are fused. Represents a connection operation, represents 1×1 convolution fusion, represents the GELU activation function. Finally, the shallow features and deep features are connected through skip connections to enhance residual learning, and the super-resolution image is obtained through the image reconstruction module. This process can be expressed as , Stands for Image Reconstruction.

[0049] The above are only preferred embodiments of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A super-resolution method for automobile parts images based on a large-core distillation network, characterized in that: The following steps are involved: S1, automobile parts image collection, using industrial cameras to shoot automobile parts and collect low-resolution automobile parts images; S2. Construct a feature distillation block FRB. The feature distillation block FRB consists of two branches, the upper and lower branches, which obtain feature information of different scales, use convolution operations, deep convolutions and ReLU activation functions to extract features, and use skip connections to enhance residual learning; S3. Construct a dual-channel large convolution kernel decomposition block DBKB. The dual-channel large convolution kernel decomposition block DBKB consists of two branches, the upper and lower branches are large convolution kernels of different scales, and the large convolution kernel is decomposed into 1×1 convolution, depth convolution and depth expansion convolution by using convolution decomposition. The skip connection is used to enhance residual learning, and the output of the upper and lower branches is weighted fused to obtain the final output. S4, construct a large kernel distillation block BKDB, which includes three feature distillation blocks FRB, 1×1 convolution, connection operation, dual-channel large convolution kernel decomposition block DBKB and pixel normalization module; S5. Construct a large core distillation network BKDN, which includes N large core distillation blocks BKDB stacking, feature fusion module and image reconstruction module; S6. Build an image super-resolution model, including input, feature extraction, feature distillation, feature fusion, image reconstruction and output; S7. Input the low-resolution image of automobile parts into the image super-resolution model, and obtain the super-resolution image after a series of processing.

2. According to the method for super-resolution of automobile parts images based on a large-core distillation network according to claim 1, in step S2, a feature distillation block is constructed, characterized in that: S21, the feature distillation block contains two branches. For the upper branch, input the feature , the initial features It is obtained by 1×1 convolution and skip connection. , 1×1 convolution extracts richer feature information, and skip connections prevent gradient disappearance. , In represents the upper branch in the characteristic distillation process, + represents element-by-element addition, represents a 1×1 convolution operation, which is then obtained by depthwise convolution and skip connection. , , represents a 1×1 depthwise convolution, Represents a 3×3 deep convolution, using 1×1 deep convolution to fuse features, 3×3 deep convolution to enhance feature propagation and jump connection to capture local feature information while avoiding gradient disappearance, and then activates the upper branch through the ReLU activation function , ReLU activation function is used to improve the expressiveness of local features, , Represents the ReLU activation function; S22. For the lower branch, input feature , the initial features It is obtained by 3×3 convolution and skip connection. , 3×3 convolution extracts richer local feature information, and skip connections prevent gradient disappearance. , In They represent the lower branches in the characteristic distillation process, + represents element-by-element addition, represents a 3×3 convolution operation, which is then obtained by depthwise convolution and skip connection. , , represents a 3×3 depth convolution, Represents a 5×5 deep convolution, using 3×3 deep convolution to enhance feature propagation, 5×5 deep convolution to avoid deep network degradation and jump connection to capture a wider range of local feature information, while avoiding gradient disappearance, and then activate the output of the lower branch through the ReLU activation function , ReLU activation function is used to improve the expressiveness of local features, , Represents the ReLU activation function; S23, feature distillation After the two branches have extracted features, the outputs of the two branches are , Connect and input features Add to the output to avoid gradient disappearance and get the final output of the feature distillation block FRB , , + represents element-by-element addition.

3. According to the method for super-resolution of automobile parts images based on a large kernel distillation network according to claim 1, in step S3, a dual-channel large convolution kernel decomposition block is constructed, characterized in that: S31, dual-channel large convolution kernel decomposition block is divided into upper branch and lower branch, input feature , the initial features Divided into There are two parts. The upper branch is a large convolution kernel decomposition block with a size of 17×17. Obtained through 1×1 channel convolution , fuse the features of different channels and enhance the expressiveness of features, , Represents a 1×1 convolution operation, and then extracts the features of the local area of ​​the image through spatial local convolution , , Represents a 5×5 deep convolution, and then extracts the image remote features through spatial remote convolution , , Represents a deep dilated convolution with a kernel size of 5 and a dilation of 3, and then a skip connection is used to avoid gradient vanishing to obtain the output of the upper branch , , stands for element-wise multiplication; S32, for the lower branch, decompose the large convolution kernel of size 31×31 into Obtained through 1×1 channel convolution , fuse the features of different channels and enhance the expressiveness of features, , Represents a 1×1 convolution operation, and then extracts the features of the local area of ​​the image through spatial local convolution , , Represents a 7×7 deep convolution, and then extracts the image remote features through spatial remote convolution , , Represents a deep dilated convolution with a kernel size of 7 and a dilation of 4, and then a skip connection is used to avoid gradient vanishing to obtain the output of the lower branch , , stands for element-wise multiplication; S33. After the two branches have finished extracting features, the outputs of the two branches are weightedly connected to extract features of different scales, enhance the stability of the model, and obtain the final output of the dual-channel large convolution kernel decomposition block DBKB , , represents the weight, Stands for element-wise addition.

4. According to the method for super-resolution of automobile parts images based on a large core distillation network according to claim 1, in step S4, a large core distillation block is constructed, characterized in that: Input image features , use the feature distillation block to perform three distillations to extract deep features , , , represents the i-th distillation, Represents the output of the i-th distillation, using 1×1 convolution to , , , Distilled , , , , represents the i-th information distillation, Represents the output of the i-th information distillation. The distilled features of the four distillation outputs are connected and fused through 1×1 convolution to obtain the fused feature , , Represents a connection operation, It represents 1×1 convolution fusion, extracts global features through dual-channel large convolution kernel decomposition blocks, uses 1×1 convolution for feature transformation, and adds pixel normalization to ensure the stability of model training to obtain features. , , Represents a dual-channel large convolution kernel decomposition block, represents feature transformation, represents the pixel normalization operation, and finally a long skip connection is used to enhance the residual learning ability of the model to obtain the final output of the large core distillation block , , + represents skip connection.

5. According to the method for super-resolution of automobile parts images based on a large-core distillation network according to claim 1, in step S5, a large-core distillation network is constructed, characterized in that: Input image features , extract deep features by stacking N large core distillation blocks , represents the nth large core distillation block, Represents the output features of the m-th large core distillation block, all intermediate features are connected and fused and activated through a 1×1 convolution layer and GELU activation , Represents the output feature after all intermediate features are fused. Represents a connection operation, represents 1×1 convolution fusion, represents the GELU activation function, and finally uses skip connections to enhance residual learning, and obtains super-resolution images through the image reconstruction module , Stands for Image Reconstruction.

Images