A super-resolution transformer-based automobile parts detection method

By using super-resolution Transformer in automotive parts detection combined with cross-modal attention CMA and category-content multi-head attention module CCMSA, the problem of computational complexity and performance balance in single-image super-resolution technology is solved, and the detection is improved The accuracy and efficiency of the car parts can achieve more stable and efficient inspection.

CN118552956BActive Publication Date: 2024-12-03SHANDONG BLUEBIRD IND INTERNET CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411017290.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-12-03
Estimated Expiration
2044-07-29

AI Technical Summary

Technical Problem

Existing single image super-resolution technology has computational complexity and performance balance problems in automotive parts detection, resulting in limited detection accuracy and efficiency, especially poor performance when processing incomplete or noisy data.

Method used

Adopt the automotive parts detection method based on super-resolution Transformer, combining cross-modal attention CMA and category-content multi-head attention module CCMSA, to improve the robustness and universality of the model by integrating different modal information and establishing global feature connections. to enhance the accuracy and efficiency of detection.

Benefits of technology

It improves the accuracy and efficiency of automotive parts inspection, enhances the stability of the model when processing incomplete or noisy data, reduces errors caused by human factors, and realizes the automation and intelligence of the inspection process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118552956B_ABST
    Figure CN118552956B_ABST
Patent Text Reader

Abstract

The present invention proposes a method for detecting automobile parts based on super-resolution Transformer, and relates to the technical field of image recognition. The present invention proposes a vehicle parts detection process, including a convolution layer, an attention layer and an upsampling convolution module, the attention layer includes a Transformer module, the Transformer module includes a cross-modal attention module and a category-content multi-head attention module, and proposes a cross-modal attention CMA and a category-content multi-head attention module CCMSA. The cross-modal attention CMA focuses on the association between different modalities and can effectively integrate information from different modalities. The category-content multi-head attention module CCMSA can use tokens that are far apart but similar to enhance input features, pay attention to each category, and then perform content classification, which exceeds the limitation of local windows.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of image recognition, and in particular relates to an automobile parts detection method based on super-resolution Transformer. Background Art

[0002] In vehicle inspection, parts inspection is a key link in vehicle qualification. Using cross-modal attention CMA and category-content multi-head attention module CCMSA to inspect vehicle parts can not only accurately capture the subtle features of vehicle parts and improve the accuracy of inspection, but also its non-contact and non-destructive inspection method ensures the integrity of the sample. At the same time, the rapid inspection of cross-modal attention CMA and category-content multi-head attention module CCMSA technology provides a guarantee for whether the vehicle inspection is qualified. The application of this technology not only improves the efficiency and accuracy of vehicle parts inspection, but also injects new vitality into the innovative development of vehicle inspection.

[0003] Using super-resolution technology to replace manual detection of cross-modal attention CMA and category-content multi-head attention module CCMSA detection not only greatly improves detection efficiency and accuracy, reduces errors caused by human factors, but also realizes automation and intelligence of the detection process, reducing labor costs. With the continuous advancement of technology, this alternative method will bring more comprehensive and accurate support to the field of vehicle detection and promote the innovative development of vehicle detection.

[0004] Cross-modal attention (CMA) and category-content multi-head attention module (CCMSA) detection aim to recover high-resolution images from low-resolution images. However, the currently popular single image super-resolution has limitations in balancing accuracy and managing increased computational complexity, which leads to computational load and reduces the performance of single image super-resolution. Summary of the invention

[0005] The present invention provides an automobile parts detection method based on super-resolution Transformer, aiming to propose cross-modal attention CMA and category-content multi-head attention module CCMSA. The cross-modal attention CMA can effectively fuse information from different modalities to construct a richer and more comprehensive feature representation, improve the robustness and generalization ability of the model, and enhance the stability and accuracy of the model when processing incomplete or noisy data. The category-content multi-head attention module CCMSA can enhance input features by using tokens that are far apart but similar, and pay attention to them in each category, and then pay attention to them by content classification, which goes beyond the limitation of local windows, establishes global connections between similar features, and improves the accuracy and efficiency of automobile parts detection.

[0006] The present invention aims to improve on the basis of single image super-resolution and provide an automobile parts detection method based on super-resolution Transformer, comprising the following steps:

[0007] S1. Preparation of automobile parts image dataset: Use industrial cameras to take photos of automobile parts to obtain automobile parts images, mark the positions that need to be detected, and mark other objects as irrelevant targets. After the marking is completed, the automobile parts image dataset is obtained;

[0008] S2, build convolutional layers, including 3×3 convolutions;

[0009] S3. Construct an attention layer, which includes L Transformer modules. Input image features to the attention layer, and the image features are calculated through L Transformer modules. The Transformer modules include a cross-modal attention module CMA and a category-content multi-head self-attention module CCMSA.

[0010] S4, construct a cross-modal attention module CMA, including linear transformation, calculation of attention weights, calculation of attention outputs, and fusion of enhanced features;

[0011] S5. Construct the category-content multi-head attention module CCMSA, including Classify, MSA, and UnClassify;

[0012] S6. Build an upsampling convolution module, including 3×3 convolution and upsampling.

[0013] Preferably, in step S2, for the convolutional layer, the input image , extract features and , and , , and Represents input features and The height, width and channels of the convolutional layer can extract abstract features and transmit them to the next module, thereby enhancing the representation ability of the image.

[0014] Preferably, in step S4, for the cross-modal attention module CMA, the modality features and ,Will and Through linear transformation, they are mapped to Query, Key, and Value respectively. , , ,in, , , is the weight matrix of the linear transformation, , , is the bias vector, and then use and Calculate the attention weights, usually using the dot product attention mechanism, ,in, yes The dimension is used to scale the dot product result and apply the attention weight to , get the output of cross-modal attention CMA, , combine the cross-modal attention CMA output with the modal original features Fusion is performed to generate enhanced features. ,in, and It is the learned weight used to balance the attention output and the original features, and output the enhanced features.

[0015] Preferably, in step S5, for the category-content multi-head attention module CCMSA, the input features obtained by CMA are used And the enhanced features , the category information of each pixel is implicitly fused. In order to classify the input features, we divide each pixel into Different categories, by integrating the category information of each pixel, classifying the input features according to the most similar enhanced features, ,in is the enhanced feature obtained, is the characteristic number, is the atomic number, exist is the maximum value, pixel Included in the collection , which means With enhanced features The labels belong to the same category. Therefore, each category is regarded as an irregularly shaped window containing the same category label. First, a multi-head self-attention mechanism is applied to extract important features and enhance them. , , , and then perform multi-head self-attention calculation, , in each category, in order to combine local and global information, the calculated category Further divided into categories according to content , , , where category Include Each content is flattened and connected to form , and then divide into subcategories , after division, all subcategories have the same fixed size , using a multi-head attention mechanism within each category to identify content-specific features, , , , and then perform multi-head self-attention calculation, ; The procedure for the category-content multi-head attention module CCMSA can be expressed as , , , ,in, Operation is and , The combination of input features Divided into different categories and further divided into sub-category collections , then we can convert each As an attention group, and perform multi-head self-attention calculations in each group. Finally, we use The operation puts each pixel back to its original position in the feature map, since and The sorting operation involves a random shuffle, so each Can be considered a category A random sample of The tokens in can still be distributed throughout the feature map, thereby maintaining the global receptive field, and finally a 3×3 convolution operation and upsampling operation are performed to output the image.

[0016] Compared with the prior art, the present invention has the following technical effects:

[0017] The technical solution provided by the present invention proposes cross-modal attention CMA and category-content multi-head attention module CCMSA. The cross-modal attention can effectively fuse information from different modalities to construct a richer and more comprehensive feature representation, improve the robustness and generalization ability of the model, and enhance the stability and accuracy of the model when processing incomplete or noisy data. The category-content multi-head attention module CCMSA can use tokens that are far apart but similar to enhance input features, pay attention to them in each category, and then pay attention to them by content classification, transcending the limitations of local windows, establishing global connections between similar features, and improving the accuracy and efficiency of automobile parts detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1The present invention provides an automobile parts detection flow chart.

[0019] Figure 2 This is the attention layer structure diagram provided by the present invention.

[0020] Figure 3 This is a structural diagram of the cross-modal attention module CMA provided by the present invention.

[0021] Figure 4 This is the structure diagram of the category-content multi-head attention module CCMSA provided by the present invention. DETAILED DESCRIPTION

[0022] The present invention aims to propose a method for automobile parts detection based on super-resolution Transformer, and proposes cross-modal attention CMA and category-content multi-head attention module CCMSA. The cross-modal attention CMA can effectively fuse information from different modalities to construct a richer and more comprehensive feature representation, improve the robustness and generalization ability of the model, and enhance the stability and accuracy of the model when processing incomplete or noisy data. The category-content multi-head attention module CCMSA can use tokens that are far apart but similar to enhance input features, pay attention to them in each category, and then pay attention to them by content classification, which goes beyond the limitation of local windows, establishes global connections between similar features, and improves the accuracy and efficiency of automobile parts detection.

[0023] See also Figure 1 As shown, a method for detecting automobile parts based on super-resolution Transformer in an example of this application:

[0024] S1. Production of automobile parts image dataset: Use industrial cameras to take photos of vehicle parts to obtain 3,000 images of vehicle parts. Label the rim welds in the automobile parts and mark other objects as irrelevant targets. After labeling, perform Mixup data enhancement to obtain a vehicle parts image dataset with a total of 5,000 images. The dataset and validation set are divided into 7 to 3.

[0025] S2, build a convolution layer, including a 3×3 convolution;

[0026] S3. Construct an attention layer, which includes L Transformer modules. Input image features to the attention layer, and the image features are calculated through L Transformer modules. The Transformer modules include a cross-modal attention module CMA and a category-content multi-head attention module CCMSA.

[0027] S4, construct a cross-modal attention module CMA, including linear transformation, calculation of attention weights, calculation of attention outputs, and fusion of enhanced features;

[0028] S5. Construct the category-content multi-head attention module CCMSA, including Classify, MSA, and UnClassify;

[0029] S6. Build an upsampling convolution module, including a 3×3 convolution and 1 upsampling;

[0030] The image is input into the convolution layer, image features are extracted for training, and the image features are input into the Transformer layer. Information of different modalities is fused through the cross-modal attention module CMA. The input features and the features enhanced by the cross-modal attention CMA are input into the category-content multi-head attention module CCMSA, and attention is paid to each category. Then attention is paid to the content classification, and a global connection between similar features is established. The output image enters the upsampling convolution module for convolution and upsampling. Finally, the final image is output.

[0031] Further, in step S2, for the convolutional layer, the input image , extract features and , and , , and Represents input features and The height, width, and channels of the network are both 640, and the number of channels is 224. The convolutional layer can extract abstract features and transmit them to the next module, thereby enhancing the image representation capability.

[0032] Further, in step S4, for the cross-modal attention module CMA, its structure is as follows Figure 3 As shown, the modal characteristics and ,Will and Through linear transformation, they are mapped to Query, Key, and Value respectively. , , ,in, , , is the weight matrix of the linear transformation, , , is the bias vector, and then use and Calculate the attention weights, usually using the dot product attention mechanism, ,in, yes The dimension is used to scale the dot product result and apply the attention weight to , get the output of cross-modal attention CMA, , this calculation is consistent with Figure 2 In Correspondingly, the cross-modal attention CMA output is combined with the original features of the modality Fusion is performed to generate enhanced features. , this calculation is consistent with Figure 2 In Corresponding to, among which, and It is the learned weight used to balance the attention output and the original features, and output the enhanced features.

[0033] Furthermore, in step S5, for the category-content multi-head attention module CCMSA, the input features obtained by CMA are used And the enhanced features , the category information of each pixel is implicitly fused. In order to classify the input features, we divide each pixel into Different categories, by integrating the category information of each pixel, classifying the input features according to the most similar enhanced features, ,in is the enhanced feature obtained, is the characteristic number, is the atomic number, exist is the maximum value, pixel Included in the collection , which means With enhanced features The labels belong to the same category. Therefore, each category is regarded as an irregularly shaped window containing the same category label. First, a multi-head self-attention mechanism is applied to extract important features and enhance them. , , , and then perform multi-head self-attention calculation, , in each category, in order to combine local and global information, the calculated category Further divided into categories according to content , , , where category Include Each content is flattened and connected to form , and then divide into subcategories , after division, all subcategories have the same fixed size , using a multi-head attention mechanism within each category to identify content-specific features, , , , and then perform multi-head self-attention calculation, ; The procedure for the category-content multi-head attention module CCMSA can be expressed as , this calculation is consistent with Figure 4 In correspond, , , these two calculations are Figure 4 correspond, , this calculation is consistent with Figure 4 Corresponding to, among which, Operation is and , The combination of input features Divided into different categories and further divided into sub-category collections , then we can convert each As an attention group, and perform multi-head self-attention calculations in each group. Finally, we use The operation puts each pixel back to its original position in the feature map, since and The sorting operation involves a random shuffle, so each Can be considered a category A random sample of The tokens in can still be distributed throughout the feature map, thereby maintaining the global receptive field, and finally a 3×3 convolution operation and upsampling operation are performed to output the image.

[0034] The above are only preferred embodiments of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A super-resolution Transformer-based automobile parts detection method. It is characterized in that The following steps are involved: S1. Preparation of automobile parts image dataset: Use industrial cameras to take photos of automobile parts to obtain automobile parts images, mark the positions that need to be detected, and mark other objects as irrelevant targets. After the marking is completed, the automobile parts image dataset is obtained; S2, build convolutional layers, including 3×3 convolutions; S3. Construct an attention layer, which includes L Transformer modules. Input image features to the attention layer, and the image features are calculated through L Transformer modules. The Transformer modules include a cross-modal attention module CMA and a category-content multi-head attention module CCMSA. S4, construct a cross-modal attention module CMA, including linear transformation, calculation of attention weights, calculation of attention outputs, and fusion of enhanced features; S5. Construct the category-content multi-head attention module CCMSA, including Classify, MSA, and UnClassify; S6. Build an upsampling convolution module, including 3×3 convolution and upsampling; In step S4, for the cross-modal attention module CMA, the modality feature X A and X B , X A and X B Through linear transformation, they are mapped to Query, Key, and Value respectively. in, is the weight matrix of the linear transformation, is the bias vector, and then use Q A and K B Calculate the attention weights, using the dot product attention mechanism, Among them, d k It's K B The dimension of V is used to scale the dot product result and apply the attention weight to V B , get the output of cross-modal attention CMA, O AB =A AB V B , combine the cross-modal attention CMA output with the modality original feature X A Fusion is performed to generate enhanced features, X′=αO AB +βX A , where α and β are the learned weights used to balance the attention output and the original features, outputting the enhanced features; In step S5, for the category-content multi-head attention module CCMSA, the input feature X obtained by CMA is used A and the enhanced features X′, the category information of each pixel is implicitly fused to classify the input features and divide each pixel into θ 1 ,θ 2 ,......,θ M Different categories, by integrating the category information of each pixel, classifying the input features according to the most similar enhanced features, θ i ={x j |argmax k (X′ jk )=i}, where X′∈R N×M is the enhanced feature obtained, N is the feature number, M is the number of atoms, and X′ ji In X j1 ′、X j2 ',...'X jM ′ is the maximum value, pixel x j is classified into the set θ i , which means x j The i-th label of the enhanced feature belongs to the same category. Therefore, each category is regarded as an irregularly shaped window containing the same category label. First, a multi-head self-attention mechanism is applied to extract important features and enhance them. i =θ i W Q , K i =θ i W K , V i =θ i W V , and then perform multi-head self-attention calculation, In each category, in order to combine local and global information, the calculated category θ is further divided into categories according to content Among them, the category θ i Contains i Each content is flattened and connected to form Then divide into subcategories After partitioning, all subcategories have the same fixed size n s , a multi-head attention mechanism is used within each category to identify content-specific features, Q j =φ j W Q , K j =φ j W K , V j =φ j W V , and then perform multi-head self-attention calculation, The procedure for the category-content multi-head attention module CCMSA is expressed as Among them, the Classify operation is θ i ={x j |argmaxk(X′ jk )=i} and The combination of input feature X A Divided into different categories and further divided into sub-category collections Then, each is considered as an attention group, and multi-head self-attention calculation is performed within each group. Finally, the UnClassify operation is used to put each pixel back to its original position in the feature map. and The sorting operation involves a random shuffle, so each Considered as a certain category θ i A random sample of The tokens in are still distributed throughout the feature map, thereby maintaining the global receptive field, and finally a 3×3 convolution operation and upsampling operation are performed to output the image.

2. According to claim 1, a super-resolution Transformer-based automobile parts detection method, It is characterized in that In step S2, for the convolution module, the image R is input and the feature X is extracted A and X B , X A and X B ∈R H×W×C , H, W and C represent the input features X A and X B The height, width and channels of the convolutional layer are used to extract abstract features and transmit them to the next module, thereby enhancing the representation ability of the image.

Citation Information

Patent Citations

  • Dual-modal target detection method and system based on cross-modal attention mechanism fusion

    CN117422971A

  • Deep learning-based anomaly detection in images

    WO2021191908A1