Weld defect identification method based on visual Transform

Abstract

The invention discloses a visual Transform-based welding seam defect identification method, which improves the welding seam identification precision and solves the problem of insufficient diversity of training data of welding seam defects. The system is characterized by being provided with the following modules in advance: a data preparation module, a model design module and a training and optimizing module, a standard image data set is obtained through a training and optimizing module, an obtained new image is added into the standard image data set through the training and optimizing module, and model parameters are optimized according to a test result, so that after an image of a to-be-detected welding seam is input, a model design module is combined with the input image to divide the input image into reasonable small blocks; the method comprises the following steps: firstly, firstly, forming a corresponding Swin Transformer module, then, comparing a Petri feature fusion module with a standard image data set to carry out first-time welding seam detection, finally, comparing a multi-layer network detection head with the standard image data set to carry out second-time welding seam detection, and finally, giving out the type of the welding seam and the position of the welding seam in an original image.

Description

Technical Field

[0001] The present invention relates to the technical field of weld defect recognition, and in particular to a weld defect recognition method based on a visual Transformer. Background Art

[0002] Welding is a common connection method in engineering structures, and its quality directly affects the strength, toughness and safety of the structure. Weld defects (such as pores, slag inclusions, cracks, etc.) are common problems in the welding process. Traditional detection methods mostly rely on manual inspection, which is not only inefficient but also easily affected by human factors. Therefore, more and more researchers have begun to explore the application of computer vision and deep learning technology in weld defect detection.

[0003] The existing technology uses the YOLO algorithm to detect weld defects, but the algorithm is less effective when detecting small targets in complex backgrounds. Factors such as complex textures and lighting changes around the weld can easily interfere with YOLO's feature extraction and increase the false detection rate. At the same time, its single detection characteristics make it difficult to fully utilize the contextual information of the overall structure of the weld, which makes the accuracy and comprehensiveness of weld judgment irrational.

[0004] In addition, the YOLO algorithm is used for weld defect detection. The accuracy of its model detection is highly dependent on high-quality labeled data. Insufficient data or inaccurate labeling will directly affect the detection effect.

[0005] As an emerging deep learning architecture, Visual Transformer (ViT) has achieved remarkable results in tasks such as image classification. Unlike the YOLO algorithm, ViT can capture long-range dependencies through the self-attention mechanism and effectively process large-scale image data, which enables it to show better results when processing complex image features; however, when using the ViT model for image classification tasks, the amount of self-attention calculation is proportional to the square of the pixels, which leads to relatively slow calculation processing and low image processing efficiency. Summary of the invention

[0006] In view of the above problems, the present invention provides a weld defect recognition method based on visual Transformer, which improves the accuracy of weld recognition and can solve the problem of insufficient diversity of training data for weld defects through adaptive data enhancement.

[0007] A weld defect recognition method based on visual Transformer, characterized in that it is pre-set with the following modules: a data preparation module, a model design module, and a training and optimization module;

[0008] The data preparation module includes weld image data acquisition, annotation and preprocessing, and finally obtains a standard data set, and obtains a standard image data set through the data preparation module;

[0009] The model design module first divides the input image into blocks and stretches it into a sequence form, embeds the position code, and then sends it to the Swin Transformer to extract features; secondly, through the Petri feature fusion module, the low-level specific detail features are fused with the high-level abstract semantic features, and the fused features are passed through the region recommendation network to perform the first weld detection, give the region recommendation box, and mark the region of interest where the weld may appear. Finally, the multi-layer network detection head performs a second weld detection on the region of interest, and finally gives the category of the weld and the position in the original image;

[0010] The training and optimization module uses a standard image dataset to train the model in the model design module, introduces cross-validation and hyperparameter tuning to obtain the best training effect, incorporates a dynamic learning rate adjustment algorithm to adapt to the needs of different training stages, and optimizes model parameters according to test results;

[0011] A standard image data set is obtained through the training and optimization module. At the same time, the new image obtained is added to the standard image data set through the training and optimization module. At the same time, the model parameters are optimized according to the test results, so that after the image of the weld to be detected is input, the model design module combines the input image to divide it into reasonable small blocks, and then forms the corresponding SwinTransformer module. Then, the first weld detection is carried out by comparing the Petri feature fusion module with the standard image data set, and finally the second weld detection is carried out by comparing the multi-layer network detection head with the standard image data set, and finally the category of the weld and its position in the original image are given.

[0012] It is further characterized by:

[0013] Weld image data acquisition uses a high-resolution camera to capture weld images under different welding environment conditions. The images include normal weld images and weld images with various defects such as pores, slag inclusions, and cracks.

[0014] Weld image data annotation: Use annotation tools to annotate the collected weld images and generate defect locations and category labels. The category labels are used to annotate defect names such as pores, slag inclusions, and cracks.

[0015] Weld image data preprocessing includes denoising, image enhancement and normalization. It obtains diversified training samples by denoising and / or enhancing the weld images, and then normalizes the images corresponding to the training samples to facilitate model training.

[0016] Denoising uses image processing techniques to reduce image noise and improve image quality;

[0017] Image enhancement generates diverse training samples for applying data enhancement technology to enhance the generalization ability of the model;

[0018] Normalization is to normalize the image so that the pixel values ​​are adjusted to the same range;

[0019] The Petri feature fusion module extracts and fuses feature information from different levels. The Petri feature fusion module comprehensively utilizes low-level detail information and high-level semantic information to achieve accurate defect detection. The Petri feature fusion module first extracts features from multiple encoding layers of the Swin Transformer module. Finally, the contribution of features at different levels is dynamically adjusted in combination with the self-attention mechanism. By calculating the correlation between features, the most helpful features for classification are automatically selected for fusion.

[0020] The features extracted by the Petri feature fusion module from multiple encoding layers of the Swin Transformer module are expressed as high-dimensional feature vectors or feature maps, with different dimensions and structures;

[0021] The features extracted by the Petri feature fusion module are classified into low-level features, middle-level features and high-level features. The low-level features are responsible for capturing the detailed information of the input image; the middle-level features contain certain semantic information and can capture more structured parts; the high-level features are highly abstract and are used to represent the overall content and contextual information of the image, and can understand the defect features in a larger range;

[0022] The multi-layer network detection head is used to accurately classify and locate weld defects. The multi-layer network detection head uses a multi-layer network structure to efficiently extract and classify features of weld images. The multi-layer network detection head includes a convolution layer, a pooling layer, an activation function, a fully connected layer, and a Softmax layer.

[0023] The convolutional layer uses a multi-layer convolutional neural network to extract features from the input weld image. These convolutional layers can capture detailed information about edges and textures, providing a basis for subsequent classification.

[0024] The pooling layer reduces the dimension of the feature map by downsampling, retaining the most important information and improving computational efficiency;

[0025] The activation function usually uses a linear rectifier unit as the activation function, introducing nonlinear characteristics to help the network learn more complex patterns;

[0026] After feature extraction, the extracted features are mapped to classification outputs through fully connected layers, which are responsible for synthesizing the input information to generate the final classification results;

[0027] Softmax layer is the last layer, and the Softmax function is used to convert the model output into a probability distribution, so that the probabilities of each category can be compared with each other, which facilitates the determination of the final classification results.

[0028] After adopting the above technical scheme, it performs adaptive data enhancement through training and optimization modules to solve the problem of insufficient diversity of training data for weld defects; and when setting the model for the detection image, it is divided into blocks to generate a SwinTransformer model, and the first weld detection is performed by comparing the Petri feature fusion module with the standard image data set, and finally the second weld detection is performed by comparing the multi-layer network detection head with the standard image data set, and finally the category of the weld and the position in the original image are given; the weld defect recognition method based on visual Transformer reduces the complexity of calculation through double detection and block detection, improves the accuracy of weld recognition, and can solve the problem of insufficient diversity of training data for weld defects through adaptive data enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 The working flow chart of the model design module of the present invention. DETAILED DESCRIPTION

[0030] A weld defect recognition method based on visual Transformer, which is pre-set with a data preparation module, a model design module, and a training and optimization module;

[0031] A standard image data set is obtained through the training and optimization module. At the same time, the new image obtained is added to the standard image data set through the training and optimization module. At the same time, the model parameters are optimized according to the test results, so that after the image of the weld to be detected is input, the model design module combines the input image to divide it into reasonable small blocks, and then forms the corresponding SwinTransformer module. Then, the first weld detection is carried out by comparing the Petri feature fusion module with the standard image data set, and finally the second weld detection is carried out by comparing the multi-layer network detection head with the standard image data set, and finally the category of the weld and its position in the original image are given.

[0032] The data preparation module includes weld image data acquisition, annotation and preprocessing, and finally obtains a standard data set. The standard image data set is obtained through the data preparation module;

[0033] Weld image data acquisition uses a high-resolution camera to capture weld images under different welding environmental conditions (such as temperature, humidity, welding speed, etc.). The images include normal weld images and weld images with various defects such as pores, slag inclusions, and cracks, ensuring the diversity and comprehensiveness of the data;

[0034] Weld image data annotation uses annotation tools (such as LabelImg, VGG Image Annotator, etc.) to annotate the collected weld images, generate defect locations and category labels, and annotate the defect names such as pores, slag inclusions, and cracks for subsequent training;

[0035] Weld image data preprocessing includes denoising, image enhancement and normalization. It obtains diversified training samples by denoising and / or enhancing the weld images, and then normalizes the images corresponding to the training samples to facilitate model training.

[0036] Denoising uses image processing techniques (such as Gaussian filtering, median filtering, etc.) to reduce image noise and improve image quality;

[0037] Image augmentation generates diverse training samples for applying data augmentation techniques (such as rotation, scaling, flipping, and color jittering) to enhance the generalization ability of the model;

[0038] Normalization is to normalize the image so that the pixel values ​​are adjusted to the same range.

[0039] Model design module (its workflow is shown in Figure 1 ), firstly, the input image is divided into blocks and stretched into a sequence, embedded with position code, and then sent to the Swin Transformer module to extract features; secondly, the low-level specific detail features are fused with the high-level abstract semantic features through the Petri feature fusion module, and the fused features are passed through the region recommendation network to perform the first weld detection, give the region recommendation box, and identify the region of interest where the weld may appear. Finally, the multi-layer network detection head performs a second weld detection on the region of interest, and finally gives the category of the weld and its position in the original image. In the specific implementation, the multi-layer network detection head is named LoCategory detection head;

[0040] The Petri feature fusion module extracts and fuses feature information from different levels. The Petri feature fusion module comprehensively utilizes low-level detail information and high-level semantic information to achieve accurate defect detection. The Petri feature fusion module first extracts features from multiple encoding layers of the Swin Transformer module. Finally, it dynamically adjusts the contribution of features at different levels by combining the self-attention mechanism. By calculating the correlation between features, it automatically selects the features that are most helpful for classification for fusion.

[0041] The features extracted by the Petri feature fusion module from multiple encoding layers of the Swin Transformer module are expressed as high-dimensional feature vectors or feature maps, with different dimensions and structures;

[0042] The features extracted by the Petri feature fusion module are classified into:

[0043] Low-level features: responsible for capturing detailed information of the input image, such as texture and edges;

[0044] Middle-level features: contain certain semantic information and can capture more structured parts, such as different types of weld structures;

[0045] High-level features: highly abstract, used to characterize the overall content and contextual information of the image, and able to understand defect features in a larger range.

[0046] The LoCategory detection head is used to accurately classify and locate weld defects. The LoCategory detection head uses a multi-layer network structure to efficiently extract and classify weld images. The composition of the LoCategory detection head is as follows:

[0047] Convolutional layer, using multi-layer convolutional neural network (CNN) to extract features from the input weld image. These convolutional layers can capture detailed information of edges and textures, providing a basis for subsequent classification;

[0048] Pooling layer: Reduce the dimension of the feature map by downsampling (such as maximum pooling), retain the most important information and improve computational efficiency;

[0049] Activation function: ReLU (rectifier linear unit) is usually used as the activation function to introduce nonlinear characteristics to help the network learn more complex patterns;

[0050] Fully connected layers: After feature extraction, the extracted features are mapped to the classification output through fully connected layers. These layers are responsible for synthesizing the input information to generate the final classification results;

[0051] Softmax layer: In the last layer, the Softmax function is used to convert the model output into a probability distribution, so that the probabilities of each category can be compared with each other, making it easier to determine the final classification results.

[0052] The training and optimization module uses a standard image dataset to train the model in the model design module, introduces cross-validation and hyperparameter tuning to obtain the best training effect, incorporates a dynamic learning rate adjustment algorithm to adapt to the needs of different training stages, and optimizes model parameters according to test results to achieve better weld recognition performance.

[0053] The weld defect recognition method based on visual Transformer performs adaptive data enhancement through training and optimization modules to solve the problem of insufficient diversity of training data for weld defects; and separates the detection image into blocks to generate a Swin Transformer model when setting the model, performs the first weld detection through Petri feature fusion module and standard image data set comparison, and finally performs the second weld detection through multi-layer network detection head and standard image data set comparison, and finally gives the category of the weld and its position in the original image; the weld defect recognition method based on visual Transformer reduces the complexity of calculation through double detection and block detection, improves the accuracy of weld recognition, and can solve the problem of insufficient diversity of training data for weld defects through adaptive data enhancement.

[0054] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0055] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A weld defect recognition method based on visual Transformer, characterized in that: It is pre-set with the following modules: data preparation module, model design module, and training and optimization module; The data preparation module includes weld image data acquisition, annotation and preprocessing, and finally obtains a standard data set, and obtains a standard image data set through the data preparation module; The model design module first divides the input image into blocks and stretches it into a sequence form, embeds the position code, and then sends it to the Swin Transformer to extract features; secondly, through the Petri feature fusion module, the low-level specific detail features are fused with the high-level abstract semantic features, and the fused features are passed through the region recommendation network to perform the first weld detection, give the region recommendation box, and mark the region of interest where the weld may appear. Finally, the multi-layer network detection head performs a second weld detection on the region of interest, and finally gives the category of the weld and the position in the original image; The training and optimization module uses a standard image dataset to train the model in the model design module, introduces cross-validation and hyperparameter tuning to obtain the best training effect, incorporates a dynamic learning rate adjustment algorithm to adapt to the needs of different training stages, and optimizes model parameters according to test results; A standard image data set is obtained through the training and optimization module. At the same time, the new image obtained is added to the standard image data set through the training and optimization module. At the same time, the model parameters are optimized according to the test results, so that after the image of the weld to be detected is input, the model design module combines the input image to divide it into reasonable small blocks, and then forms the corresponding SwinTransformer module. Then, the first weld detection is carried out by comparing the Petri feature fusion module with the standard image data set, and finally the second weld detection is carried out by comparing the multi-layer network detection head with the standard image data set, and finally the category of the weld and its position in the original image are given.

2. According to claim 1, a weld defect recognition method based on visual Transformer is characterized in that: Weld image data acquisition uses a high-resolution camera to capture weld images under different welding environment conditions. The images include normal weld images and weld images with various defects such as pores, slag inclusions, and cracks.

3. The weld defect recognition method based on visual Transformer according to claim 1 is characterized in that: Weld image data annotation uses annotation tools to annotate the collected weld images and generate defect locations and category labels. The category labels mark the names of defects such as pores, slag inclusions, and cracks.

4. The weld defect recognition method based on visual Transformer according to claim 1 is characterized in that: Weld image data preprocessing includes denoising, image enhancement and normalization. Diversified training samples are obtained by denoising and / or enhancing the weld images, and then the images corresponding to the training samples are normalized to facilitate model training.

5. The weld defect recognition method based on visual Transformer according to claim 1 is characterized in that: The Petri feature fusion module extracts and fuses feature information from different levels. The Petri feature fusion module comprehensively utilizes low-level detail information and high-level semantic information to achieve accurate defect detection. The Petri feature fusion module first extracts features from multiple encoding layers of the Swin Transformer module. Finally, the contribution of features at different levels is dynamically adjusted in combination with the self-attention mechanism. By calculating the correlation between features, the features that are most helpful for classification are automatically selected for fusion.

6. The weld defect recognition method based on visual Transformer according to claim 5 is characterized in that: The features extracted by the Petri feature fusion module from multiple encoding layers of the Swin Transformer module are represented as high-dimensional feature vectors or feature maps with different dimensions and structures.

7. The weld defect recognition method based on visual Transformer according to claim 6 is characterized in that: The features extracted by the Petri feature fusion module are classified into low-level features, middle-level features and high-level features. The low-level features are responsible for capturing the detailed information of the input image; the middle-level features contain certain semantic information and can capture more structured parts; High-level features are highly abstract and are used to characterize the overall content and contextual information of the image, and can understand defect characteristics over a larger range.

8. The weld defect recognition method based on visual Transformer according to claim 1 is characterized in that: The multi-layer network detection head is used to accurately classify and locate weld defects. The multi-layer network detection head efficiently extracts and classifies features of weld images through a multi-layer network structure. The multi-layer network detection head includes convolutional layers, pooling layers, activation functions, fully connected layers, and Softmax layers.

Images