A Visual Transformer-Based Method for Weld Defect Identification

Abstract

This invention discloses a weld defect recognition method based on visual Transformer, which improves the accuracy of weld defect recognition and solves the problem of insufficient diversity in training data for weld defects. Its key feature is that it pre-configures the following modules: a data preparation module, a model design module, and a training and optimization module. The training and optimization module obtains a standard image dataset and adds newly obtained images to the standard image dataset. Simultaneously, it optimizes model parameters based on test results. When the image of the weld to be detected is input, the model design module segments the input image into reasonable small blocks, forming a corresponding Swing Transformer module. Then, a Petri feature fusion module compares the image with the standard image dataset for the first weld detection. Finally, a multi-layer network detection head compares the image with the standard image dataset for the second weld detection, ultimately determining the weld category and its location in the original image.

Description

Technical Field

[0001] This invention relates to the technical field of weld defect identification, specifically a weld defect identification method based on a visual Transformer. Background Technology

[0002] Welding is a common connection method in engineering structures, and its quality directly affects the strength, toughness, and safety of the structure. Weld defects (such as porosity, slag inclusions, and cracks) are common problems in the welding process. Traditional inspection methods mostly rely on manual inspection, which is not only inefficient but also easily affected by human factors. Therefore, more and more researchers are beginning to explore the application of computer vision and deep learning technologies in weld defect detection.

[0003] Existing technologies use the YOLO algorithm for weld defect detection, but this algorithm performs poorly when detecting small targets in complex backgrounds. Complex textures and lighting changes around the weld can easily interfere with YOLO's feature extraction, increasing the false detection rate. At the same time, its single-detection characteristic makes it difficult to fully utilize the contextual information of the overall weld structure, which in turn makes the accuracy and comprehensiveness of weld judgment unreasonable.

[0004] Furthermore, the accuracy of weld defect detection using the YOLO algorithm is highly dependent on high-quality labeled data; insufficient data or inaccurate labeling will directly affect the detection results.

[0005] Visual Transformer (ViT), as an emerging deep learning architecture, has achieved remarkable results in tasks such as image classification. Unlike the YOLO algorithm, ViT can capture long-range dependencies through self-attention and effectively process large-scale image data, which makes it perform better when dealing with complex image features. However, when using the ViT model for image classification tasks, the computational cost of self-attention is proportional to the square of the pixels, which leads to relatively slow processing and low image processing efficiency. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides a weld defect recognition method based on visual Transformer, which improves the accuracy of weld recognition and can solve the problem of insufficient diversity in training data for weld defects through adaptive data augmentation.

[0007] A weld defect identification method based on visual Transformer is characterized by having the following pre-set modules: a data preparation module, a model design module, and a training and optimization module.

[0008] The data preparation module includes weld image data acquisition, annotation and preprocessing, and finally obtains a standard dataset. The standard image dataset is obtained through the data preparation module.

[0009] The model design module first divides the input image into blocks and stretches them into a sequence, embeds positional codes, and then feeds them into a Swing Transformer to extract features. Second, the Petri feature fusion module fuses low-level concrete detail features with high-level abstract semantic features. The fused features are then processed by a region recommendation network to perform the first weld detection, providing region recommendation boxes and identifying regions of interest where welds may appear. Finally, a multi-layer network detection head performs a second weld detection on the regions of interest, ultimately providing the weld category and its location in the original image.

[0010] The training and optimization module uses a standard image dataset to train the model in the model design module, and introduces cross-validation and hyperparameter tuning to obtain the best training effect. It incorporates a dynamic learning rate adjustment algorithm to adapt to the needs of different training stages and optimizes the model parameters based on the test results.

[0011] The training and optimization modules obtain a standard image dataset and add new images to it. The model parameters are optimized based on test results. When an image of the weld to be detected is input, the model design module segments it into appropriate small blocks, forming a corresponding SwinTransformer module. The Petri feature fusion module then compares the image with the standard image dataset for the first weld detection. Finally, a multi-layer network detection head compares the image with the standard image dataset for the second weld detection, ultimately determining the weld's category and location in the original image.

[0012] Its further features are:

[0013] Weld image data acquisition involves capturing weld images under different welding environment conditions using a high-resolution camera. The images include normal weld images and weld images with various defects such as porosity, slag inclusions, and cracks.

[0014] Weld image data annotation uses annotation tools to annotate the acquired weld images, generating defect location and category labels. The category labels indicate defect names such as porosity, slag inclusion, and cracks.

[0015] Weld seam image data preprocessing includes denoising, image enhancement and normalization. It obtains diverse training samples by denoising and / or enhancing the weld seam images, and then normalizes the images corresponding to the training samples to facilitate model training.

[0016] Denoising uses image processing techniques to reduce image noise and improve image quality;

[0017] Image augmentation generates diverse training samples for the application of data augmentation techniques, thereby enhancing the generalization ability of the model;

[0018] Standardization is a process of normalizing an image so that the pixel values ​​are adjusted to the same range.

[0019] The Petri feature fusion module extracts and fuses feature information from different levels. It comprehensively utilizes low-level detailed information and high-level semantic information to achieve accurate defect detection. The Petri feature fusion module first extracts features from multiple coding layers of the Swin Transformer module. Finally, it dynamically adjusts the contribution of features at different levels by combining a self-attention mechanism. By calculating the correlation between features, it automatically selects the features most helpful for classification for fusion.

[0020] The features extracted from multiple coding layers of the Swin Transformer module by the Petri feature fusion module are represented as high-dimensional feature vectors or feature maps, with different dimensions and structures.

[0021] The features extracted by the Petri feature fusion module are classified into low-level features, mid-level features, and high-level features. Low-level features are responsible for capturing detailed information of the input image; mid-level features contain certain semantic information and can capture more structured parts; high-level features are highly abstract and are used to represent the overall content and contextual information of the image, and can understand defect features over a larger range.

[0022] Multilayer network detection heads are used to accurately classify and locate weld defects. Multilayer network detection heads use a multilayer network structure to efficiently extract and classify features from weld images. The multilayer network detection head includes convolutional layers, pooling layers, activation functions, fully connected layers, and Softmax layers.

[0023] Convolutional layers use multi-layer convolutional neural networks to extract features from the input weld seam image. These convolutional layers can capture detailed information about edges and textures, providing a foundation for subsequent classification.

[0024] Pooling layers reduce the dimensionality of feature maps through downsampling, retaining the most important information and improving computational efficiency;

[0025] Activation functions typically use linear rectified units as activation functions to introduce non-linear characteristics, which helps the network learn more complex patterns.

[0026] After feature extraction, fully connected layers map the extracted features to the classification output. These layers are responsible for integrating the input information to generate the final classification result.

[0027] The Softmax layer is the last layer. It uses the Softmax function to convert the model output into a probability distribution, so that the probabilities of each category can be compared with each other, making it easier to determine the final classification result.

[0028] After adopting the above technical solution, it uses adaptive data augmentation through training and optimization modules to solve the problem of insufficient training data diversity for weld defects; and when setting the model for the detection image, it divides it into blocks to generate a SwinTransformer model. The first weld detection is performed by comparing it with the standard image dataset through the Petri feature fusion module, and the second weld detection is performed by comparing it with the standard image dataset through a multi-layer network detection head. Finally, it gives the category of the weld and its position in the original image. The weld defect recognition method based on visual Transformer reduces the computational complexity and improves the accuracy of weld recognition through dual detection and block detection. Moreover, it can solve the problem of insufficient training data diversity for weld defects through adaptive data augmentation. Attached Figure Description

[0029] Figure 1 This is a flowchart of the model design module of the present invention. Detailed Implementation

[0030] A method for weld defect recognition based on visual Transformer, which is pre-set with a data preparation module, a model design module, and a training and optimization module;

[0031] The training and optimization modules obtain a standard image dataset and add new images to it. The model parameters are optimized based on test results. When an image of the weld to be detected is input, the model design module segments it into appropriate small blocks, forming a corresponding SwinTransformer module. The Petri feature fusion module then compares the image with the standard image dataset for the first weld detection. Finally, a multi-layer network detection head compares the image with the standard image dataset for the second weld detection, ultimately determining the weld's category and location in the original image.

[0032] The data preparation module includes weld image data acquisition, annotation and preprocessing, and finally obtains a standard dataset. The standard image dataset is obtained through the data preparation module.

[0033] Weld seam image data acquisition uses a high-resolution camera to capture weld seam images under different welding environmental conditions (such as temperature, humidity, welding speed, etc.). The images include normal weld seam images and weld seam images with various defects such as porosity, slag inclusions, and cracks, ensuring the diversity and comprehensiveness of the data.

[0034] Weld image data annotation uses annotation tools (such as LabelImg, VGG Image Annotator, etc.) to annotate the acquired weld images, generating defect location and category labels. The category labels are labeled with defect names such as porosity, slag inclusion, and cracks, for subsequent training.

[0035] Weld seam image data preprocessing includes denoising, image enhancement and normalization. It obtains diverse training samples by denoising and / or enhancing the weld seam images, and then normalizes the images corresponding to the training samples to facilitate model training.

[0036] Denoising uses image processing techniques (such as Gaussian filtering, median filtering, etc.) to reduce image noise and improve image quality;

[0037] Image augmentation generates diverse training samples by applying data augmentation techniques (such as rotation, scaling, flipping, and color jitter) to enhance the model's generalization ability;

[0038] Standardization is a process of normalizing an image so that pixel values ​​are adjusted to the same range.

[0039] Model design module (its workflow is shown in...) Figure 1 First, the input image is divided into blocks and stretched into a sequence, with positional encoding embedded. Then, it is fed into the Swing Transformer module to extract features. Second, the Petri feature fusion module fuses low-level detailed features with high-level abstract semantic features. The fused features are then passed through a region recommendation network to perform the first weld detection, providing region recommendation boxes and identifying regions of interest where welds may appear. Finally, a multi-layer network detection head performs a second weld detection on the regions of interest, ultimately providing the weld category and its location in the original image. In practice, the multi-layer network detection head is named the LoCategory detection head.

[0040] The Petri feature fusion module extracts and fuses feature information from different levels. It comprehensively utilizes low-level detailed information and high-level semantic information to achieve accurate defect detection. The Petri feature fusion module first extracts features from multiple coding layers of the Swin Transformer module. Finally, it dynamically adjusts the contribution of features at different levels by combining a self-attention mechanism. By calculating the correlation between features, it automatically selects the features most helpful for classification for fusion.

[0041] The features extracted from multiple coding layers of the Swin Transformer module by the Petri feature fusion module are represented as high-dimensional feature vectors or feature maps, with different dimensions and structures.

[0042] The features extracted by the Petri feature fusion module are classified as follows:

[0043] Low-level features: responsible for capturing detailed information about the input image, such as texture and edges;

[0044] Mid-level features: contain certain semantic information and can capture more structured parts, such as different types of weld structures;

[0045] High-level features: Highly abstract, used to represent the overall content and contextual information of an image, capable of understanding defect features over a wide range.

[0046] The LoCategory inspection head is used to accurately classify and locate weld defects. It employs a multi-layer network structure for efficient feature extraction and classification of weld images. The LoCategory inspection head consists of the following components:

[0047] Convolutional layers use multi-layer convolutional neural networks (CNNs) to extract features from the input weld seam image. These convolutional layers can capture detailed information about edges and textures, providing a foundation for subsequent classification.

[0048] Pooling layers: Reduce the dimensionality of feature maps by downsampling (such as max pooling), retain the most important information and improve computational efficiency;

[0049] Activation function: ReLU (Rectified Linear Unit) is usually used as the activation function to introduce non-linear characteristics, which helps the network learn more complex patterns;

[0050] Fully connected layers: After feature extraction, fully connected layers map the extracted features to the classification output. These layers are responsible for integrating the input information to generate the final classification result.

[0051] Softmax layer: This is the last layer. It uses the Softmax function to convert the model output into a probability distribution, so that the probabilities of each category can be compared with each other, making it easier to determine the final classification result.

[0052] The training and optimization module uses a standard image dataset to train the model in the model design module, and introduces cross-validation and hyperparameter tuning to obtain the best training effect. It incorporates a dynamic learning rate adjustment algorithm to adapt to the needs of different training stages, and optimizes the model parameters based on the test results to achieve better weld seam recognition performance.

[0053] The visual Transformer-based weld defect recognition method addresses the lack of diversity in training data for weld defects by employing adaptive data augmentation through training and optimization modules. When setting up the model for the detection image, it divides the image into blocks to generate a Swin Transformer model. The first weld detection is performed by comparing the model with a standard image dataset using a Petri feature fusion module. Finally, a second weld detection is performed by comparing the second weld with the standard image dataset using a multi-layer network detection head. This process ultimately identifies the weld category and its location in the original image. The visual Transformer-based weld defect recognition method reduces computational complexity and improves weld recognition accuracy through dual detection and block-based detection, while also addressing the lack of diversity in training data for weld defects through adaptive data augmentation.

[0054] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0055] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for weld defect identification based on visual Transformer, characterized in that, It is pre-configured with the following modules: data preparation module, model design module, and training and optimization module; The data preparation module includes weld image data acquisition, annotation and preprocessing, and finally obtains a standard dataset. The standard image dataset is obtained through the data preparation module. The model design module first divides the input image into blocks and stretches them into a sequence, embeds positional codes, and then feeds them into a Swing Transformer to extract features. Second, the Petri feature fusion module fuses low-level concrete detail features with high-level abstract semantic features. The fused features are then processed by a region recommendation network to perform the first weld detection, providing region recommendation boxes and identifying regions of interest where welds may appear. Finally, a multi-layer network detection head performs a second weld detection on the regions of interest, ultimately providing the weld category and its location in the original image. The training and optimization module uses a standard image dataset to train the model in the model design module, and introduces cross-validation and hyperparameter tuning to obtain the best training effect. It incorporates a dynamic learning rate adjustment algorithm to adapt to the needs of different training stages and optimizes the model parameters based on the test results. The training and optimization modules obtain a standard image dataset and add new images to it. The model parameters are optimized based on test results. When an image of the weld to be detected is input, the model design module segments it into appropriate small blocks, forming a corresponding SwinTransformer module. The Petri feature fusion module then compares the image with the standard image dataset for the first weld detection. Finally, a multi-layer network detection head compares the image with the standard image dataset for the second weld detection, ultimately determining the weld's category and location in the original image.

2. The weld defect identification method based on visual Transformer according to claim 1, characterized in that: Weld seam image data acquisition involves capturing weld seam images under different welding environment conditions using a high-resolution camera. The images include normal weld seam images and weld seam images with various defects such as porosity, slag inclusions, and cracks.

3. The weld defect identification method based on visual Transformer according to claim 1, characterized in that: The weld image data annotation uses annotation tools to annotate the acquired weld images, generating defect location and category labels. The category labels indicate defect names such as porosity, slag inclusion, and cracks.

4. The weld defect identification method based on visual Transformer according to claim 1, characterized in that: Weld seam image data preprocessing includes denoising, image enhancement, and normalization. It obtains diverse training samples by denoising and / or enhancing the weld seam images, and then normalizes the images corresponding to the training samples to facilitate model training.

5. The weld defect identification method based on visual Transformer according to claim 1, characterized in that: The Petri feature fusion module extracts and fuses feature information from different levels. It comprehensively utilizes low-level detailed information and high-level semantic information to achieve accurate defect detection. The Petri feature fusion module first extracts features from multiple coding layers of the Swin Transformer module. Finally, it dynamically adjusts the contribution of features at different levels by combining a self-attention mechanism. By calculating the correlation between features, it automatically selects the features most helpful for classification for fusion.

6. The weld defect identification method based on visual Transformer according to claim 5, characterized in that: The features extracted from multiple coding layers of the Swing Transformer module by the Petri feature fusion module are represented as high-dimensional feature vectors or feature maps, each with different dimensions and structures.

7. The weld defect identification method based on visual Transformer according to claim 6, characterized in that: The features extracted by the Petri feature fusion module are classified into low-level features, mid-level features, and high-level features. Low-level features are responsible for capturing detailed information of the input image; mid-level features contain certain semantic information and can capture more structured parts. High-level features are highly abstract and used to represent the overall content and contextual information of an image, enabling the understanding of defect features over a wide range.

8. The weld defect identification method based on visual Transformer according to claim 1, characterized in that: Multilayer network detection heads are used to accurately classify and locate weld defects. They perform efficient feature extraction and classification of weld images through a multilayer network structure, which includes convolutional layers, pooling layers, activation functions, fully connected layers, and softmax layers.

Images