A method for rapid annotation of defect images in industrial radiographic inspection using Deeplabv3plus network

By combining the Deeplabv3plus network with traditional image processing methods, we have achieved rapid and efficient annotation of defect images for industrial X-ray inspection. This solves the problems of high cost and low efficiency of manual annotation, improves the quality and speed of annotation data, and promotes the application of deep learning in the industrial field.

CN118334429BActive Publication Date: 2026-11-10DALIAN UNIV OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410451377.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-16
Publication Date
2026-11-10
Estimated Expiration
2044-04-16

AI Technical Summary

Technical Problem

Existing deep learning semantic segmentation algorithms rely on a large amount of manually labeled training data, especially precise pixel labeling, which leads to high labeling costs and is prone to omissions or mislabeling, affecting the model training effect.

Method used

We use the Deeplabv3plus network for rapid annotation of defect images from industrial radiographic inspection. Through initial training and data augmentation with a small amount of data, combined with traditional image processing methods, we generate high-quality annotated data, reducing the workload of manual annotation.

Benefits of technology

It improves the speed and quality of semantic segmentation annotation for defective images, reduces labor costs, decreases the occurrence of missed or incorrect annotations, and promotes the application of deep learning in the industrial field.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118334429B_ABST
    Figure CN118334429B_ABST
Patent Text Reader

Abstract

This invention belongs to the field of deep learning technology and is a method for rapid annotation of defect images in industrial X-ray inspection using the Deeplabv3plus network. This invention selects the Deeplabv3plus network as the deep learning semantic segmentation model. First, a portion of the dataset is labeled and cropped using Photoshop software. Then, it is divided into training and validation sets, and trained using the Deeplabv3plus network. After achieving the required mIoU and mAcc on the validation set, the trained model is used to annotate the remaining data. Finally, the annotated data is fine-tuned. This annotation method significantly improves the speed of labeling semantic segmentation data for defect images, reduces annotation costs, and can, to some extent, reduce the occurrence of missed or incorrect annotations during manual annotation, thereby improving the quality of the labeled data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of deep learning technology and is a method for rapid annotation of defect images in industrial X-ray inspection by combining the Deeplabv3plus network. Background Technology

[0002] In recent years, with the development of deep learning, deep learning neural networks have achieved remarkable results in computer vision-related tasks. Deep learning semantic segmentation is one of the key tasks in the field of computer vision today. From a macro perspective, semantic segmentation is a high-level task that lays the foundation for a complete understanding of a scene. Some applications include medical imaging and remote sensing maps, where its accuracy and efficiency far surpass other methods.

[0003] However, most existing deep learning semantic segmentation algorithms are supervised learning algorithms, relying on large amounts of manually labeled training data. Semantic segmentation annotation costs are significantly higher than previous object detection annotation costs, requiring precise pixel-level labeling, which is extremely labor-intensive. Therefore, effectively reducing data annotation costs during the deep learning semantic segmentation data annotation stage will benefit the practical application of semantic segmentation.

[0004] In semantic segmentation training, the quantity and quality of labeled data directly affect the detection performance of the trained model. During manual labeling, due to the difficulty of labeling semantic segmentation pixels, the small number of individual defects, and the visual fatigue that can easily occur with prolonged manual labeling, omissions or errors are highly likely. This directly impacts the model's training performance. Therefore, the labeling of semantic segmentation data plays a crucial role in deep learning training.

[0005] To address the aforementioned problems, this invention proposes a method for rapid annotation of defect images in industrial X-ray inspection using the Deeplabv3plus network. By employing the Deeplabv3plus network, a limited amount of data is used to initially train the deep learning model. This model is then used to predict and annotate the remaining data. Finally, the annotated images are manually fine-tuned, thereby reducing annotation costs. Summary of the Invention

[0006] The main objective of this invention is to provide a method for rapid marking of defect images in industrial radiographic testing, which can improve the efficiency of data marking and reduce labor costs to a certain extent.

[0007] The technical solution of this invention:

[0008] A method for rapid annotation of defect images from industrial radiographic inspection using the Deeplabv3plus network is described below:

[0009] (1) Collect DR image data of defective industrial products, and divide the original DR image data into two parts according to the proportion, named Dataset A and Dataset B respectively.

[0010] Preferably, the image data volume of dataset A is smaller than that of dataset B;

[0011] (2) Use Photoshop to smudge dataset A as a defect marker and save the image as a PNG file;

[0012] (3) Process the image obtained in step (2) so that only the color of the defect mark is in the image, and use it as the label data for deep learning semantic segmentation;

[0013] (4) The Deeplabv3plus network was used as the deep learning semantic segmentation model;

[0014] (5) Crop the original DR image in dataset A in step (1) into a 512*512 image to obtain the original DR cropped image, and crop the corresponding png format label file in step (3) in the same way to obtain the label cropped image.

[0015] Furthermore, during the image cropping process, if the original image size is an integer multiple of the cropped image size, it can be directly segmented from left to right and from top to bottom. If the original image is not an integer multiple of the cropped image size, it is necessary to first segment from left to right and from top to bottom to crop it into a 512*512 image. Then, the remaining part of the cropped image is judged. If the remaining part is less than 1 / 8 of the 512*512 size, it is deleted. Otherwise, it is pieced together from the left or bottom image to make all the cropped images 512*512 in order to make full use of the data.

[0016] (6) Filter the cropped images of the labels obtained in step (5) and delete the images in which all pixel values ​​are 0, i.e. the data without defect labels. At the same time, the original DR cropped images corresponding to the deleted data without defect labels are kept only for the images with defect labels.

[0017] (7) Divide the images in dataset A from step (6) into two parts according to the proportion. The training set is used to train the deep learning semantic segmentation model, and the validation set is used to detect the training effect of the deep learning semantic segmentation model.

[0018] (8) The training set after the dataset was divided in step (7) was augmented with two-dimensional Gaussian filtering, contrast enhancement, horizontal and vertical flipping, rotation, partial cropping and padding to a size of 512*512. The data augmentation was completed by randomly performing the data augmentation operation. The augmented data was 2 to 4 times larger than the data before augmentation.

[0019] The two-dimensional Gaussian function is given by formula (1):

[0020]

[0021] In the formula, (x,y) represents the pixel coordinates, and σ is the standard deviation of the Gaussian function;

[0022] (9) Train the deep learning semantic segmentation model. Validate the trained deep learning semantic segmentation model using the validation set obtained in step (8). Terminate training when the mIoU on the validation set is consistently above 80% and mAcc is consistently above 85%, and export the model. If the requirements are not met, optimize the parameters and retrain the model. The average intersection-union ratio (mIoU) is given by formula (2):

[0023]

[0024] Where k represents the number of categories to be segmented, i = 0 to k; TP i FP represents the number of positive samples correctly classified into the i-th category. i FN represents the number of positive samples misclassified in the i-th class. i This represents the number of negative samples that are misclassified in the i-th category.

[0025] The average accuracy mAcc is given by formula (3):

[0026]

[0027] Where k represents the number of categories to be segmented, i = 0 to k; T P i N represents the number of pixels correctly segmented in the i-th class. p i This represents the total number of pixels in the i-th class;

[0028] (10) Crop the original DR images in dataset B into 512*512 images; the cropping principle is the same as in step (5);

[0029] (11) The deep learning semantic segmentation model that meets the requirements of step (9) is used to detect and label the DR images in the B dataset obtained in step (10), and the labeled output images are saved as png format images as the original label files of the B dataset;

[0030] (12) Use Photoshop software to fine-tune the cropped B dataset image obtained in step (10) and the original label file obtained in step (11), remove some mislabeled defects, add unlabeled defects, and complete the labeling of the entire dataset after verification.

[0031] This invention can generate defect image label files that are similar to those produced by manual labeling. By combining the Deeplabv3plus network with traditional image processing methods, it greatly improves the labeling speed of semantic segmentation label data for defect images, reduces the workload of labeling workers, lowers labeling costs, and facilitates the promotion of deep learning in the industrial field. This method can also reduce the occurrence of omissions and errors in manual labeling to a certain extent, and improve the labeling quality of the labeled data. Attached Figure Description

[0032] Figure 1 A flowchart illustrating the overall process of the rapid annotation method;

[0033] Figure 2 Original DR image of the casting;

[0034] Figure 3 Image (a) shows a portion of the DR image containing inclusion defects before labeling. Figure 3 (b) is a partial image of the DR containing inclusion defects after annotation;

[0035] Figure 4 Image (a) shows a portion of the DR image containing shrinkage defects before labeling. Figure 4 (b) is a partial image of the DR containing shrinkage defects after annotation;

[0036] Figure 5 (a) is the image before data augmentation. Figure 5 (b) is the image after Gaussian filtering;

[0037] Figure 6 (a) is the image before data augmentation. Figure 6 (b) is the image after random rotation;

[0038] Figure 7 (a) is the image before data augmentation. Figure 7 (b) is the image after being flipped. Specific implementation methods

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the following will use castings, a common type of industrial product, as an example, and describe the technical solutions of this invention in detail with reference to the accompanying drawings. The process is as follows: Figure 1 As shown.

[0040] (1) Collect 720 DR image data of defective castings, each 3072*3072 pixels in size, such as... Figure 2 As shown, the original DR image data is divided into two parts in a 1:5 ratio, named Dataset A and Dataset B respectively, with Dataset A containing 120 images and Dataset B containing 600 images.

[0041] (2) Then, the images in dataset A were labeled using Photoshop software. Since there are two types of defects, they were named inclusion and defect respectively. They were labeled with green and red respectively, and the labels were saved as png files.

[0042] (3) Process the marked image obtained in step (2) and keep only the red and green markings. Then set the pixel value marked as green to 1 and the pixel value marked as red to 2, corresponding to inclusion and defect respectively. Then convert the image into a single-channel image as training label data.

[0043] (4) Crop the original DR images in dataset A obtained in step (1) to a size of 512*512, such as Figure 3 (a) Figure 4 As shown in (a), the label images obtained in step (3) are cropped in the same way, and then the defect-free casting images and their corresponding label images are deleted. After filtering, 2000 images in dataset A meet the requirements. The labeled images are as follows: Figure 3 (b) Figure 4 As shown in (b) in the image;

[0044] (5) Data augmentation methods. Data augmentation mainly utilizes traditional image processing techniques such as Gaussian filtering, contrast enhancement, flipping, and rotation. During training, Gaussian filtering, contrast enhancement, flipping, and rotation are randomly selected with a certain probability in each epoch to improve the model's robustness. After Gaussian filtering, the image appears as follows: Figure 5 As shown in (b) above, the image after random rotation is as follows: Figure 6 As shown in (b) above, the image after being flipped is as follows: Figure 7 As shown in (b) above, after data augmentation, dataset A contains 6000 images;

[0045] (6) The dataset A obtained in step (5) is divided into two parts in a 5:1 ratio. The 5000 images and corresponding labeled images in dataset A are used as the training set for training the model, and the remaining 1000 images and corresponding labeled images are used as the validation set for dynamically monitoring the training effect of the model.

[0046] (7) Train the model and then test the trained model. Validate the trained model file using the validation set in dataset A. Terminate training when the mIoU is consistently above 80% and mAcc is consistently above 85% on the validation set, and export the model. If the requirements are not met, optimize the parameters, such as adjusting the learning rate, changing the optimizer, or changing the learning rate descent strategy. Then retrain the model until the requirements are met.

[0047] (8) Crop the original DR image in the B dataset obtained in step (1) to a size of 512*512;

[0048] (9) Use the model that meets the requirements in step (7) to detect and annotate the DR images in the B dataset obtained in step (8), and save the annotated output images as png format;

[0049] (10) The marked images and corresponding casting images obtained in step (8) are fine-tuned using Photoshop software. By comparing the marked images with the casting images, some missing or incorrect markings and cases where the marked area is too large or too small are adjusted. After verification, the entire dataset is labeled.

Claims

1. A method for rapid annotation of defect images in industrial radiographic inspection using a Deeplabv3plus network, characterized in that, The steps are as follows: (1) Collect DR image data of defective industrial products, and divide the original DR image data into two parts according to the proportion, named Dataset A and Dataset B respectively. (2) Use Photoshop to smudge dataset A as a defect marker and save the image as a PNG file; (3) Process the image obtained in step (2) so that only the color of the defect mark is in the image, and use it as the label data for deep learning semantic segmentation; (4) The Deeplabv3plus network was used as the deep learning semantic segmentation model; (5) Crop the original DR image in dataset A in step (1) into a 512*512 image to obtain the original DR cropped image, and crop the corresponding png format label file in step (3) in the same way to obtain the label cropped image. (6) Filter the cropped images of the labels obtained in step (5) and delete the images in which all pixel values ​​are 0, i.e. the data without defect labels. At the same time, the original DR cropped images corresponding to the deleted data without defect labels are kept only for the images with defect labels. (7) Divide the images in dataset A from step (6) into two parts according to the proportion. The training set is used to train the deep learning semantic segmentation model, and the validation set is used to detect the training effect of the deep learning semantic segmentation model. (8) The training set after the dataset is divided in step (7) is augmented with data. The data augmentation method is two-dimensional Gaussian filtering, contrast enhancement, left and right flipping, up and down flipping, rotation, partial cropping and filling to a size of 512*512. The data augmentation of the image is completed by randomly performing data augmentation operations. The augmented data is 2 to 4 times the data before augmentation. The two-dimensional Gaussian function is given by formula (1): In the formula, (x,y) represents the pixel coordinates, and σ is the standard deviation of the Gaussian function; (9) Train the deep learning semantic segmentation model. Validate the trained deep learning semantic segmentation model using the validation set obtained in step (8). Terminate training when the mIoU on the validation set is consistently above 80% and mAcc is consistently above 85%, and export the model. If the requirements are not met, optimize the parameters and retrain the model. The average intersection-union ratio (mIoU) is given by formula (2): Where k represents the number of categories to be segmented, i = 0 to k; TP i FP represents the number of positive samples correctly classified into the i-th category. i FN represents the number of positive samples misclassified in the i-th class. i This represents the number of negative samples that are misclassified in the i-th category. The average accuracy mAcc is given by formula (3): Where k represents the number of categories to be segmented, i = 0 to k; T P i N represents the number of pixels correctly segmented in the i-th class. p i This represents the total number of pixels in the i-th class; (10) Crop the original DR images in dataset B into 512*512 images; the cropping principle is the same as in step (5); (11) The deep learning semantic segmentation model that meets the requirements of step (9) is used to detect and label the DR images in the B dataset obtained in step (10), and the labeled output images are saved as png format images as the original label files of the B dataset; (12) Use Photoshop software to fine-tune the cropped B dataset image obtained in step (10) and the original label file obtained in step (11), remove some mislabeled defects, add unlabeled defects, and complete the labeling of the entire dataset after verification.

2. The method for rapid annotation of defect images in industrial radiographic inspection using a Deeplabv3plus network according to claim 1, characterized in that, In step (5), during the image cropping process, if the original image size is an integer multiple of the cropped image size, it can be directly divided from left to right and from top to bottom in sequence; if the original image is not an integer multiple of the cropped image size, it is necessary to first divide it from left to right and from top to bottom to crop it into a 512*512 image. Then, the remaining part of the cropped image is judged. If the remaining part is less than 1 / 8 of the 512*512 size, it is deleted. Otherwise, it is pieced together to the left or bottom of the image so that the cropped images are all 512*512 in size, so as to make full use of the data.

Citation Information

Patent Citations

  • Industrial ray image evaluation method and device based on deep learning

    CN109859177A

  • Weld joint digital image defect marking method based on deep learning

    CN116805418A