A self-piercing riveting profile image segmentation method based on UNet deep neural network

Abstract

The present invention discloses a self-piercing riveting profile image segmentation method based on UNet deep neural network, which obtains self-piercing riveting profile images as sample images, and annotates each sample image through an annotation tool, generates a label image corresponding to the sample image and a text file containing the annotation result, thereby constructing an initial data set; performing data enhancement on the sample images in the initial data set, and then obtaining an image-enhanced training set; constructing an initial Unet deep neural network based on a deep learning framework; setting the training parameters of the Unet deep neural network; training the Unet segmentation model using the enhanced training set; and applying the trained Unet segmentation model to segment the self-piercing riveting profile. This method can perform image segmentation on self-piercing riveting profile images, overcome the shortcomings of current engineers manually measuring and annotating profile images, and provide support for subsequent automatic measurement of profile geometric parameters.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and specifically to a self-piercing riveting profile image segmentation method based on a UNet deep neural network. Background Art

[0002] As a mechanical joining process, self-piercing riveting is suitable for joining structures of the same or different materials, especially lightweight materials such as aluminum alloy. In the past 20 years, due to the strong demand of the automotive industry, the self-piercing riveting process has achieved rapid development. At present, the self-piercing riveting process has been widely used in the assembly of aluminum alloy body-in-white structures. The quality of self-piercing riveted joints is evaluated by three characteristic indicators measured on the profile of the joint, including rivet head height, interlocking value and minimum remaining base plate thickness (Tmin). Among them, the rivet head height directly affects the appearance of the connected structure and the corrosion resistance of the joint, and also affects the final position of the rivet in the bottom layer, thereby affecting the interlocking value and the minimum remaining base plate thickness; the interlocking value has a significant impact on the mechanical strength and failure behavior of the joint. Self-piercing riveted joints with too small an interlocking value usually have lower shear strength and are more likely to fail due to the rivet foot falling out of the base plate; Tmin is highly correlated with the corrosion resistance and waterproof performance of the joint. A zero or negative Tmin will inevitably lead to moisture intrusion at the joint, which will accelerate the corrosion at the contact between the rivet and the aluminum plate, leading to premature corrosion failure of the self-piercing riveted joint.

[0003] Therefore, accurate measurement of the characteristic indicators of self-piercing riveted joints is very important for evaluating the quality of joints. The traditional method is to take cross-sectional photos with a macro fixed-focus camera fixed on a stand, and manually measure and annotate the cross-sectional parameters based on image processing software (such as Photoshop, SPOT, etc.); this method is inefficient and greatly affected by the subjective factors of the measurer; and the collected cross-sectional photos have the characteristics of high-resolution noise interference. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention proposes a self-piercing riveting profile image segmentation method based on the UNet deep neural network, which can perform image segmentation on self-piercing riveting profile pictures, overcome the shortcomings of current engineers' manual measurement and annotation of profile images, and provide support for subsequent automatic measurement of profile geometric parameters.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A self-piercing riveting profile image segmentation method based on UNet deep neural network includes the following steps:

[0007] Step 1: Obtain a self-piercing riveted cross-section image as a sample image, and annotate each sample image using an annotation tool to generate a label image corresponding to the sample image and a text file containing the annotation results, thereby constructing an initial data set;

[0008] Step 2: Perform data enhancement on the sample images in the initial data set to obtain an image-enhanced training set;

[0009] Step 3: Based on the deep learning framework, build the initial Unet deep neural network;

[0010] Step 4: Set the training parameters of the Unet deep neural network;

[0011] Step 5: Use the enhanced training set to train the Unet segmentation model;

[0012] Step 6: Use the trained Unet segmentation model to segment the self-piercing riveting section.

[0013] Furthermore, each sample image is annotated by using an annotation tool, and the contours of the upper plate, the lower plate and the rivet are respectively annotated in each sample image, and a label image corresponding to the sample image and a text file containing the annotation results are generated.

[0014] Furthermore, the annotation tool in step 1 can be Labelme, CVAT, VIA, PixlAnnotationTool or EISeg.

[0015] Furthermore, the initial UNet neural network includes a downsampling part and an upsampling part; the downsampling part includes 5 layers, each of the downsampling layers includes two convolutional layers, two batch normalization layers corresponding to the convolutional layers, and two activation layers, and a maximum pooling layer is connected between two adjacent layers of the downsampling part;

[0016] The upsampling part also contains 5 layers. Each layer of the upsampling part consists of a bilinear interpolation layer, two convolutional layers, and a batch normalization layer and an activation layer corresponding to the convolutional layer.

[0017] Furthermore, a resizing layer is added before the downsampling part of the network to ensure the feature extraction effect and segmentation accuracy, and a reverse resizing layer is added after the upsampling part to keep the resolution of the label image consistent with the original image without changing the segmentation result.

[0018] Furthermore, the training parameters in step 4 include: training optimizer, learning rate, L2 regularization coefficient, momentum coefficient, batch size, number of iterations and loss function.

[0019] Furthermore, the method for performing data enhancement on the initial training set in step 2 is: randomly performing random scale translation, flipping, rotation, cropping, and scaling operations on the sample images in the initial training set.

[0020] Furthermore, there is a 100% probability of performing a random angle rotation with a maximum angle of 10°, a 50% probability of performing a left-right flip, a 50% probability of performing a cropping with a scale of 0.8, and a 20% probability of performing an up-down flip.

[0021] Beneficial effects of the present invention:

[0022] The present invention adjusts the UNet deep neural network architecture in the field of deep learning and introduces it into the engineering application of automatic image segmentation of self-piercing riveting profiles. Compared with the prior art, the present invention can effectively shorten the measurement time of geometric parameters of self-piercing riveting profiles, avoid interference from subjective factors of the measurer, and improve measurement accuracy. Therefore, applying the present invention to self-piercing riveting quality assessment can significantly improve efficiency and accuracy, reduce experimental costs, and shorten experimental cycles.

[0023] The UNet deep neural network is a commonly used network architecture in the field of computer vision image segmentation. Its downsampling-upsampling U-shaped structure can convert the original image into a mask image that is easy to measure feature indicators. The pixel size of the mask image is consistent with the original image size, and it has a good segmentation effect.

[0024] The UNet neural network structure designed in this application, and the size scaling layer added before the network downsampling part, ensure the feature extraction effect and segmentation accuracy, and add a reverse size scaling layer after the upsampling part, so that the resolution of the label image is consistent with the original image without changing the segmentation result; it can solve the problem of obvious noise interference in the high resolution of self-piercing riveting images. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the process of the present invention;

[0026] Figure 2 In the embodiment, 2a and 2b are respectively the sample images of the initial training set of the self-piercing riveting section and the corresponding label images;

[0027] Figure 3 In the embodiment, 3a and 3b are respectively the training set sample images and the corresponding label images after the self-piercing riveting profile data is enhanced;

[0028] Figure 4 A schematic diagram of the UNet neural network architecture in the embodiment;

[0029] Figure 5 In the embodiment, 5a and 5b are the test image and the corresponding image segmented by the improved UNet, respectively. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0031] like Figure 1 A self-piercing riveting profile image segmentation method based on a UNet deep neural network is shown, comprising the following steps:

[0032] Step 1: Construct the initial training data set. Specifically:

[0033] Step 1.1, collect a certain number of self-piercing riveting cross-section images as sample images, and construct an initial data set from the sample images. In this example, a total of 44 sample images are collected, of which 40 are used as training sets and 4 are used as test sets; each sample image is annotated by an annotation tool, and the contours of the upper plate, lower plate and rivet are annotated in each sample image, and a label image corresponding to the sample image and a text file containing the annotation results are generated; in this embodiment, the annotation tool can be selected from Labelme, CVAT, VIA, PixlAnnotationTool, EISeg, etc.

[0034] Step 1.2: Repeat the processing of the sample images in step 1.1 above to obtain the training samples corresponding to all the sample images, and then obtain the initial training set. The training sample images in the initial training set and their corresponding label images are as follows: Figure 2 As shown in 2a and 2b.

[0035] Step 2, perform data enhancement on the sample images in the initial training set. The specific process is: based on the image preprocessing tool, the sample images in the initial training set in step 1 are randomly translated, flipped, rotated, cropped, scaled, and other operations with a certain probability. In this embodiment, there is a 100% probability of performing a random angle rotation with a maximum angle of 10°, a 50% probability of flipping left and right, a 50% probability of cropping with a scale of 0.8, and a 20% probability of flipping up and down, thereby generating sample images that are 50 times the size of the initial training set, and using the same transformation to generate the corresponding label image for each sample image, thereby obtaining the image-enhanced training set. The enhanced training set sample images and the corresponding label images are as shown in the figure. Figure 3 As shown in 2a and 2b.

[0036] Step 3: Construct an initial UNet neural network based on the deep learning framework. In this embodiment, the UNet neural network adjusts the traditional UNet deep neural network to address the obvious high-resolution noise interference of self-piercing riveting images. The downsampling part contains 5 layers, each of which contains two convolutional layers and corresponding batch normalization layers and activation layers, and each layer is connected by a maximum pooling layer; the upsampling part also contains 5 layers, each of which consists of a bilinear interpolation layer and two convolutional layers and corresponding batch normalization layers and activation layers. The network architecture parameters are as follows: Figure 4 shown.

[0037] In addition, due to the high resolution of the self-piercing riveting cross-section image, a size scaling layer is added before the downsampling part of the network to ensure the feature extraction effect and segmentation accuracy. A reverse size scaling layer is added after the upsampling part to keep the resolution of the label image consistent with the original image without changing the segmentation result.

[0038] Step 4: Set training parameters. The training parameters involved in this embodiment include: training optimizer, learning rate, L2 regularization coefficient, momentum coefficient, batch size, number of iterations, loss function, etc.

[0039] Step 5: Introduce the image enhancement training in step 2 into the UNet deep neural network model for training. After the training is completed, a UNet segmentation model suitable for self-piercing riveting profile image segmentation is obtained.

[0040] Step 6: Input the unlabeled self-piercing riveting cross-section image into the UNet deep neural network model in step 5. The deep neural network model will output a label image with the corresponding colors of the upper plate, lower plate, and rivet (such as Figure 5 The output label image can be used for subsequent geometric parameter measurement. The label image output by the model is compared with the accuracy of the manually annotated image in step 1. The average segmentation accuracy exceeds 98%, which meets the subsequent self-piercing riveting geometric parameter measurement requirements.

[0041] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A self-piercing riveting profile image segmentation method based on UNet deep neural network, characterized in that: The steps include: Step 1: Obtain a self-piercing riveted cross-section image as a sample image, and annotate each sample image using an annotation tool to generate a label image corresponding to the sample image and a text file containing the annotation results, thereby constructing an initial data set; annotate each sample image using an annotation tool, annotate the contours of the upper plate, lower plate and rivet in each sample image, and generate a label image corresponding to the sample image and a text file containing the annotation results; the annotation tool in step 1 is selected from Labelme, CVAT, VIA, PixlAnnotationTool or EISeg; Step 2: Perform data enhancement on the sample images in the initial data set to obtain an image-enhanced training set; Step 3: Based on the deep learning framework, construct an initial Unet deep neural network; the initial UNet neural network includes a downsampling part and an upsampling part; the downsampling part includes 5 layers, each of which includes two convolutional layers, two batch normalization layers corresponding to the convolutional layers, and two activation layers, and a maximum pooling layer is connected between two adjacent layers of the downsampling part; The upsampling part also contains 5 layers. Each layer of the upsampling part consists of a bilinear interpolation layer, two convolutional layers, and a batch normalization layer and an activation layer corresponding to the convolutional layer. A resizing layer is added before the downsampling part of the network to ensure the feature extraction effect and segmentation accuracy. A reverse resizing layer is added after the upsampling part to keep the resolution of the label image consistent with the original image without changing the segmentation result. Step 4: Set the training parameters of the Unet deep neural network; The training parameters in step 4 include: training optimizer, learning rate, L2 regularization coefficient, momentum coefficient, batch size, number of iterations and loss function; Step 5: Use the enhanced training set to train the Unet segmentation model; Step 6: Use the trained Unet segmentation model to segment the self-piercing riveting section.

2. According to claim 1, a self-piercing riveting profile image segmentation method based on UNet deep neural network is characterized in that: The method for performing data enhancement on the initial training set in step 2 is: randomly performing random translation, flipping, rotation, cropping, and scaling operations on the sample images in the initial training set.

3. The self-piercing riveting cross-section image segmentation method based on UNet deep neural network according to claim 2 is characterized in that: There is a 100% probability of performing a random angle rotation with a maximum angle of 10°, a 50% probability of performing a left-right flip, a 50% probability of performing a cropping with a scale of 0.8, and a 20% probability of performing an up-down flip.

Images