Online defect detection method for vehicle door stamping part

Abstract

The invention provides an online defect detection method for a vehicle door stamping part. The online defect detection method is characterized by comprising a data acquisition module, a data augmentation module and a defect detection module, a generation model constructed by adopting a pix2pix network is arranged in the data augmentation module, and a detection model constructed by adopting a YOLOv5 network is arranged in the defect detection module; a MambaVision network is adopted as a convolution kernel in the generation model pix2pix network; the method comprises the following steps: a data acquisition module acquires a plurality of real defect images on a production line as an original training data set; the data augmentation module generates a plurality of simulation images as an augmented training data set by utilizing a generative model according to the received original training data set; the defect detection module adopts an original training data set and an augmented training data set to train a detection model to obtain an available detection model; and the defect detection module performs online real-time detection on defects of the vehicle door stamping part by using an available detection model. By adopting the method, the accuracy and efficiency of defect detection of the vehicle door stamping part can be improved.

Description

Technical Field

[0001] The invention relates to the technical field of surface defect detection of vehicle parts, in particular to an online defect detection method for a vehicle door stamping part. Background Art

[0002] In the process of automobile production, door stamping parts are part of the automobile structure. The related high-quality products can improve the overall quality of automobile assembly products and ensure the safety of automobile products. Therefore, defect detection of door stamping parts has great practical application value and significance for both automobile manufacturers and consumer user groups. In the field of industrial defect detection, traditional detection methods cannot meet the "fast" and "high accuracy" requirements of the environment on the door stamping production line.

[0003] At present, although the use of machine vision inspection methods based on deep learning can improve the defect detection efficiency of online stamping parts to a certain extent. However, the above-mentioned machine vision inspection method relies on a large amount of online stamping defect image data to train the neural network model. For any neural network model, whether it can be better used and solve practical engineering problems depends not only on whether its network architecture is suitable for the application scenario and whether the training strategy is reasonable, but also on the quality of the training data set. A training data set with uniform defect category distribution, strong feature representativeness, and unified background and actual application scenarios can make the model fit better, not only meet the needs of actual detection, but also improve the detection accuracy. However, in the production of door stamping parts, it is quite difficult to obtain defect image data of various categories to achieve defect category balance and strong feature representativeness. Even on the same production line, the frequency and probability of each category of defects are not consistent, which may lead to small differences between different types of defects and large differences within the same type of defects.

[0004] In the prior art, modular-based methods can often only target simple industrial defect image backgrounds. This task difference ignores the global features of stamped workpieces in more complex production environments, making the generated images lack guidance for the detection model. Therefore, for complex backgrounds in actual online production environments, it is difficult to achieve balanced detection accuracy and improve detection effects for stamped parts defect detection. In addition, the various defect types of door stamping parts have a strong correlation (i.e., dependency) with different parts of the parts, and the existing defect image generation algorithms often ignore the ability to capture global contextual information and long-distance spatial dependencies, which easily leads to the detection model identifying non-defect features of some parts as defects that generally do not appear in these parts, resulting in misjudgment of defect recognition and reducing the accuracy of defect recognition. Summary of the invention

[0005] In view of the problems of the background technology, the present invention provides a method for detecting defects of vehicle door stamping parts, so as to solve the problem of how to improve the precision and accuracy of defect detection of vehicle door stamping parts under the complex background of the vehicle door production line.

[0006] To achieve the purpose of the present invention, the present invention provides a method for online defect detection of door stamping parts, the innovation of which is that: it includes a data acquisition module, a data augmentation module and a defect detection module; the data augmentation module is provided with a generation model constructed by using a pix2pix network, and the defect detection module is provided with a detection model constructed by using a YOLOv5 network; the convolution kernel in the pix2pix network of the generation model adopts a MambaVision network;

[0007] The method comprises:

[0008] 1) The data acquisition module collects multiple real defect images of door stamping parts on the production line as the original training data set, and then the data acquisition module transmits the original training data set to the data augmentation module;

[0009] 2) The data augmentation module generates multiple simulated images of door stamping defects as augmented training data sets based on the received original training data sets using the generative model, and then the data augmentation module transmits the original training data sets and the augmented training data sets to the defect detection module;

[0010] 3) The defect detection module uses the original training data set and the augmented training data set to train the detection model to obtain a usable detection model;

[0011] The defect detection module uses available detection models to perform online real-time detection of defects in door stampings.

[0012] Furthermore, in step 3), the defect detection module trains the detection model in the following manner:

[0013] 1) Use the original training data set to train the detection model until the model converges to obtain the first round of updated model;

[0014] 2) Using a training data set composed of a mixture of the original training data set and the augmented training data set to train the first round of updated models until the model converges, and obtain the second round of updated models;

[0015] 3) Use the original training data set to train the second round of updated model to guide model convergence and obtain a usable detection model.

[0016] Furthermore, the deconvolutional layer feature map dimensions of the MambaVision network match those of the pix2pix network.

[0017] The principle of the present invention is as follows:

[0018] In order to improve the accuracy of the model in detecting defects of door stamping parts on the production line, the present invention has made innovations and improvements in the following aspects:

[0019] First, in terms of training data sets, in order to improve the detection accuracy of subsequent detection models, it is necessary to augment and expand the original training data sets collected online to ensure sufficient training data volume to ensure the training effect of the detection model. The present invention adopts a pix2pix network architecture based on conditional generative adversarial network (cGAN) to realize data augmentation. As an image-to-image translation model, pix2pix can be used for a variety of tasks, such as generating photos from sketches, generating photos from label images, generating color images from black and white images, etc. Its conditional generator usually uses U-Net pure convolution architecture, which helps to retain spatial details and structural information. Pix2pix itself has style transfer capabilities, which can make the background of the generated image close to the background of the real image. However, the pix2pix network lacks the extraction of long-distance dependencies in global features, which limits the restoration of complex image backgrounds and defect features of inconsistent sizes during image generation. The training data has a large degree of distortion, which affects the effect of subsequent training of the detection model and further affects the accuracy of the detection model. In the present invention, a MambaVision network module (MV block) with a strong ability to capture long-distance image dependencies is creatively integrated into the feature extraction part of the pix2pix generator. Specifically, a simple convolution kernel in the feature extraction process in the pix2pix generator is transformed into an efficient feature extractor composed of an MV block and a fully connected layer, and the downsampling branch of the MV block is reconstructed to match the dimension of the deconvolution layer feature map in the feature reconstruction process, thereby improving the network's ability to capture global context information and long-distance spatial dependencies. Finally, the multi-channel feature map is processed by convolution to output a new image that matches the dimension of the training image, so that the new generated image has features independent of the training data, and the defect features in the generated image can be close to the defect features in the real image. The generated image finally obtained can fit the combination of the real background and the real defect to the greatest extent, and the augmentation operation is completed on the data set, so that the subsequent detection model has better fitting effect and detection performance.

[0020] In terms of the training of the detection model, the real-time image object detection network model YOLOv5 is used, which can quickly classify and locate multiple objects in the image. It is one of the fastest computer vision tools currently available and is suitable for fast-response scenarios, which means that it can improve the efficiency of online door stamping defect detection. On the other hand, as an optimization, in the training phase of YOLOv5, the model is trained in three rounds using the original training data set and the augmented training data set. In the first round, only the original training data set is used for model training, which enables the model to quickly find the fitting reverse direction. In the second round, a mixed training data set of the original training data set and the augmented training data set is used for model training, which can improve the robustness and generalization ability of the model through training with more data. In the third round, the original training data set is used to train the model, which allows the model to recalibrate the fitting direction and improve the accuracy of the model prediction.

[0021] It can be seen that the present invention has the following beneficial effects: adopting the method of the present invention to perform on-line defect detection of vehicle door stamping parts can greatly improve detection accuracy, detection rate and detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings of the present invention are as follows:

[0023] Attached Figure 1 This is a schematic diagram of the generator structure of the original pix2pix network;

[0024] Attached Figure 2 Schematic diagram of the pix2pix generator structure after integrating the Mamba Vision network. DETAILED DESCRIPTION

[0025] The present invention will be further described below in conjunction with the embodiments.

[0026] The method for online defect detection of door stamping parts described in the present invention involves modules including a data acquisition module, a data augmentation module and a defect detection module; the data augmentation module is provided with a generation model constructed by using a pix2pix network, and the defect detection module is provided with a detection model constructed by using a YOLOv5 network; the convolution kernel in the generation model pix2pix network adopts a MambaVision network, and the dimension of the deconvolution layer feature map of the MambaVision network matches the pix2pix network.

[0027] The method comprises:

[0028] 1) The data acquisition module collects multiple real defect images of door stamping parts on the production line as the original training data set, and then the data acquisition module transmits the original training data set to the data augmentation module;

[0029] 2) The data augmentation module generates multiple simulated images of door stamping defects as augmented training data sets based on the received original training data sets using the generative model, and then the data augmentation module transmits the original training data sets and the augmented training data sets to the defect detection module;

[0030] 3) The defect detection module uses the original training data set and the augmented training data set to train the detection model to obtain a usable detection model;

[0031] The defect detection module uses available detection models to perform online real-time detection of defects in door stampings.

[0032] In order to obtain the best model performance, in step 3), the defect detection module trains the detection model in the following manner:

[0033] 1) Use the original training data set to train the detection model until the model converges to obtain the first round of updated model;

[0034] 2) Using a training data set composed of a mixture of the original training data set and the augmented training data set to train the first round of updated models until the model converges, and obtain the second round of updated models;

[0035] 3) Use the original training data set to train the second round of updated model to guide model convergence and obtain a usable detection model.

[0036] In the present invention, the data augmentation module of the present invention may be used to augment the original training data set using the prior art method. The method of this embodiment is specifically described as follows:

[0037] The image data in the original training data set are input into the parameter optimization model in pairs and batches; each batch contains at least one pair of training data; for each pair of training data, follow the following steps:

[0038] a) Input a pair of training data into the generative model, denoted as real_picA and real_picB. In the generator training, real_picA is processed to get fake_picB, and the discriminator distinguishes between real_picA and fake_picB, and the discriminant output result is pred_fake; in the discriminator training, real_picA and real_picB are distinguished, and real_picA and fake_picB are judged as true or false, and the discriminant output results are pred_real and pred_fake respectively;

[0039] b) According to the output result pred_fake in the generator training, determine the adversarial loss L between the generated base real_picA and the generated image fake_picBGadv =MSE(pred_fake,1), and the pixel loss L between the generated target image real_picB and the generated image fake_picB pix = L1(fake_picB, real_picB); According to the output results pred_real and pred_fake in the discriminator training, determine the discrimination loss L between the real image real_picA and the real image real_picB real =MSE(pred_real,1), and the discriminative loss L between the generated base real_picA and the generated image fake_picB fake =MSE(pred_fake,0), and the discriminator adversarial loss L is obtained Dadv =0.5(L real +L fake );MSE(·) is the mean square error loss function, L1(·) is the L1 norm loss function;

[0040] c) The generative model contains multiple MV blocks in the feature extraction stage (Encode stage), and the output feature size of a single MV block is B×H×W×C down ; B is the amount of training data contained in a single batch, H is the feature map height, W is the feature map width, C down is the number of output channels in the feature extraction stage;

[0041] The generative model includes multiple upsampling blocks in the feature reconstruction stage (Decode stage), and the output feature size of a single upsampling block is B×H×W×C up , C up is the number of output channels in the feature reconstruction stage;

[0042] Each downsampling block in the feature extraction phase matches an upsampling block in the feature reconstruction phase, and during the generator inference process, the loss characteristics of the downsampling process are reduced;

[0043] According to the above method, we can get the generator-related loss L under a single training data. Gadv , L pix And the discriminator related loss L Dadv , according to L Gadv , L pix and L Dadv Get the average loss of the generator under a set of training data The average loss associated with the discriminator According to and Get the total average loss of the generator and discriminator under this set of training data in Each round of training corresponds to a different and Then calculate the fitting curves of the generator and discriminator according to the number of training rounds [L Gi ] and [L Di ](i=1,2,3…);λ is the measure The weight constant coefficient of ;

[0044] In each round of training, according to L G , L D , the back-propagation algorithm is used to perform parameter optimization operations on the generator and discriminator in the generative model. When the number of training rounds reaches the set number, the fitting loss change curve is analyzed. If the fitting curve does not meet the requirements, the parameter settings are adjusted and the training is performed again to fit the generative model until the loss change curve converges, that is, the converged generative model is obtained, and the augmented training data set corresponding to the original training data set is obtained.

[0045] Among them, in order to adapt to the pix2pix network and match the dimension of the deconvolution layer feature map in the feature reconstruction process, the MV block can be adaptively modified in the following way:

[0046] As attached Figure 1 The generator structure diagram of the original pix2pix network shown in Figure 1 is a convolution downsampling block, and DeConv_up is a deconvolution upsampling block. Figure 2 Schematic diagram of the pix2pix generator structure after integrating the Mamba Vision network. Some convolution kernels are removed and the adaptively modified MV_Block from MambaVision is embedded. The first MV_Block is downsampled four times (this step is completed by the feature map embedding layer), so compared with the original pix2pix network structure, there is a lack of a downsampling block. In fact, the MV block contains the mamba mixer and the self-attention block. Compared with the single-layer convolution kernel, it has a deeper network layer and more parameters, and the output dimension will not be downsampled. Therefore, in order to integrate the MV block into the pix2pix generator network, adaptive adjustments are required:

[0047] 1) The dimension of the fully connected layer that performs forward linear mapping on the output of the self-attention block is modified to correspond to the input and output dimensions of each convolutional downsampling block in the pix2pix generator. For example, if the output dimension of the last Conv_down is 1 / 128 of the length and width of the input image, then the output dimension of the corresponding MV_Block also needs to be set to 1 / 128 of the length and width of the input image.

[0048] 2) The number distribution of self-attention heads, the size of the patch window, and the depth of the linear mapping layer in the four cascaded MV_Blocks are modified to balance the higher computational complexity brought by the introduction of the spatial state model and the self-attention mechanism.

[0049] The details are as follows:

[0050] The output feature corresponding to the MV block is denoted as f out , the input feature is recorded as f in ;

[0051] When the input is continuous, the input features are mapped to a one-dimensional space x(t), and the hidden state Convert to y(t), that is, y(t) = C(Ah(t-1) + Bx(t));

[0052] A, B, and C are further discretized, and the time scale is set to Δ. The discrete parameter transformation under the zero-order hold rule is:

[0053]

[0054] The discretized length T input sequence is used with the kernel The equivalent calculation of the global convolution is:

[0055]

[0056] Assume input T is the sequence length, C is the embedding dimension, and the output can be transformed into:

[0057]

[0058] Norm, Mixer and MLP represent layer normalization, token mixing block and multi-layer perceptron respectively, and n represents the nth layer;

[0059] In the token mixing block, the input operation is transformed:

[0060]

[0061] Where Linear represents the linear layer, Scan represents the selective scan sampling layer, σ is the SiLU activation function, Conv and Concat represent one-dimensional convolution and cascade operations;

[0062] f o The partition corresponds to f Q 、f K 、f V , calculate the final output value according to this formula:

[0063]

[0064] In order to verify the invention effect of this application, the inventor conducted the following verification experiments:

[0065] All experiments were completed on a self-constructed door stamping defect detection dataset. The image size in this dataset is 2448×2048 pixels, and it contains five target categories, namely cracks, scratches, holes, bumps, and oil stains. Except for holes, all other images are defect labels. There are a total of 1660 images, which are randomly divided into training set, validation set, and test set according to 7:2:1.

[0066] The hardware environment of this experiment is: Intel(R) Xeon(R) CPU E5-2696 v3 processor, NVIDIA TITANRTX graphics card; the software environment is: Ubuntu 18.04.5LTS system, Python 3.7.15 interpreter, Pytorch2.0 deep learning framework; the CUDA version is Cuda_11.3.

[0067] The statistics of the number of categories before augmentation and the control group training experiments of the detection model include:

[0068] In this experiment, according to the actual detection model's requirements for the balance of data in each category, different rounds of fitting comparison experiments were performed. The initial statistics of each category in the training set are shown in Table 1.

[0069] Table 1

[0070]

[0071] There is an obvious uneven distribution of categories in the dataset.

[0072] Set the training parameters of the detection model YOLOv5m v6.0: training rounds epoch: 600, optimizer optimizer: SGD, initial learning rate lr0: 0.01, final learning rate lrf: 0.001, learning rate decay strategy lr_schedule: linear, and record the fitting of each defect detection rate during the last 300 rounds of fitting as shown in Table 2.

[0073] Table 2

[0074]

[0075] From Table 1 and Table 2, we can see that when the defect category distribution is extremely uneven due to the small amount of data, in order to improve the overall detection accuracy, the model will fit in the direction of increasing the detection rate of the category with a larger proportion in the fitting process, which will reduce the detection effect of other categories, and thus limit the overall detection rate. For example, from Table 2, we can see that because the crack defect has a large number of labels, the detection rate of a single item increases with the increase of the number of training rounds, but the detection rates of the three defect types of oil stains, strains and bumps decrease with the increase of the number of training rounds.

[0076] The training experiments of the augmented category statistics and detection model experimental group include:

[0077] In this experiment, the generative model of the present invention is used to augment the defect categories in the self-constructed data set. The distribution of the number of category labels after augmentation is shown in Table 3.

[0078] Table 3

[0079]

[0080] The distribution of defect categories is more even.

[0081] Set the training parameters of the detection model YOLOv5m v6.0: training rounds epoch: 600, optimizer optimizer: SGD, initial learning rate lr0: 0.01, final learning rate lrf: 0.001, learning rate decay strategy lr_schedule: linear, number of iterations N: 3 times, record the last iteration, the fitting of each defect detection rate during 300 rounds of fitting is shown in Table 4.

[0082] Table 4

[0083]

[0084] After the data volume is balanced, the overall detection accuracy has been significantly improved. At the same time, the problem of decreased single-item detection rate due to the low proportion of categories has also been alleviated.

[0085] Generate image baseline comparison experiment:

[0086] In order to compare the image generation effects of the generation model before and after the improvement, five crack images were selected as the validation set in the data set for comparative testing, taking information entropy, average gray value, and gray variance as indicators. The results are shown in Table 5.

[0087] Table 5

[0088]

[0089] The test results on 5 verification images show that the three indicators of the images generated by the improved generation model of the present invention are closer to the original images than those generated by the original pix2pix model, which means that the generation model of the present invention has a better generation effect, and the generated false features are closer to the original image features, which can play a more effective guiding role in the training of the detection model.

[0090] Pix2pix, MambaVision, and YOLOv5 used in the present invention are all very common processing means or calculation methods in the prior art. Those skilled in the art can obtain relevant content from literature related to the prior art.

Claims

1. A method for online defect detection of door stamping parts, characterized in that: It includes a data acquisition module, a data augmentation module and a defect detection module; the data augmentation module is provided with a generation model constructed by using a pix2pix network, and the defect detection module is provided with a detection model constructed by using a YOLOv5 network; the convolution kernel in the pix2pix network of the generation model adopts a MambaVision network; The method comprises: 1) The data acquisition module collects multiple real defect images of door stamping parts on the production line as the original training data set, and then the data acquisition module transmits the original training data set to the data augmentation module; 2) The data augmentation module generates multiple simulated images of door stamping defects as augmented training data sets based on the received original training data sets using the generative model, and then the data augmentation module transmits the original training data sets and the augmented training data sets to the defect detection module; 3) The defect detection module uses the original training data set and the augmented training data set to train the detection model to obtain a usable detection model; The defect detection module uses available detection models to perform online real-time detection of defects in door stampings.

2. The method for online defect detection of vehicle door stamping parts according to claim 1, characterized in that: In step 3), the defect detection module trains the detection model in the following manner: 1) Use the original training data set to train the detection model until the model converges to obtain the first round of updated model; 2) Using a training data set composed of a mixture of the original training data set and the augmented training data set to train the first round of updated models until the model converges, and obtain the second round of updated models; 3) Use the original training data set to train the second round of updated model to guide model convergence and obtain a usable detection model.

3. The method for online defect detection of door stamping parts according to claim 1 or 2, characterized in that: The deconvolutional layer feature map dimensions of the MambaVision network match those of the pix2pix network.

Images