Automobile seat surface defect detection method based on deep learning

Abstract

The invention relates to the field of machine vision, and particularly discloses an automobile seat surface defect detection method based on deep learning. The automobile seat surface defect detection method based on deep learning is designed for solving the problem that the detection effect is reduced due to the fact that feature extraction is insufficient in the defect detection process of a YOLOv5 algorithm. The method comprises the following steps: a, constructing a data set; b, on the basis of a K-means + + algorithm, the size of the YOLOv5 anchor frame is clustered again; c, the YOLOv5 original network is improved according to requirements; d, training the improved YOLOv5 model obtained in the step c by using the data set obtained in the step a; e, obtaining a weight trained by the YOLOv5; f, testing the verification set image by using the model trained in the step e; and g, detecting the surface of the seat by using the model trained in the step e.

Description

Technical Field

[0001] The present invention relates to the field of machine vision, and specifically discloses a method for detecting surface defects of automobile seats based on deep learning. Technical Background

[0002] Car seats are one of the essential parts of cars. However, during the production process, the surface of car seats is prone to various damages. Therefore, car seats must be inspected in many aspects before leaving the factory. For consumers, seat surface defects may reduce riding comfort and increase safety risks. Therefore, in order to avoid these effects, companies should pay attention to the quality control of seat surfaces to ensure that products meet consumer expectations and requirements. How to quickly and accurately detect defects online, and provide feedback and adjustments to defects to improve product quality has become an urgent need for companies to improve their core competitiveness.

[0003] The surface of car seats is generally made of leather, and most of them are artificial leather, the most common of which are polyurethane and polyvinyl chloride. At present, there are relatively few tests on the reproduction process of artificial leather, and more research focuses on the small defects produced in the production process of artificial leather, such as wormholes, wrinkles, etc. Therefore, the research focus of this invention is the defect detection in the reproduction process of artificial leather, that is, the production process of car seats.

[0004] For a long time, manual inspection still occupies a dominant position in the surface defect detection of automobile seats. This is mainly due to its flexibility and low initial cost. However, with the development of industrial automation and intelligent detection technology, the low efficiency, inconsistency, high cost and potential safety hazards of manual inspection have gradually exposed its limitations. Deep learning technology is one of the core technologies that has achieved breakthrough success in recent years. The mainstream target detection algorithm is YOLO (You Only Look Once), which is widely used in defect detection tasks because it can balance speed and accuracy.

[0005] For existing deep learning target detection algorithms, such as YOLOv5, as the number of network layers becomes deeper and deeper, a large number of ordinary convolution operations are no longer sufficient to extract complete data features, and irrelevant information accounts for a large proportion in the feature extraction process, which will increase network training time and reduce defect detection accuracy and generalization ability. Summary of the invention

[0006] In order to solve the above technical problems, the present invention discloses a method for detecting surface defects of automobile seats based on deep learning, which aims to solve the problems of insufficient feature extraction and large proportion of irrelevant information in existing defect detection technologies. Based on the data set collected by the industrial camera as the model input, the improved YOLOv5 network is used for training, and finally the surface defect detection of automobile seats is realized. The technical solution of the present invention is as follows:

[0007] A method for detecting surface defects of automobile seats based on deep learning comprises the following steps:

[0008] a. Build a data set and divide it into training set, validation set and test set in a ratio of 8:1:1;

[0009] b. Based on the K-means++ algorithm, the YOLOv5 anchor box size is re-clustered so that it can better cover all the targets in this dataset;

[0010] c. Improve the original YOLOv5 network according to needs;

[0011] d. Use the data set obtained in a to train the improved YOLOv5 model obtained in c;

[0012] e. Get the weights trained by YOLOv5;

[0013] f. Use the model trained in e to test the validation set images;

[0014] g. Use the trained model in e to detect the seat surface.

[0015] Furthermore, the improved network model in step c specifically includes:

[0016] S1: A new E-SPP (Enhanced-Spatial Pyramid Pooling) structure is proposed to replace the original SPP structure. Multiple branches are added on the basis of the original SPP. Each branch contains multiple two-dimensional convolution operations, so that the model can extract richer and more complex features, which helps to capture more local and global information and enhance the model's receptive field and feature expression capabilities.

[0017] S2: A new S-C3 (Sufficient-Cross Stage Partial Bottleneck with 3Convolutions) structure is proposed to replace the original C3 structure, which can extract more fully the features of each picture and prevent the model from overfitting during training; the snake convolution DSConv is used to replace the ordinary convolution in Bottleneck, which allows the convolution operation to be offset in the x-axis and y-axis directions, giving the convolution kernel more flexibility and enabling it to focus on the complex geometric features of the target; Dropout prevents overfitting by randomly discarding neurons during the training of the neural network. The specific formula of the convolution kernel is: K={(x-1,y-1),(x-1,y),…,(x+1,y+1)} Among them, K is a standard 3×3 two-dimensional convolution kernel. DSConv linearizes the standard convolution kernel in the x-axis and y-axis directions. i±c =(x i+c ,y i+c ) represents the specific position of a convolution kernel of size 9 in the x-axis direction grid, c = 0, 1, 2, 3 represents the horizontal distance from the center grid, Δ is the deformation offset, K j±c Changes in the y-axis direction.

[0018] S3: Use Dropout to operate on the output of Bottleneck and prevent the model from overfitting by randomly losing neurons. The specific expression is: dropout(x)=x⊙m Where m and x are vectors of the same shape, each element is a decimal between 0 and 1 generated by the random generator function rand, and ⊙ represents the dot product between elements, i.e., the Hadamard product;

[0019] S4: The BiFormer attention module is introduced in the Neck part of the YOLOv5 network. It is an attention module that contains a two-layer routing attention mechanism, which can capture global and local feature information, increase the weight of important information, and suppress irrelevant information, thereby improving the performance of the model in visual tasks.

[0020] Compared with the prior art, the advantages of the present invention are:

[0021] The present invention proposes a method for detecting surface defects of automobile seats based on deep learning. The method is characterized in that the present invention takes a YOLOv5 network as a basic model. In order to make the network model perform well on the data set in the present invention, the original network structure of YOLOv5 is improved, including improving the SPP structure and the C3 structure, and introducing the BiFormer attention mechanism. Compared with the original network model, the improved network model has improvements in feature extraction and generalization capabilities, effectively improving the detection accuracy of surface defects of automobile seats, and solving the technical problems existing in the current surface defects of automobile seats.

[0022] Figure 1 This is the original YOLOv5 network structure diagram;

[0023] Figure 2 This is a flow chart of a method for detecting surface defects of automobile seats based on deep learning according to the present invention;

[0024] Figure 3 This is the structure diagram of the improved E-SPP;

[0025] Figure 4 This is the improved Bottleneck structure diagram;

[0026] Figure 5 This is the improved S-C3 structure diagram;

[0027] Figure 6 This is the improved YOLOv5 network structure diagram;

[0028] FIG7(a) is a schematic diagram of an image with defect 1, and FIG7(b) is a schematic diagram of the detection effect;

[0029] FIG8(a) is a schematic diagram of an image with defect 2, and FIG7(b) is a schematic diagram of the detection effect;

[0030] FIG9(a) is a schematic diagram of an image with defect 3, and FIG7(b) is a schematic diagram of the detection effect. DETAILED DESCRIPTION

[0031] The present invention discloses a method for detecting surface defects of automobile seats based on deep learning. The technical route of the present invention is clearly and completely described below in conjunction with the accompanying drawings of the present invention:

[0032] YOLOv5 is a widely used target detection algorithm. Its training and testing are performed on the COCO dataset, so its generalization ability needs to be improved. For different industries and different types of datasets, YOLOv5 faces problems such as insufficient feature extraction and a large proportion of irrelevant information. These problems will lead to a decrease in detection accuracy and fail to meet industrial production needs. In order to solve this problem, this embodiment improves YOLOv5, such as Figure 1 As shown in the figure, it is the original YOLOv5 network, which includes three parts: Backbone, Neck and Head.

[0033] The Backbone part includes Focus structure, Conv structure, C3 structure and SPP structure. Focus is to slice the image before it enters the network model and expand the input channel; Conv is a defined feature extraction operation, which includes a 3×3 two-dimensional convolution, BN (Batch Normalization) and SiLU activation function; C3 uses the residual idea, one part performs convolution operation, and the other part is concatenated with the result of the convolution operation of the previous part; SPP performs maximum pooling on feature maps of different sizes to enhance the model's perception of targets of different scales. Figure 1, the Focus structure, Conv structure, C3 structure and SPP structure together constitute the Backbone part of the network. The Neck part is an intermediate feature extraction network added on the basis of the Backbone, which is mainly used to enhance the expression ability and receptive field of the model. SPP performs different pooling operations on the input feature maps, fuses information of different scales, and then sends them to the Neck part. After multi-level fusion, the feature maps of three different scales are sent to the Head part. The Head part mainly outputs prediction boxes, each of which is composed of confidence score, class probability and bounding box coordinates. The confidence indicates the probability of whether there is an object in the box, the class probability indicates the probability that the object in the box belongs to each category, and the bounding box position indicates the position and size of the object.

[0034] Figure 2 The flowchart of the method for detecting surface defects of automobile seats based on deep learning of the present invention is as follows: Figure 2 As shown, the specific method is as follows:

[0035] a. Build a data set. Training deep learning network models requires a large amount of data. It is difficult for both enterprises and individuals to obtain rich data sets, which has become a limiting factor in the development of the defect detection industry. The present invention uses industrial cameras to collect three common defect images from the surface of car seats, including dirt, scratches and burns. For convenience, these three defects are referred to as defect 1, defect 2 and defect 3. The specific steps are:

[0036] S1: Use industrial cameras to collect images of seat surface defects;

[0037] S2: Expand and enhance the images in the dataset, including adding Gaussian noise, changing brightness, and cropping. The dataset is divided into training set, validation set, and test set in a ratio of 8:1:1;

[0038] S3: Use the open source software Labelimg to annotate each image in the dataset and obtain the corresponding label.

[0039] b. Based on the K-means++ algorithm, the YOLOv5 anchor box size is re-clustered so that it can better cover all targets in this dataset. The specific expression is: A sample is randomly selected from the input data set as the first cluster center. For each sample in the data set, the distance D(x) between it and the nearest cluster center among the selected cluster centers is calculated. After obtaining the P value, it is determined whether the sample is the next cluster center. Then the above steps are repeated to obtain all cluster centers. Finally, the standard K-means algorithm is used to iterate based on the obtained cluster center as the initial cluster center to obtain the final result.

[0040] c. Improve the original network model of YOLOV5 according to the needs. The specific steps are:

[0041] S1: Figure 3 It is a newly improved E-SPP network. It uses the residual idea to introduce multiple Conv operations for feature extraction based on SPP, including a 3×3 two-dimensional convolution, BN (Batch Normalization) and SiLU activation function. The number of the first layer of Conv is three, and the second and third layers are six respectively. Then, it is spliced ​​with the feature map processed by Maxpool to obtain a new feature map with rich semantics.

[0042] S2: Figure 4 For the improved Bottleneck, Figure 5 It is a new and improved S-C3 structure. Use snake convolution DSConv to replace the ordinary convolution in Bottleneck, which allows the convolution operation to be offset in the x-axis and y-axis directions, which can give the convolution kernel more flexibility and enable it to focus on the complex geometric features of the target; then use dropout to calculate the output value of the Bottleneck module. Snake convolution can make the convolution kernel more flexible, allowing it to focus on the complex geometric features of the target; Dropout can prevent overfitting by randomly discarding neurons during the training of the neural network. The specific expression of the DSConv convolution kernel is: K=Σ K′ B(K′,K)·K′ B(K,K′)=b(K x ,K′ x )·b(K y ,K′ y ) DSConv linearizes the standard convolution kernel in both the x-axis and y-axis directions. i±c =(x i+c ,y i+c) represents the specific position of a convolution kernel of size 9 in the x-axis direction, where c = 0, 1, 2, 3 represents the horizontal distance from the center grid, and Δ is the deformation offset, which makes the convolution kernel more flexible and can focus on the complex geometric features of the target. Each grid position K in the convolution kernel k i±c The selection of is a cumulative process. Therefore, the offset Δy needs to be accumulated to ensure that the convolution kernel conforms to the linear morphological structure, K j±c Then it changes in the y-axis direction. K represents the decimal position of Equation 2 and Equation 3, K′ is listed as all integer spatial positions, B is the bilinear interpolation kernel, which can be decomposed into two one-dimensional kernels;

[0043] S3: Use dropout to calculate the output value of Bottleneck. The specific expression is: dropout(x)=x⊙m Where m is a vector with the same shape as x, and each element is a decimal between 0 and 1 generated by the random generator function rand. ⊙ represents the dot product between elements, i.e., the Hadamard product.

[0044] S4: Introduce the BiFormer attention mechanism in the YOLOv5 network. BiFormer is an attention module combined with BRA (Bi level Routing Attention). Through the two-layer routing mechanism of BRA, BiFormer can capture global and local feature information at the same time, thereby improving the performance of the model in visual tasks. BRA first filters out irrelevant key-value pairs at a macro level, retaining only a small number of relevant key-value pairs to achieve sparsity and reduce computational and memory overhead. The specific formula of BRA is: Q=X r W q ,K=X r W k ,V=X r W v A r =Q r (K r ) T . I r =topkIndex(A r ) O=Attention(Q,K g ,V g )+LCE(V) K g =gather(K,I r ),V g =gather(V,I r ) First, the feature map is divided into S×S non-overlapping regions and linearly mapped to the projected query, key, and value tensors. The region-level query Q is derived by applying the average of each region to Q and K respectively. r and key K r , then through Q r and transpose K r Multiply them together to derive the adjacency matrix A of the region-to-region affinity graph r , adjacency matrix A r It can measure the semantic relevance of two regions. r is the area-to-area routing index matrix, K g and V g The collected key-value tensor is finally focused on the collected key-value pairs, where the function LCE(·) is parameterized using depthwise convolution and the kernel size is set to 5.

[0045] d. Use the data set obtained in a to train the improved YOLOv5 model obtained in c. The specific steps are:

[0046] S1: The labeled training set images obtained in a are sent to the improved YOLOv5 network, and are trained in the Backbone, Neck and Head parts in turn. The validation set images are used to test and display the training effect.

[0047] e. Get the weights trained by YOLOv5;

[0048] f. Use the model trained in c to test the test set images obtained in a;

[0049] g. Use the trained model in e to detect the seat surface.

[0050] The present invention mainly solves the problem of seat surface defect detection. By improving the YOLOv5 network, a method for detecting car seat surface defects based on deep learning is proposed. Finally, the purpose of intelligent detection of car seats produced in the factory is achieved. Compared with the existing detection algorithm, the present invention realizes the full extraction of image features by improving the network structure, while increasing the weight of important information, suppressing irrelevant information, and improving the detection effect. Three groups of different detection examples are given below. As shown in Figure 7(a), Figure 7(b), Figure 8(a), Figure 8(b), Figure 9(a), and Figure 9(b), Figure 7(a), Figure 8(a) and Figure 9(a) are three defective images, and Figure 7(b), Figure 8(b) and Figure 9(b) are corresponding defect detection effect diagrams based on the improved YOLOv5.

Claims

1. A method for detecting surface defects of automobile seats based on deep learning, characterized in that: The method comprises: a. Construct a dataset and divide it into training set, validation set and test set in a ratio of 8:1:1; b. Based on the K-means++ algorithm, re-cluster the YOLOv5 anchor box size so that it can better cover all targets in this dataset; c. Improve the original YOLOv5 network as needed; d. Use the dataset obtained in a to train the improved YOLOv5 model obtained in c; e. Obtain the trained weights of YOLOv5; f. Use the model trained in e to test the validation set images; g. Use the model trained in e to detect the seat surface.

2. The method for detecting surface defects of automobile seats based on deep learning according to claim 1, characterized in that: The improved YOLOv5 network model includes the following steps: S1: Improve the original SPP (spatial pyramid pooling) structure and add multiple branches on the basis of the original SPP. Each branch contains multiple two-dimensional convolution operations, so that the model can extract richer and more complex features, which helps to capture more local and global information and enhance the model's receptive field and feature expression capabilities. S2: Improve the original C3 (Cross Stage Partial Bottleneck with 3Convolutions) structure to make the network extract more features of each image and prevent the model from overfitting during training; Use DSConv to replace the ordinary convolution in Bottleneck so that the convolution operation can be offset in the x-axis and y-axis directions, giving the convolution kernel more flexibility so that it can focus on the complex geometric features of the target; Dropout prevents overfitting by randomly discarding neurons during the training of the neural network; The specific expression of the convolution kernel in Dsconv is as follows: DSConv linearizes the standard convolution kernel in both the x-axis and y-axis directions. i±c =(x i+c ,y i+c ) represents the specific position of a convolution kernel of size 9 in the x-axis direction grid, where c = 0, 1, 2, 3 represents the horizontal distance from the center grid, Δ is the deformation offset, K j±c is the change in the y-axis direction; S3: Use dropout to operate on the output of Bottleneck and prevent the model from overfitting by randomly losing neurons. The specific expression is: dropout(x)=x⊙m Where m is a vector of the same shape as x, each element is a decimal between 0 and 1 generated by the random generator function rand, and ⊙ represents the dot product between elements, i.e., the Hadamard product; S4: The BiFormer attention module is introduced in the Neck part of the YOLOv5 network. It is an attention module that includes a two-level routing attention mechanism (Bi level Routing Attention BRA), which can capture global and local feature information at the same time, increase the weight of important information, and suppress irrelevant information, thereby improving the performance of the model in visual tasks; BiFormer consists of BiFormer blocks, which include self-attention and feedforward neural networks, which can enhance the adaptability of BiFormer; the specific expression of the two-level routing mechanism BRA is: Q=X r W q ,K=X r W k ,V=X r W v A r =Q r (K r ) T I r =topkIndex(A r ) O=Attention(Q,K g ,V g )+LCE(V) K g =gather(K,I r ),V g =gather(V,I r ) The feature map is divided into S×S non-overlapping regions and linearly mapped to the projected query, key, and value tensors respectively; the region-level query Q is derived by applying the average value of each region to Q and K respectively r and key K r , via Q r and transpose K r Multiply to get the adjacency matrix A of the region-to-region affinity graph r , the adjacency matrix A r It can measure the semantic relevance of two regions; K g and V g For the collected key-value tensor, we focus on the collected key-value pairs, where the function LCE(·) is parameterized using depthwise convolution and the kernel size is set to 5.

Images