A rapid surface scratch detection system based on shallow anisotropic pyramid network

Abstract

The present invention provides a surface scratch rapid detection system based on a shallow anisotropic pyramid network, comprising the following steps: S1 data preprocessing; S2 data enhancement; S3 model building; S4 model training. The present invention has the following beneficial effects: 1. The present invention has a lower number of network parameters. 2. The present invention uses an additive method in Dense connection to fuse features, which has fewer parameters and less training time than traditional Dense connection. The present invention uses anisotropic convolution operations to replace traditional convolution operations, further reducing the number of network parameters.

Description

Technical Field

[0001] The present invention relates to the field of surface scratch detection, and in particular to a surface scratch rapid detection system based on a shallow anisotropic pyramid network. Background Art

[0002] Scratches on product surfaces not only affect the product's appearance but also significantly impact its performance (such as lubrication, sealing, and load-bearing properties). Therefore, accurate detection and identification of surface scratches is a crucial component of product quality inspection. Currently, most companies rely heavily on manual inspection for product quality. This method is labor-intensive and susceptible to human influence. Furthermore, manual inspection methods cannot effectively and promptly summarize relevant scratches, making it difficult to provide timely guidance for subsequent product improvements and impacting the adjustment and upgrade of the entire production line. Therefore, factories urgently need a fast, accurate, and automatic edge detection method.

[0003] In recent years, deep learning-based methods, with their powerful feature representation capabilities, have replaced traditional detection methods based on manually selected features and become the mainstream research method in multiple fields, such as target recognition, defect detection, and face recognition. In the field of surface defect detection, current research methods use strategies such as Dense and Res structures, modules such as pyramids, and multi-model and multi-stage combinations to alleviate problems such as information loss and scale imbalance during the network feature extraction process, thereby improving detection effectiveness. Although these methods / strategies have effectively improved detection results, they still have the following problems: 1) The improvement is not significant (<2%), making it difficult to achieve significant performance improvements in practical applications; 2) The network parameters are large, and the training process relies heavily on large samples, resulting in serious overfitting problems; 3) The overall system is complex, making system deployment difficult; and 4) The prediction speed is slow, making it difficult to meet the quality inspection needs of factories. Summary of the Invention

[0004] To address the following problems with surface scratch detection in existing technologies: 1) difficulty in sample collection, limited availability of samples, and significant similarity between samples; 2) detection accuracy being heavily dependent on detailed information; and 3) high prediction accuracy requirements, the present invention provides a surface scratch rapid detection system based on a shallow anisotropic pyramid network, comprising the following steps:

[0005] S1 data preprocessing;

[0006] S2 data enhancement;

[0007] S3 model building;

[0008] S4 model training;

[0009] Furthermore, the step S1 includes:

[0010] The original image and the corresponding label are reduced to 256×256 size, and then normalized using the following formula:

[0011]

[0012] Where μ and σ are the mean and variance of the original image data respectively, and x represents the original image data.

[0013] Furthermore, the step S2 includes:

[0014] Before training, the Pix2Pix model is used to generate images, and the number of generated images is three times that of the original images. Then, the first strategy is used to further enhance the images enhanced by the GAN network (generative adversarial network). The first strategy enhancement formula is as follows:

[0015] x * =λx i +(1-λ)x j

[0016] y * =λy i +(1-λ)y j

[0017] Where x and y represent the original image data, λ represents the label and fusion coefficient. The image is enhanced by adjusting λ and the combination of different images. The number of generated images is 6 times that of the original image. During the training process, random selection, random cropping, and noise processing are used to further enhance the data input to the network.

[0018] The Pix2pix model establishes a general architecture to solve the image translation problem, so that we don’t have to redesign a loss function for each feature.

[0019] Furthermore, step S3 includes:

[0020] The SPN network was constructed using the PyTorch deep learning library. It employs an encoder-decoder framework. The encoder module extracts features using a stack of convolutional blocks and reduces the spatial resolution of features using a max-pooling operation with a stride of 2. The decoder module uses trilinear interpolation to gradually restore the semantically rich feature maps extracted by the encoder module to the original image size and classifies each pixel in the image. The encoder consists of three stages, with the number of channels set to 64, 128, and 256, respectively. The dilation coefficient d of the convolution in the DASAPM module is set to 1, 3, 5, 7, and 12, respectively. Auxiliary losses are added to stages 1 and 2.

[0021] The Encoder-Decoder model is a framework within which different algorithms can be used to solve different tasks. First, the encoder transforms the input sequence into a dense vector of fixed dimension. The decoding phase converts this activation state into the target translation.

[0022] PyTorch is an open-source Python machine learning library based on Torch, used for applications such as natural language processing.

[0023] Furthermore, step S3 includes:

[0024] The model uses a combination of the first loss function and the second loss function as the network loss function. The formula of the first loss function is as follows

[0025] CE(y,p)=ylogp+(1-y)log(1-p)

[0026] The second loss function formula is as follows:

[0027]

[0028] The hybrid loss function is:

[0029] Loss(y,p)=CE(y,p)+DiceLoss(y,p)

[0030] Auxiliary losses are added in stage1 and stage2, and the overall loss function of the network is:

[0031] Loss total =Loss main (y,p)+λ1Loss aux1 (y,p)+λ2Loss aux2 (y,p)

[0032] Among them, Loss main , Loss aux1 , Loss aux2 They represent the main loss, the auxiliary loss of stage1, and the auxiliary loss of stage2 respectively. λ1 and λ2 are weight coefficients, which are set to 0.5 and 0.8.

[0033] Furthermore, the step S4 includes:

[0034] The validation was performed using metal surface scratch data collected on-site. The data consisted of 65 original images in .png format and their corresponding manual annotation results. 50 sets were randomly selected as training sets, 5 sets as validation sets, and 10 sets as test sets. The training data was enhanced using the method in step 2.

[0035] The model was optimized using the SGD (Stochastic Gradient Descent) optimizer with a kinetic energy of 0.9, a learning rate of 0.0001, a batch size of 6, and a maximum number of iterations of 1000 epochs. The early-stopping mechanism was used to control model training, with a tolerance limit of 10 epochs.

[0036] SGD performs gradient updates on each sample each time it is updated. For a large data set, there may be similar samples, and SGD only performs one update at a time, so there is no redundancy, it is faster, and new samples can be added.

[0037] An epoch means that all data are sent into the network and a forward calculation + back propagation process is completed.

[0038] Dense represents a fully connected layer.

[0039] Furthermore, the method further includes step S5 of model performance evaluation, wherein step S5 includes:

[0040] The first indicator is used to evaluate the accuracy of model segmentation on the test set. The value range of the first indicator is [0,1]. The calculation formula of the first indicator is as follows

[0041]

[0042] The beneficial effects of the present invention are:

[0043] 1) The present invention has a lower amount of network parameters.

[0044] 2) The present invention adopts the addition method to fuse features in the Dense connection, which has a small number of parameters and takes less training time than the traditional Dense connection. The present invention adopts various anisotropic convolution operations to replace the traditional convolution operations, further reducing the number of network parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of surface scratch defects.

[0046] Figure 2 Schematic diagram of SPN network structure.

[0047] Figure 3 Schematic diagram of relevant structures in the SPN network.

[0048] Figure 4 Schematic diagram of image enhancement.

[0049] Figure 5 Schematic diagram of GAN network image enhancement results.

[0050] Figure 6 Flowchart of the present invention. DETAILED DESCRIPTION

[0051] The inventive concept of the present invention is:

[0052] 1 Shallow Anisotropic Pyramid Network (SPN). Figure 2As shown, the SPN employs a classic encoder-decoder architecture. The encoder extracts high-level semantic features through a stack of convolution and pooling operations, while the decoder restores these features to the original image size through upsampling and performs pixel classification. To mitigate detail loss during feature extraction and simplify the network to improve prediction speed, the SPN encoder module consists of only three stages and performs two downsampling operations. Furthermore, to compensate for the potential lack of network performance and insufficient high-level feature extraction caused by only two downsampling operations, stage 3 employs a dense atrous spatial anisotropic pyramid module (DASAPM). The decoder utilizes an attention feature fusion module (AFFM) to mitigate the semantic gap that exists when fusing features at different levels. To reduce the number of network parameters, this method replaces the traditional 3×3 convolution with anisotropic convolution in the AFFM. At the same time, in order to further reduce the number of network parameters, the present invention uses a trilinear spline interpolation method to perform feature upsampling in the encoder.

[0053] 2 Dense Atrous Spatial Anisotropic Pyramid Module (DASAPM). In semantic segmentation tasks, large receptive fields and rich high-level semantic feature information play an important role in improving the segmentation effect of the model. A common method is to reduce the resolution of features through pooling operations, thereby increasing the receptive field of the network, but this method will lead to serious loss of details and distortion of the target morphology. Using deep networks is a common method for extracting high-level semantic features, but this method will greatly increase the parameters of the model and reduce the prediction speed of the network. The present invention uses the atrous spatial anisotropic pyramid to increase the receptive field of the network, and adopts dense connections to improve the high-level feature extraction capability of the network. Figure 3 The Dense connection shown fuses features of different scales, effectively utilizing previously extracted semantic features to guide subsequent semantic feature extraction, thereby improving the network's ability to extract deep semantic features. To alleviate the problems of large parameters and time-consuming training in traditional Dense connections, the present invention uses an additive method to fuse features in Dense connections. In addition, to further reduce the number of network parameters, the present invention uses anisotropic convolution operations to replace traditional convolution operations.

[0054] Example 1:

[0055] In the present invention, x∈R w×h and y∈R w×h Denote the original image and its corresponding manual annotation result, w and h are the length and width of the image respectively. The segmentation model is represented by M:p=f(θ,x), where θ represents the network parameters, p represents the probability map of the predicted result, and M is the abbreviation of the model. Represents the prediction results:

[0056]

[0057] The present invention comprises the following steps:

[0058] S1 Data Preprocessing

[0059] The original image and the corresponding label are resized to 256×256 respectively, and then normalized using Formula 2.

[0060]

[0061] Where μ and σ are the mean and variance of the original image data respectively.

[0062] S2 data enhancement

[0063] The defect collection of scratches on the product surface is difficult, and there are few data sets available for model training. Therefore, it is necessary to enhance the image. Figure 4 The image enhancement scheme shown: Before training (offline enhancement), the present invention first adopts the first strategy to enhance the original image. In the specific operation, the present invention uses the Pix2Pix model to generate the image ( Figure 5 ), the number of generated images is about 3 times that of the original image. The first strategy enhancement is shown in Formula 3:

[0064] x * =λx i +(1-λ)x j

[0065] y * =λy i +(1-λ)y j (Formula 3)

[0066] Where x, y, and λ represent the original image, label, and fusion coefficient, respectively. By adjusting λ and combining different images, the number of images generated is approximately six times that of the original image. During the training process (online enhancement), the present invention further enhances the input network data using methods such as random selection, random cropping, and noise processing to alleviate potential overfitting issues during model training.

[0067] S3 model building

[0068] The SPN network was constructed using the PyTorch deep learning library. The SPN employs the classic Encoder-Decoder framework. The Encoder module extracts features using a stack of convolutional blocks and reduces the spatial resolution of features using a maximum pooling operation with a stride of 2. The Decoder module gradually restores the semantically rich feature map extracted by the Encoder module to the original image size using trilinear interpolation and classifies each pixel in the image. The Encoder consists of three stages, with the number of channels set to 64, 128, and 256, respectively. The dilation coefficient d of the convolution in the DASAPM module is set to 1, 3, 5, 7, and 12, respectively. To improve the convergence speed of the model, auxiliary losses are added to the network in stages 1 and 2.

[0069] The following is an explanation of the model's loss function:

[0070] The combination of the first loss function and the second loss function is used as the loss function of the network. The formula of the first loss function is as follows:

[0071] CE(y,p)=ylpgp+(1-y)log(1-p) (Formula 4)

[0072] The second loss function formula is as follows:

[0073]

[0074] Then the mixed loss function is:

[0075] Loss(y,p)=CE(y,p)+DiveLoss(y,p) (Formula 6)

[0076] In order to accelerate the convergence of the network, this paper adopts a deep supervision strategy and adds auxiliary losses in stage 1 and stage 2. The overall loss function of the network is:

[0077] Loss total =Loss main (y,p)+λ1Loss aux1 (y,p)+λ2Loss aux2 (y,p) (Formula 7)

[0078] Among them, Loss main , Loss aux1 , Loss aux2They represent the main loss, the auxiliary loss of stage 1, and the auxiliary loss of stage 2. λ1 and λ2 are weight coefficients, which are set to 0.5 and 0.8 respectively in the present invention.

[0079] S4 model training

[0080] This method was validated using field-collected metal surface scratch data. This data set consisted of 65 original images in .png format and their corresponding manually annotated images. Fifty sets were randomly selected as the training set, five as the validation set, and ten as the test set. The training data was augmented using the method from step 2.

[0081] The model was optimized using the SGD optimizer with a kinetic energy of 0.9, a learning rate of 0.0001, a batch size of 6, and a maximum number of iterations of 1000 epochs. An early-stopping mechanism was used to control model training, with a tolerance limit of 10 epochs. This means that if the model performance on the test and validation sets does not improve after 10 epochs, model training is terminated.

[0082] S5 model performance evaluation

[0083] The first metric was used to evaluate the segmentation accuracy of the model on the test set. The value range of the first metric is [0, 1]. A higher value indicates better segmentation. The calculation method is shown in Formula 8. The FPS (Frames Per Second) metric was used to evaluate the prediction speed of the model. A higher FPS indicates a faster prediction speed.

[0084]

[0085] Table 1 Experimental results of surface defect model performance evaluation

[0086] Method Diyi (%) FPS RCF 33.7 64 BDCN 39.5 20 ENFNet 39.35 72 SPN 39.1 95

[0087] The experimental results in Table 1 confirm that the prediction speed of the SPN network adopted by the present invention is significantly better than that of other networks, and has unexpected technical effects and significant progress.

[0088] The above embodiments are only preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any changes that adopt the design principles of the present invention and any changes made through non-creative work on this basis should fall within the scope of protection of the present invention.

Claims

1. A surface scratch rapid detection system based on a shallow anisotropic pyramid network, comprising the following steps: S1 data preprocessing; S2 data enhancement; S3 model building; S4 model training; Based on a shallow anisotropic pyramid network, the proposed method uses an encoder-decoder framework. The encoder module extracts features using a stack of convolutional blocks and employs a max-pooling operation with a stride of 2 to reduce the spatial resolution of the features. The decoder module uses trilinear interpolation to gradually restore the semantically rich feature maps extracted by the encoder module to the original image size and classifies each pixel in the image. The encoder consists of three stages, with the number of channels set to 64, 128, and 256, respectively. The shallow anisotropic pyramid network adopts an encoder-decoder structure. The encoder extracts high-level semantic features through a stack of convolution and pooling operations. The decoder restores the semantic features extracted by the encoder to the original image size through upsampling and realizes pixel classification. The shallow anisotropic pyramid network contains three stages in the encoder module and performs two downsampling. Stage 3 uses a dense void space anisotropic pyramid module. The attention feature fusion module is used in the decoder to alleviate the semantic gap problem in the process of fusing features at different levels. The attention feature fusion module uses anisotropic convolution operations instead of traditional 3×3 convolutions. The encoder uses trilinear spline interpolation to upsample features.

2. The surface scratch rapid detection system based on shallow anisotropic pyramid network according to claim 1, wherein step S1 comprises: The original image and the corresponding label are reduced to 256×256 size, and then normalized using the following formula: ； Where μ and σ are the mean and variance of the original image data respectively, and x represents the original image data.

3. The surface scratch rapid detection system based on shallow anisotropic pyramid network according to claim 1, wherein step S2 comprises: Before training, the Pix2Pix model is used to generate images, and the number of generated images is three times that of the original images. Then, the first strategy is used to further enhance the images enhanced by the GAN network. The first strategy enhancement formula is as follows: ； ； in , represents the original image data, Represents labels and fusion coefficients, by adjusting The image is augmented by combining different images. The number of generated images is 6 times that of the original images. During the training process, random selection, random cropping and noise processing are used to further enhance the data input to the network.

4. The surface scratch rapid detection system based on shallow anisotropic pyramid network according to claim 1, wherein step S3 comprises: The model uses a combination of the first loss function and the second loss function as the network loss function. The formula of the first loss function is as follows: ； A probability map representing the predicted results; The second loss function formula is as follows: ； The hybrid loss function is: ； Auxiliary losses are added in stage1 and stage2, and the overall loss function of the network is: ； in , , Represent the main loss, the auxiliary loss of stage1, and the auxiliary loss of stage2 respectively. and are weight coefficients, set to 0.5 and 0.

8.

5. The surface scratch rapid detection system based on shallow anisotropic pyramid network according to claim 1, wherein step S4 comprises: The validation was performed using metal surface scratch data collected on-site. The data consisted of 65 original images in .png format and their corresponding manual annotation results. 50 sets were randomly selected as training sets, 5 sets as validation sets, and 10 sets as test sets. The training data was enhanced using the method in step 2. The SGD optimizer with a kinetic energy of 0.9 is used to optimize the model, the learning rate is set to 0.0001, the batch size is set to 6, the maximum number of iterations is 1000 epochs, and the early-stopping mechanism is used to control the training of the model, with a tolerance limit of 10 epochs.

6. The surface scratch rapid detection system based on shallow anisotropic pyramid network according to claim 1 further comprises step S5 of model performance evaluation, wherein step S5 comprises: The first indicator is used to evaluate the accuracy of model segmentation on the test set. The value range of the first indicator is [0, 1]. The calculation formula of the first indicator is as follows: 。

Images