A scratch detection method based on deep convolutional neural network and image segmentation

Abstract

The present invention discloses a scratch detection method based on deep convolutional neural network and image segmentation. A multi-feature fusion module is designed according to the scratch morphological characteristics, added to the deep convolutional neural network structure, to identify and locate the scratches, and extract the scratch prediction frame image; the principal component points of the scratch prediction frame image are calculated using the principal component analysis method, and the scratch prediction frame image after adaptive mean segmentation is grown in 8 neighborhoods with the principal component points as the growth points, so as to achieve accurate segmentation of scratch pixels. The present invention can identify, locate and segment scratches with high precision, and provide convenience for subsequent precise measurement of scratch size.

Description

Technical Field

[0001] The present invention belongs to the field of automatic defect detection and precision measurement, and specifically relates to a scratch detection method based on deep convolutional neural network and image segmentation. Background Art

[0002] The increasing intelligence of modern industry places higher and higher demands on product surface quality. Usually, product surface quality problems occur in areas with uneven physical or chemical properties on the surface of the product, such as scratches on key parts of cars and trains, welds of laser-welded products, cracks in concrete, scratches on chip packages, etc. These defects can be attributed to scratches based on their morphological characteristics. Effective detection of surface scratches is of great significance to ensuring product quality and performance, improving the rate of newness, and ensuring safe use. Machine vision technology, as a non-contact, non-destructive automatic detection technology, has outstanding advantages such as safety and reliability, a wide spectral response range, and the ability to work for a long time in harsh environments. Applying machine vision technology to automatic scratch detection is the current research direction in this field.

[0003] However, scratch images at industrial sites often contain a lot of noise, and the lighting conditions are also complex and varied, resulting in large grayscale changes in the image and low contrast between scratches and background. This creates great difficulties in ensuring detection accuracy and speed for scratch detection tasks with complex surface structures and large detection volumes. In addition, in precision manufacturing fields such as aerospace, military weapons, semiconductors, and microelectronics, the automatic scratch detection task not only includes scratch recognition and positioning, that is, identifying whether there are scratches on the surface to be tested and determining the location coordinates of the scratches, but also requires pixel-by-pixel segmentation of the scratches and further measurement of the precise size of the scratches, which puts more stringent and demanding requirements on the automatic scratch detection task.

[0004] Traditional scratch detection methods based on machine vision usually rely on manually designed features, and the algorithms need to be optimized and designed according to actual detection needs and application scenarios. These algorithms focus on the segmentation and extraction of scratches, but ignore their recognition, which makes them susceptible to interference from noise or other types of defects. In recent years, the gradual rise of scratch detection methods based on deep learning has made up for the shortcomings of traditional methods in scratch recognition. This method requires training and learning of a large amount of labeled data, focusing on the recognition and location of scratches, but it is difficult to meet the requirements of precision detection for the accuracy of scratch pixel segmentation. Therefore, it is of practical significance to study a detection method that can identify, locate and accurately segment scratches. Summary of the invention

[0005] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art, to provide a scratch detection method based on deep convolutional neural network and image segmentation, to add a multi-feature fusion module designed according to the scratch morphological characteristics on the basis of the SSD network, to design a principal component growth segmentation algorithm for the extracted scratch prediction frame, to accurately segment the scratch pixels while effectively suppressing the background noise, to identify, locate and accurately segment the low-contrast and tiny scratches, to facilitate the subsequent precise measurement of the scratch size; on scratch data sets under two different application scenarios, the present invention has a higher scratch recognition accuracy than the current mainstream target recognition method while ensuring a faster detection speed, and a scratch segmentation effect is better than the current mainstream image segmentation method.

[0006] The technical solution of the present invention is: a scratch detection method based on deep convolutional neural network and image segmentation, comprising the following steps:

[0007] S1: According to the morphological characteristics of scratches, a multi-feature fusion module consisting of an upsampling layer, a lateral connection layer and a feature fusion layer is designed. The multi-feature fusion module upsamples high-level features to obtain the same scale as low-level features, and then fuses them through the lateral connection layer. The six feature layers used for prediction in the single-shot multi-box detector SSD network are respectively input into the multi-feature fusion module, and the feature layers output by the multi-feature fusion module are used for prediction in step S2;

[0008] S2: Construct the loss function of category and position, use the loss function of category and position to perform regression analysis on the feature layer output by the multi-feature fusion module, identify scratches based on the loss function, calculate the coordinates of the scratch prediction box, and output the scratch prediction box image;

[0009] S3: Use the principal component analysis method to calculate the scratch direction in the scratch prediction frame image, and calculate the mean point and the starting point from the scratch direction as the region growth seed point;

[0010] S4: Adaptively threshold segment the scratch prediction frame image, perform 8-neighborhood region growth on the pixel points in the scratch prediction frame image after adaptive threshold segmentation, restore the growth point set to the original image prediction frame position, and obtain the scratch segmentation result.

[0011] The specific operation steps of step S1 include:

[0012] S101: The upsampling layer in the multi-feature fusion module performs deconvolution operations on the last feature layer used for prediction in the SSD network and the 2nd, 3rd, 4th, and 5th feature layers in the fusion layer, and expands the size of the last feature layer used for prediction and the 2nd, 3rd, 4th, and 5th feature layers in the fusion layer to twice the original size;

[0013] S102: The lateral connection layer in the multi-feature fusion module performs convolution operations on the first five feature layers used for prediction in the SSD network, so that the number of channels of the first five feature layers used for prediction in the SSD network is reduced to 1 / 2 of the original number, which is consistent with the number of channels of the upsampled feature map to meet the fusion condition;

[0014] S103: The fusion layer in the multi-feature fusion module performs tensor calculation on the feature maps of the same size after upsampling and horizontal connection, and obtains 5 feature layers for prediction containing deep and shallow feature information. The last feature layer used for prediction in the SSD network is retained as the 6th feature layer for prediction, and the 6 feature layers output from the multi-feature fusion module are used for prediction in step S2.

[0015] The specific operation steps of step S2 include:

[0016] S201: Calculate the category and position loss functions, and extract and filter the priori boxes according to the loss functions. The calculation formulas for the category loss function and the position loss function are:

[0017]

[0018]

[0019] Among them, in the category loss function, Indicates whether the i-th prior frame matches the real object frame of the j-th target of the p-th feature. If A value of 1 indicates a match, and a value of 0 indicates a mismatch. represents the i-th prior box and the p-th feature, Indicates that the i-th prior frame does not match any feature, N is the number of matches between the scratch real object frame and the prior frame, pos is the positive sample prior frame, and neg is the negative sample prior frame; in the position loss function, is the deviation between the coordinates of the i-th prior box and the coordinates of the v-th prior box, represents the deviation between the jth scratch real object box and the vth prior box, Smooth L1 is the L1 regularization function, c x ,c y ,w,h are the center coordinates, width and height of the prior box respectively;

[0020] S202: Decode the scratch prediction frame and its coordinate position according to the center coordinates, width and height of the selected prior frame. Suppose the width and height of the input image and the scratch annotation object frame are w i ,h i ,w o ,h o , then the scratch prediction box coordinates are:

[0021]

[0022] S203: Cropping the input image to obtain a scratch prediction frame image.

[0023] The specific operation steps of step S3 include:

[0024] S301: Binarize the scratch prediction frame image, and set the scratch pixel after binarization to X=(x (1) ,x (2) ,...,x (n) ), each pixel is represented by its 2D coordinates as:

[0025]

[0026] In order to predict the scratch direction, the 2D data is mapped to a 1D space. Let the projection transformation matrix of the mapping be W, and the resulting projection is:

[0027] z (i) =W T x (i) ,i=1,2,...,n (5)

[0028] In order to preserve as much information as possible from the original data, the variance and Maximum, that is, maximize the trace of the sum of the variances of the projected data:

[0029]

[0030] Where I is the identity matrix, and the Lagrange multiplier method is used to solve it. The Lagrange function is assumed to be:

[0031] J(W)=tr(W T XX T W+λ(W T WI)) (7)

[0032] Among them, λ is the Lagrangian factor, and the derivative of W is:

[0033] XX T W=(-λ)W (8)

[0034] It can be seen from formula (8) that W is the covariance matrix XX T The eigenvector matrix of , -λ is XX T The eigenvalue matrix of

[0035] S302: Calculate the sample covariance matrix XX T The eigenvector corresponding to the maximum eigenvalue is recorded as the scratch direction;

[0036] S303: Calculate sample center As the mean point, the leftmost binary scratch pixel in the scratch direction is calculated from the scratch direction as the starting point, and the mean point and the starting point are used as the region growth seed points.

[0037] The specific operation steps of step S4 include:

[0038] S401: Adaptively perform threshold segmentation on the scratch prediction frame image, and use the grayscale mean of the neighborhood window centered on the pixel as the segmentation threshold to reduce the influence of uneven lighting and noise on the segmentation, and obtain the target scratch pixel point set A;

[0039] S402: Perform 8-neighborhood region growth on the points in the point set, and consider whether the 8-neighborhood pixels P of the seed point meet the growth criterion R g (P):

[0040]

[0041] If R g (P) = 1, then P is pushed into the seed point stack and recorded in the growth point set S, and the growth criteria are determined for all pixels in the stack until the stack is empty. The growth point set S is the segmented scratch pixel;

[0042] S403: Restoring the segmented scratch pixels to the positions of the original image prediction boxes to obtain the scratch segmentation results.

[0043] The advantages of the present invention compared with the prior art are:

[0044] (1) The present invention combines a deep learning network with an image segmentation algorithm to analyze and segment scratches from qualitative to quantitative perspectives, which not only utilizes the strong target recognition capability of deep learning and suppresses the interference of background noise, but also retains the accuracy of scratch segmentation based on the image segmentation algorithm designed by hand. The present invention is compared with the existing traditional segmentation technology and the current mainstream semantic segmentation network, including the Ostu threshold segmentation method, the Sobel edge detection method, the Gabor filter method, and the semantic segmentation networks such as U-Net, PSPNet, and DeepLabv3. The experimental results on the two data sets are shown in Tables 1 and 2, respectively, where Precision and Recall measure the proportion of correctly predicted scratch pixels in the predicted image and the annotated image, respectively. The two need to be comprehensively analyzed to reflect the overall segmentation effect. Both mIoU and mPA comprehensively describe the predicted image and the annotated image, and comprehensively measure the segmentation accuracy.

[0045] Table 1 Comparison of scratch segmentation results of electronic commutator scratch dataset

[0046]

[0047] Table 2 Comparison of scratch segmentation results of weld X-ray image dataset

[0048]

[0049] Among the segmentation methods based on digital image processing, the Ostu threshold segmentation method is easily disturbed by non-interested areas or non-scratch defects, resulting in low detection accuracy. The Sobel edge detection method has strong adaptability to changing brightness and noise, and is better than the Ostu threshold segmentation method in eliminating noise interference, but it still cannot filter out certain background noise, and the scratch edge positioning is inaccurate, and the accuracy is lower than the method in this paper. In the electronic commutator scratch dataset, the Gabor filter method has a good detection effect due to the high contrast between the background and the scratches, but in the images of the weld X-ray image dataset, the image edges are significantly disturbed by illumination changes, and the scratches cannot be effectively extracted from the interference.

[0050] In the semantic segmentation method based on deep learning, U-Net sets a large number of channels in the upsampling process, effectively utilizing the contextual information of the high-resolution feature layer, so it has a better detection effect on subtle scratches. PSPNet and DeepLabv3 have a larger receptive field and are more suitable for tasks such as scene parsing that focus on global information, but they are inaccurate in extracting local information of scratches.

[0051] The present invention only performs image segmentation on the prediction frame image and then restores the prediction frame image to the original image position, thereby effectively avoiding background information interference and accurately extracting scratch details, which reflects the advantage of the idea of ​​combining target detection with image segmentation in the present invention.

[0052] (2) In order to effectively identify scratches, a scratch recognition network is designed. On the basis of the SSD network, a multi-feature fusion module designed according to the scratch morphological characteristics is added. Under the premise of ensuring a faster detection speed, the accuracy is higher than the current mainstream target recognition method. The scratch recognition network in the present invention is compared with other mainstream target recognition methods, including two-stage Faster R-CNN and R-FCN, and one-stage YOLOv4, SSD, and RetinaNet. The experimental results on the two data sets are shown in Tables 3 and 4, respectively.

[0053] Table 3 Comparison of scratch recognition results of electronic commutator scratch dataset

[0054]

[0055] Table 4 Comparison of scratch segmentation results of weld X-ray image dataset

[0056]

[0057] In the two-stage method, although the detection accuracy of Faster R-CNN is not much different from that of the present invention, the detection time is 2-5.5 times that of the method of the present invention. R-FCN is an improvement on Faster R-CNN. On the scratch dataset in this experiment, although the detection speed of R-FCN is improved compared with Faster R-CNN, the accuracy is reduced, and both the speed and accuracy are lower than the present invention. The detection speed of the one-stage method is faster than that of the two-stage method. Among them, the detection speed of YOLOv4 and RetinaNet is not much different from that of the present invention, but the accuracy is lower than that of the present invention. SSD is an improved prototype of the present invention. Due to the increase in the complexity of the network structure, the detection speed is slightly reduced, but the accuracy is significantly improved. On the whole, compared with the mainstream target detection network, the present invention has the highest detection accuracy while ensuring a faster detection speed.

[0058] (3) A principal component growing segmentation algorithm is designed for the extracted scratch prediction frame, which effectively suppresses background noise while ensuring the integrity and accuracy of scratch extraction. The principal component growing segmentation algorithm in the present invention is compared with the existing traditional segmentation technology and the current mainstream semantic segmentation network, including Ostu threshold segmentation method, Sobel edge detection method, Gabor filter method, and semantic segmentation networks such as U-Net, PSPNet, and DeepLabv3. The experimental results on the two datasets are shown in Tables 5 and 6, respectively.

[0059] Table 5 Comparison of scratch prediction box segmentation results of electronic commutator scratch dataset

[0060]

[0061] Table 6 Comparison of scratch segmentation results of weld X-ray image dataset

[0062]

[0063] Since the scratch images in the weld X-ray image dataset are presented by x-rays, the contrast of the scratches relative to the background is low, and the amplitude of illumination variation is large, the segmentation difficulty is greater than that of the electronic commutator scratch dataset. The segmentation algorithm based on digital image processing is sensitive to illumination changes and has difficulty in suppressing background noise, especially for images with obviously uneven brightness in the weld X-ray image dataset. The detection effect is poor, and its quantitative indicators mIoU and mPA are lower than those of the present invention. The semantic segmentation method based on deep learning has better background suppression effect than the segmentation algorithm based on digital image processing because it marks and trains scratches pixel by pixel, but the segmentation accuracy and completeness of some low-contrast scratches and subtle scratches are insufficient, and its quantitative indicators mIoU and mPA are lower than those of the present invention. The principal component growth segmentation algorithm in the present invention uses the principal component points of the scratch prediction box image to perform regional growth, which effectively eliminates the interference of background noise while ensuring the integrity of scratch extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a flow chart of the scratch detection method based on deep convolutional neural network and image segmentation of the present invention;

[0065] Figure 2 This is a diagram of the scratch recognition and positioning network structure in the present invention;

[0066] Figure 3 This is a structural diagram of the multi-feature fusion module in the present invention;

[0067] Figure 4 This is a flow chart of the principal component growth segmentation algorithm in the present invention. DETAILED DESCRIPTION

[0068] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0069] like Figure 1 As shown, the scratch detection method based on deep convolutional neural network and image segmentation of the present invention has the following specific steps:

[0070] S1: Scratch recognition and positioning network structure diagram Figure 2 As shown in FIG. 1 , according to the morphological characteristics of scratches, a multi-feature fusion module consisting of an upsampling layer, a lateral connection layer and a feature fusion layer is designed. The multi-feature fusion module obtains the same scale as the low-level features through upsampling, and then fuses them through the lateral connection layer. The 6 feature layers used for prediction in the SSD network are respectively input into 6 multi-feature fusion modules. The feature layers output by the multi-feature fusion module are used for prediction in step S2. Figure 3 It is the structure diagram of multi-feature fusion module;

[0071] Specifically, S101: the upsampling layer in the multi-feature fusion module performs a deconvolution operation on the last feature layer used for prediction in the SSD network and the 2nd, 3rd, 4th, and 5th feature layers in the fusion layer, and expands the size of the last feature layer used for prediction and the 2nd, 3rd, 4th, and 5th feature layers in the fusion layer to twice the original size;

[0072] S102: The lateral connection layer in the multi-feature fusion module performs convolution operations on the first five feature layers used for prediction in the SSD network, so that the number of channels of the first five feature layers used for prediction in the SSD network is reduced to 1 / 2 of the original number, which is consistent with the number of channels of the upsampled feature map to meet the fusion condition;

[0073] S103: The fusion layer in the multi-feature fusion module performs tensor calculation on the feature maps of the same size after upsampling and horizontal connection, and obtains 5 feature layers for prediction containing deep and shallow feature information. The last feature layer used for prediction in the SSD network is still retained as the sixth feature layer for prediction. A multi-feature fusion module designed according to the scratch morphological characteristics is added to the SSD network, which improves the detection ability of the SSD network for small-sized and low-contrast scratches.

[0074] S2: Construct the loss function of category and position, use the loss function of category and position to perform regression analysis on the feature layer output by the multi-feature fusion module, identify scratches based on the category loss function, calculate the coordinates of the scratch prediction box, and output the scratch prediction box image.

[0075] Specifically, S201: Calculate the category and position loss functions, and extract and filter the priori boxes according to the loss functions. The calculation formulas of the category loss function and the position loss function are:

[0076]

[0077]

[0078] Among them, in the category loss function, Indicates whether the i-th prior frame matches the real object frame of the j-th target of the p-th feature. If A value of 1 indicates a match, and a value of 0 indicates a mismatch. represents the i-th prior box and the p-th feature, Indicates that the i-th prior frame does not match any feature, N is the number of scratch real object frames that match the prior frame, pos is the positive sample prior frame, and neg is the negative sample prior frame. In the position loss function, is the deviation between the coordinates of the i-th prior box and the coordinates of the v-th prior box, represents the deviation between the jth scratch real object box and the vth prior box, SmoothL1 is the L1 regularization function, c x ,c y ,w,h are the center coordinates, width and height of the prior box respectively.

[0079] S202: Decode the scratch prediction frame and its coordinate position according to the center coordinates, width and height of the selected prior frame. Suppose the width and height of the input image and the scratch annotation object frame are w i ,h i ,w o ,h o , then the scratch prediction box coordinates are

[0080]

[0081] S203: Cropping the input image to obtain a scratch prediction frame image.

[0082] S3: The flow chart of the principal component growth segmentation algorithm in the present invention is as follows Figure 4 As shown, the scratch direction in the scratch prediction frame image is calculated using the principal component analysis method, and the mean point and the starting point are calculated from the scratch direction as the region growth seed point;

[0083] Specifically, S301: Binarize the scratch prediction frame image, and set the scratch pixel after binarization to be X=(x (1) ,x (2) ,...,x (n) ), each pixel is represented according to its 2D coordinates as

[0084]

[0085] In order to predict the scratch direction, the 2D data is mapped to a 1D space. Let the projection transformation matrix of the mapping be W, and the resulting projection is:

[0086] z (i) =W T x (i) ,i=1,2,...,n (5)

[0087] In order to preserve as much information as possible from the original data, the variance and Maximum, that is, maximize the trace of the sum of the variances of the projected data:

[0088]

[0089] Where I is the identity matrix, and the Lagrange multiplier method is used to solve it. The Lagrange function is assumed to be:

[0090] J(W)=tr(W T XX T W+λ(WT WI)) (7)

[0091] Among them, λ is the Lagrangian factor, and the derivative of W is obtained:

[0092] XX T W=(-λ)W (8)

[0093] It can be seen from formula (8) that W is the covariance matrix XX T The eigenvector matrix of , -λ is XX T The eigenvalue matrix of .

[0094] S302: Calculate the sample covariance matrix XX T The eigenvector corresponding to the maximum eigenvalue is recorded as the scratch direction;

[0095] S303: Calculate sample center As the mean point, the leftmost binary scratch pixel in the scratch direction is calculated from the scratch direction as the starting point, and the mean point and the starting point are used as the region growth seed points.

[0096] S4: Perform 8-neighborhood region growth on the pixel points in the scratch prediction frame image after adaptive threshold segmentation, restore the growth point set to the original image prediction frame position, and obtain the scratch segmentation result.

[0097] Specifically, S401: performing adaptive threshold segmentation on the scratch prediction frame image, taking the grayscale mean of the neighborhood window centered on the pixel as the segmentation threshold to reduce the influence of uneven lighting and noise on the segmentation, and obtaining the target scratch pixel point set A;

[0098] S402: Perform 8-neighborhood region growth on the points in the point set, and consider whether the 8-neighborhood pixels P of the seed point meet the growth criterion R g (P):

[0099]

[0100] If R g (P) = 1, then P is pushed into the seed point stack and recorded in the growth point set S, and the growth criterion is determined for all pixels in the stack until the stack is empty. The growth point set S is the segmented scratch pixels.

[0101] S403: Restoring the segmented scratch pixels to the positions of the original image prediction boxes to obtain the scratch segmentation results.

[0102] The application of the present invention is as follows: experiments are conducted on electronic commutator scratch datasets and weld X-ray image datasets, using the PyTorch deep learning framework, under the Ubuntu 16.04 Linux operating system, using a 3.40GHz Intel Corei7-6800K CPU, 8GB memory GeForce GTX 1080Ti GPU. In order to be more suitable for practical applications, data enhancement operations such as flipping, rotating, scaling, cropping, translating, and adding noise are performed to improve the generalization ability and robustness of the model.

Claims

1. A scratch detection method based on deep convolutional neural network and image segmentation, Features: The following steps are involved: S1: According to the morphological characteristics of scratches, a multi-feature fusion module consisting of an upsampling layer, a lateral connection layer and a feature fusion layer is designed. The multi-feature fusion module upsamples high-level features to obtain the same scale as low-level features, and then fuses them through the lateral connection layer. The six feature layers used for prediction in the single-shot multi-box detector SSD network are respectively input into the multi-feature fusion module; S2: Construct the loss function of category and position, use the loss function of category and position to perform regression analysis on the feature layer output by the multi-feature fusion module, identify scratches based on the loss function, calculate the coordinates of the scratch prediction box, and output the scratch prediction box image; The specific steps of step S2 include: S201: Calculate the category and position loss functions, and extract and filter the priori boxes according to the loss functions. The calculation formulas for the category loss function and the position loss function are: (1) (2) Among them, in the category loss function, Indicates The prior box and The first characteristic Whether the real object frame of the target matches or not, if A value of 1 indicates a match, and a value of 0 indicates a mismatch. Indicates The prior box and Feature matching, Indicates A priori boxes do not match any features. is the number of matches between the scratch real object box and the prior box, is the positive sample prior frame, is the negative sample prior frame; in the position loss function, It is The coordinates of the prior box and the The deviation between the coordinates of the prior boxes, Indicates The scratched real object frame and The deviation between the prior boxes, is the L1 regularization function, They are the center coordinates, width and height of the prior box respectively; S202: Decode the scratch prediction frame and its coordinate position according to the center coordinates, width and height of the selected prior frame. Suppose the width and height of the input image and the scratch annotation object frame are , then the scratch prediction box coordinates are: (3) S203: cropping the input image to obtain a scratch prediction frame image; S3: Use the principal component analysis method to calculate the scratch direction in the scratch prediction frame image, and calculate the mean point and the starting point from the scratch direction as the region growth seed point; S4: Adaptively threshold segment the scratch prediction frame image, perform 8-neighborhood region growth on the pixel points in the scratch prediction frame image after adaptive threshold segmentation, restore the growth point set to the original image prediction frame position, and obtain the scratch segmentation result.

2. The scratch detection method based on deep convolutional neural network and image segmentation according to claim 1, Features: The specific operation steps of step S1 include: S101: The upsampling layer in the multi-feature fusion module performs deconvolution operations on the last feature layer used for prediction in the SSD network and the 2nd, 3rd, 4th, and 5th feature layers in the fusion layer, and expands the size of the last feature layer used for prediction and the 2nd, 3rd, 4th, and 5th feature layers in the fusion layer to twice the original size; S102: The lateral connection layer in the multi-feature fusion module performs convolution operations on the first five feature layers used for prediction in the SSD network, so that the number of channels of the first five feature layers used for prediction in the SSD network is reduced to 1 / 2 of the original number, which is consistent with the number of channels of the upsampled feature map to meet the fusion condition; S103: The fusion layer in the multi-feature fusion module performs tensor calculation on the feature maps of the same size after upsampling and horizontal connection, and obtains 5 feature layers for prediction containing deep and shallow feature information. The last feature layer used for prediction in the SSD network is retained as the 6th feature layer for prediction, and the 6 feature layers output from the multi-feature fusion module are used for prediction in step S2.

3. The scratch detection method based on deep convolutional neural network and image segmentation according to claim 1, Features: The specific steps of step S3 include: S301: Binarize the scratch prediction frame image, and set the scratch pixel after binarization as , each pixel is represented by its 2D coordinates: (4) In order to predict the scratch direction, the 2D data is mapped to 1D space, and the projection transformation matrix of the mapping is , then the resulting projection is: (5) In order to preserve as much information as possible from the original data, the variance and Maximum, that is, maximize the trace of the sum of the variances of the projected data: (6) in is the unit matrix, and the Lagrange multiplier method is used to solve it. Assume that the Lagrange function is: (7) in, is the Lagrangian factor, The derivative is: (8) is the covariance matrix The eigenvector matrix of for The eigenvalue matrix of S302: Calculate sample covariance matrix The eigenvector corresponding to the maximum eigenvalue is recorded as the scratch direction; S303: Calculate sample center As the mean point, the leftmost binary scratch pixel in the scratch direction is calculated from the scratch direction as the starting point, and the mean point and the starting point are used as the region growth seed points.

4. The scratch detection method based on deep convolutional neural network and image segmentation according to claim 1, Features: The specific steps of step S4 include: S401: Adaptively perform threshold segmentation on the scratch prediction frame image, and use the grayscale mean of the neighborhood window centered on the pixel as the segmentation threshold to reduce the impact of uneven lighting and noise on segmentation, and obtain the target scratch pixel point set. ; S402: Target scratch pixel point set The eight-neighborhood region growth is performed on the points in the Whether the growth criteria are met : (9) like , then Push into seed point stack and record into growth point set , the growth criteria are determined for all pixels in the stack until the stack is empty, and the growth point set That is the segmented scratch pixel; S403: Restoring the segmented scratch pixels to the positions of the original image prediction boxes to obtain the scratch segmentation results.

Images