A defect detection system and method for detecting arbitrary scale distributions

Abstract

The application discloses a kind of defect detection system and detection method for detecting arbitrary scale distribution, which can be carried on special integrated chip for carrying out defect detection to cloth, and the detection system specifically includes: image feature extraction module, scale feature extraction module and defect detection module.The defect detection module specifically includes: defect position detection module, defect frame detection module.The detection method reduces the number of hyperparameters, while reducing the imbalance problem of positive and negative samples in the training process, so that the generalization ability of the model is stronger, the defect detection accuracy is higher, and the experimental results show that the method proposed in the application has better performance than the existing detection method.

Description

Technical Field

[0001] The present invention includes a defect detection system and method for detecting defects of arbitrary scale distribution. Background Technology

[0002] A defect is a localized area on an image that does not match the surrounding information. Such defective areas usually disrupt the consistency of the image and should be avoided as much as possible. For example, in fabrics, the number of defects and the size of the defective areas directly affect the quality grade of the fabric. Therefore, it is necessary to detect defects in the produced fabrics on a fabric inspection machine to reject fabrics containing defects.

[0003] Detecting defects distributed at arbitrary scales is quite challenging. Arbitrary scale distribution means that the aspect ratio and area distribution of different defects are not limited.

[0004] Current detection algorithms can be broadly categorized into anchor-based and anchor-free types. Anchor-based algorithms include SSD, YOLOv2, YOLOv3, and R-CNN. These algorithms require pre-setting the anchor size, which is detrimental to detecting defects of arbitrary scales. This is because during training, when encountering targets with extreme sizes and aspect ratios, anchor-based algorithms face increased difficulty in bounding box regression. Furthermore, when using these algorithms for defect detection, the uncertainty of defect scale means that pre-set anchors weaken the algorithm's generalization ability, thus reducing detection accuracy.

[0005] Anchor-free detection algorithms can be used to detect defects distributed at any scale, but existing algorithms have their own defects and limitations, such as low recall and complex processing. For example, YOLOv1 (Redmon J, Divvala S, Girshick R, et al. You Only Look Once: Unified, Real-Time Object Detection[J]. Computer Vision & Pattern Recognition, 2016.) only uses points near the center of the target to predict the bounding box, resulting in low accuracy and low recall for detecting defective points; CornerNet (Law H, Deng J. CornerNet: Detecting Objects as Paired Keypoints[J]. International Journal of Computer Vision, 2020, 128(3): 642-656.) uses a pair of corner points of the target box to predict the bounding box, but requires complex post-processing steps; the detection algorithm based on DenseBox (Huang L, Yang Y, Deng Y, et al. DenseBox: Unifying Landmark Localization with End to End Object Detection[J]. Computer Science, 2015.) is difficult to handle overlapping target boxes, has long computation time and low recall. Summary of the Invention

[0006] The purpose of this invention is to provide a defect detection system and method with high accuracy and strong model generalization ability for detecting defects of arbitrary scale distribution.

[0007] This invention discloses a defect detection system with arbitrary scale distribution. The defect detection system is used to detect defects with arbitrary scale distribution, which means that the aspect ratio and area distribution of different defects are not limited. The detection system includes: an image feature extraction module, a scale feature extraction module, and a defect detection module.

[0008] The image feature extraction module inputs the image to be detected into the support network after preprocessing to extract features, and obtains a feature map containing all information.

[0009] The scale feature extraction module performs scale feature extraction on the feature map to obtain corresponding scale feature maps in different scale dimensions. The scale dimension is the scale reduction ratio of the scale feature map relative to the feature map, and the scale feature map contains scale information.

[0010] The defect detection module performs defect location detection and defect border detection on the scale feature map to obtain the defect location information and border information.

[0011] Preferably, the preprocessing operations include one or more of the following: image digitization, image labeling, image smoothing and enhancement, image cropping, and data normalization; the backbone network is an existing feature extraction network model, or a feature extraction network model modified from an existing feature extraction network model; the feature map is defined as F∈R H×W H and W represent the height and width of the feature map F, respectively, and R is a real number. H×W This indicates that there are H×W real values ​​in F, where H and W are both positive integers.

[0012] Preferably, the scale feature extraction step includes:

[0013] S31: Define the scale dimension S i =2 i+2 (i = 1, 2, 3, ..., n), S i The corresponding scale feature map is A. i (i = 1, 2, 3, ..., n), where n is a positive integer;

[0014] S32: Based on the feature map F and the scale dimension S i The scale feature map can be obtained.

[0015]

[0016] in Indicates a step size of S i A convolution operation with a kernel size of 1×1. This represents a convolution operation with a stride of 2 and a kernel size of 2×2. Indicates to Rounding down, Indicates to Rounding;

[0017] Preferably, the defect detection module includes a defect location detection module and a defect border detection module;

[0018] The defect location detection module obtains a location feature map after the scale feature map is convolved by one layer. Based on each location feature map, the module performs defect location detection on the image to be detected. The detection results of all location feature maps are fused to obtain the location information of the defects contained in the image to be detected.

[0019] The defect border detection module obtains a border feature map by performing four convolutions on the scale feature map. Based on each border feature map, it performs defect border detection on the image to be detected. The detection results of all border feature maps are fused and a non-maximum suppression method is used to obtain the border information of the defects.

[0020] The essence of the Non-maximum suppression (NMS) algorithm is to search for local maxima and suppress non-maximum elements. In defect detection, it can be used to suppress redundant detection bounding boxes. The algorithm process is as follows: (1) Sort the scores of all boxes and select the highest score box and its corresponding box; (2) Traverse the remaining boxes. If the overlap area with the current highest score box is greater than a certain threshold, delete the box; (3) Select the highest score box from the unprocessed boxes and repeat the above process.

[0021] Preferably, the defect location detection module acquires the defect location information by including:

[0022] S51: For the scale feature map A i The location feature map is obtained after one convolution operation. The convolution kernel size is 1×1, i=1,2,3,…,n;

[0023] S52: Define the location feature map O i The position of each feature point on the x-axis is (x-x-y). i ,y i ), The value of the location feature point is defined as p(x) i ,y i );

[0024] S53: Define the position of each feature point on the feature map F as (u,v), u∈{0,1,2…W}, v∈{0,1,2…H}, and define the value of the feature point as p(u,v). i The position of each feature point on the map is mapped to F:

[0025]

[0026] S54: If p(x) i ,y iIf ) > ε, where ε is the detection threshold, then it indicates that the location feature point (x) is within the range of ε. i ,y i The corresponding feature point (u,v) i Located within the defect region, such that p(u,v) i =1, otherwise p(u,v) i =0;

[0027] S55: For all the aforementioned location feature maps O i (i = 1, 2, 3, ..., n) Perform steps S42 to S44, and fuse the results to obtain the final defect location detection result, i.e.:

[0028] p(u,v)=max i∈1～n p(u,v) i

[0029] If p(u,v) = 1, it means that the feature point (u,v) is located within the defect area; if p(u,v) = 0, it means that the feature point (u,v) is located outside the defect area. Thus, the location information of the defects contained in the image to be detected can be obtained.

[0030] Preferably, the defect border detection module performs defect border detection by including the following steps:

[0031] S61: For the scale feature map A i The bounding box feature map is obtained after four layers of convolution operations. The kernel size is 1×1, and i = 1, 2, 3, ..., n;

[0032] S62: Due to the border feature map L i and the location feature map O i The height and width are the same, use (x i ,y i ) to represent L i The position of the bounding box feature points, and the value of the bounding box feature points is a four-dimensional vector l(x). i ,y i )=[l ai ,l bi ,l ci ,l di ], by steps

[0033] From S53, we can obtain (x) i ,y i The feature points mapped onto the feature map F are (u,v). i Then the l ai ,l bi ,l ci ,l diLet (u, v) represent feature points respectively. i The distance to the left, top, right, and bottom borders of the defect;

[0034] S63: If 8S i-1 <max[l ai ,l bi ,l ci ,l di ]<8S i (i=1 hour 8S i-1 If replaced with 0), then for feature points (u,v) located within the defect area... i Perform border detection:

[0035]

[0036] S64: For all the aforementioned location feature maps O i (i = 1, 2, 3, ..., n) Perform steps S62 to S63 as described above, and fuse the results to obtain the final defect border detection result, i.e.:

[0037]

[0038] Here, Set{l(u,v)} represents the set of l(u,v), where l(u,v) is a four-dimensional vector used to represent the defect border. This indicates that we iterate through all i∈1~n and select all non-zero l(u,v). i NMS[·] is a nonmaximum suppression method.

[0039] This invention also discloses a defect detection method for arbitrary scale distribution, the defect detection method comprising the following steps:

[0040] S71: After preprocessing, the image to be detected is input into the pillar network for feature extraction, resulting in a feature extraction algorithm that includes all features.

[0041] Feature map F of the part information;

[0042] S72: At different scales, dimension S i Extract the corresponding scale feature map A from the top. i Scale feature map A i Including scale information, the specific operation is as follows:

[0043]

[0044] in Indicates a step size of S i A convolution operation with a kernel size of 1×1. This represents a convolution operation with a stride of 2 and a kernel size of 2×2;

[0045] S73: Calculate the scale feature map A obtained in step S72. i The location feature map O is obtained after one convolution layer. i Based on each location feature map O i The image to be detected is subjected to defect location detection, and the detection results of all location feature maps are fused to obtain the location information of the defects contained in the image to be detected.

[0046] S74: Calculate the scale feature map A obtained in step S72. i After four convolutional layers, the bounding box feature map L is obtained. i Based on each bounding box feature map L i The image to be detected is subjected to defect border detection. The detection results of all border feature maps are fused and the border information of the defects is obtained by using a non-maximum suppression method.

[0047] Preferably, the preprocessing operations in step S71 include image digitization, image labeling, image smoothing and enhancement, image cropping, data normalization, etc.; the pillar network is an existing feature extraction network model, or a feature extraction network model modified on the existing feature extraction network model.

[0048] Preferably, the scale dimension S in step S72 i It is scale feature map A i The scaling factor relative to the feature map F is given by the formula S. i =2 i+2 (i = 1, 2, 3, ..., n), where n is a positive integer and can be set according to the actual situation.

[0049] Preferably, the step S73 based on each location feature map O i The steps for detecting defect locations and fusing the detection results include:

[0050] S91: Transfer the location feature map O i The position (x) of each location feature point on the map i ,y i All of these are mapped onto F, and the positions of the corresponding feature points on F are obtained as (u,v). i ;

[0051] S92: When the location feature point (x) i ,y i The value of p(x) i ,y i When the value is greater than the detection threshold ε, it indicates that the location feature point (x) is... i ,y i The corresponding feature point (u,v) iLocated within the defect region, such that p(u,v) i =1, otherwise p(u,v) i =0;

[0052] S93: For all location feature maps O i Repeat steps S91 and S92, and then fuse all the detection results using the following formula to obtain the location information of the defect:

[0053] p(u,v)=max i∈1～n p(u,v) i

[0054] Where p(u,v)=1 indicates that the feature point (u,v) is located within the defect area, and p(u,v)=0 indicates that the feature point (u,v) is located outside the defect area.

[0055] Preferably, the step S74 based on each bounding box feature map L i The steps for performing defect border detection and fusing the detection results include:

[0056] S101: Transfer the bounding box feature map L i The position of each bounding box feature point (x) i ,y i All of these are mapped onto F, and the positions of the corresponding feature points on F are obtained as (u,v). i Border feature points (x) i ,y i The value of ) is l(x i ,y i )=[l ai ,l bi ,l ci ,l di ], l ai ,l bi ,l ci ,l di Let (u, v) represent feature points respectively. i The distance to the left, top, right, and bottom borders of the defect;

[0057] S102: If 8S i-1 <max[l ai ,l bi ,l ci ,l bi ]<8S i (i=1 hour 8S i-1 If replaced with 0), then for feature points (u,v) located within the defect area... i Perform border detection;

[0058] S103: For all bounding box feature maps Li Repeat steps S101 and S102, and then fuse all the detection results using the following formula to obtain the border information of the defects:

[0059]

[0060] Here, Set{l(u,v)} represents the set of l(u,v), where l(u,v) is a four-dimensional vector used to represent the defect border. This indicates that we iterate through all i∈1~n and select all non-zero l(u,v). i NMS[·] is a nonmaximum suppression method.

[0061] Beneficial effects:

[0062] 1. This invention combines positional features and border features to detect defects, thereby improving the recall rate of the detection results;

[0063] 2. This invention uses a four-dimensional vector to represent the defect border, avoiding subsequent complex calculations. The parallel defect location detection and border detection improve the algorithm's running speed and achieve higher performance in real-time systems.

[0064] 3. This invention has high accuracy in detecting defects of arbitrary scale and strong model generalization ability. Attached Figure Description

[0065] Figure 1 A diagram illustrating methods for augmenting fabric defect detection datasets.

[0066] Figure 2 This is a structural diagram of the defect detection system proposed in this invention.

[0067] Figure 3 This is a schematic diagram of the defect detection method proposed in this invention. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0069] Clustering was performed on the collected fabric defect dataset to obtain the overall characteristics of defect shapes, as shown in Table 1. Table 1 reveals that the defect shape distribution in this dataset exhibits significant scale variations: the aspect ratio varies greatly, with the ratio of the longer side to the shorter side of the defect region border reaching a maximum of 9.4 and a minimum of 1, a difference of nearly 10 times; the area distribution also varies greatly, with a maximum area of ​​636,779 pixels and a minimum of only 9,191 pixels, a difference of nearly 70 times. This dataset is suitable for defect detection using the method of this invention.

[0070]

[0071] Table 1

[0072] The dataset was used to detect defects in fabrics with arbitrary scale distributions. A total of 1168 defect images were collected, with a resolution of 2560x1920. The dataset was divided into a training set (998 images) and a validation set (170 images). The dataset labels include the location of the fabric defect and its border, described by a rectangular box. The label format is [x...]. min ,y min ,x max ,y max ],(x min ,y min (x) represents the coordinates of the top left corner of the defect's border. max ,y max () indicates the coordinates of the lower right corner of the defect's border.

[0073] Firstly, the dataset needs to be augmented for several reasons: The number of defect images collected is insufficient, while deep learning models require a large amount of image data for parameter training; therefore, dataset augmentation is necessary. Secondly, the original image resolution of 2560x1920 is too high, requiring significant GPU memory for training. Thirdly, defect areas occupy very small pixels; downsampling the original image would further shrink the defects, making them harder to detect. Therefore, appropriate methods are needed to augment the defect data. The specific methods are as follows:

[0074] like Figure 1 As shown, a sub-image with a resolution of 1280x960 is cropped. The location of the sub-image is random, but each sub-image must contain defects. Simultaneously, the border labels of the defects in the original image are processed, and new labels are generated in the sub-image. The method for generating the new labels is as follows: Let the coordinates of the original image's position be [x...]. min ,y min ,x max ,y max Let the coordinates of the top-left corner of the cropped sub-image in the original image be (x0, y0). Then the border label of the defect in the sub-image is: [max(x0, y0)]min -x 0, 0), max(y min -y 0, 0),min(x max -x 0, 1280),min(y max After expansion, the training set contained 3310 images, and the test set contained 525 images. To enhance the network's generalization ability and reduce overfitting, random online augmentation was performed on the data during training, including horizontal and vertical mirroring.

[0075] like Figure 2 As shown, this invention proposes a system for detecting defects distributed at arbitrary scales. The defect detection system can be used to detect defects distributed at arbitrary scales, where the arbitrary scale distribution means that the length, width, and area of ​​the defects are unlimited. The detection system includes: an image feature extraction module, a scale feature extraction module, and a defect detection module.

[0076] The image feature extraction module inputs the image to be detected into the support network after preprocessing to extract features, and obtains a feature map containing all information.

[0077] The scale feature extraction module performs scale feature extraction on the feature map to obtain corresponding scale feature maps in different scale dimensions. The scale dimension is the scale reduction ratio of the scale feature map relative to the feature map, and the scale feature map contains scale information.

[0078] The defect detection module performs defect location detection and defect border detection on the scale feature map to obtain the defect location information and border size.

[0079] The detection system of this invention uses a ResNet50 network as its backbone network. After passing through the feature extraction module, a feature map F is obtained, where F∈R. 800×1024 In the scale feature extraction module, with n=5, the scale dimension set is: {S1=8, S2=16, S3=32, S4=64, S5=128}. The scale feature map A1∈R is then calculated using the formula. 100×128 A2∈R 50×64 A3∈R 25×32 A4∈R 13×16 A5∈R 7×8 The five scale feature maps are used as inputs to the defect detection module.

[0080] In the defect location detection module, the five scale feature maps are each convolved by a 1×1 kernel to obtain five location feature maps O1∈R. 100×128×1 O2∈R 50×64×1O3∈R 25×32×1 O4∈R 13×16×1 O5∈R 7×8×1 . For each positional feature map, taking O1 as an example, the position of a feature point on O1 can be represented as (x1, y1), x1∈{0,1,2…128}, y1∈{0,1,2…100}, and the value of the feature point can be represented as p(x1, y1). Mapping the feature points on O1 to F, we can obtain the position representation of the corresponding feature points on F as (u, v)1 = (8x1+4, 8y1+4). Setting the detection threshold to 0.1, when p(x1, y1) > 0.1, p(u, v)1 = p(8x1+4, 8y1+4)1 = 1, and when p(x1, y1) ≤ 0.1, p(u, v)1 = p(8x1+4, 8y1+4)1 = 0. Calculating for each feature point on O1 completes the defect location detection for positional feature map O1. Repeating the above steps for each positional feature map yields p(u, v). i For i = 1, 2, 3, 4, 5, the formula p(u,v) = max i∈1～5 p(u,v) i By integrating the detection results of the feature maps at different locations, p(u,v) = 1 indicates that the feature point (u,v) is located within the defect area, and p(u,v) = 0 indicates that the feature point (u,v) is located outside the defect area, thus obtaining the location information of the defect.

[0081] In the defect border detection module, the five scale feature maps are each convolved with four layers of 1×1 kernels to obtain five border feature maps L1∈R. 100×128×4 L2∈R 50×64×4 L3∈R 25×32×4 L4∈R 13×16×4 L5∈R 7×8×4 Taking L1 as an example, the position of the bounding box feature point on L1 can still be represented as (x1, y1), x1∈{0,1,2…128}, y1∈{0,1,2…100}, and the value of the bounding box feature point can be represented as l(x1, y1)=[l a1 ,l b1 ,l c1 ,l d1 Mapping the bounding box feature points on L1 to F, we can obtain the position representation of the corresponding feature points on F: (u,v)1 = (8x1 + 4, 8y1 + 4). a1 ,l b1 ,l c1 ,l d1 Let (u, v) represent the distances from feature point (u, v)1 to the left, top, right, and bottom borders of the defect boundary, respectively. Since the border labels in the dataset are [x...] min ,y min,x max ,y max It is represented by ], (x min ,y min (x) represents the coordinates of the top left corner of the defect's border. max ,y max () represents the coordinates of the bottom right corner of the defect's border. After conversion, x can be obtained. min =ul a1 ,y min =vl b1 ,x max =u+l c1 ,y max =v+l d1 When 0 < max[l a1 ,l b1 ,l c1 ,l d1 When the value is less than 64, bounding box detection is performed on feature point (u,v)1 located within the defect area, i.e., the value of l(u,v)1 = l(x1,y1) is retained; otherwise, l(u,v)1 is set to 0. This is done so that different bounding box feature maps can detect bounding boxes of different sizes. Repeating the above steps for each bounding box feature map yields l(u,v). i i = 1, 2, 3, 4, 5, using the formula The detection results of each bounding box feature map are fused. Each l(u,v) represents a defect bounding box, containing parameters {u,v,l}. a1 ,l b1 ,l c1 ,l d1 The parameters can be used to determine the border information of the defects.

[0082] like Figure 3 As shown, the present invention also discloses a defect detection method for arbitrary scale distribution, the defect detection method comprising the following steps:

[0083] S71: After preprocessing, the image to be detected is input into the pillar network for feature extraction, resulting in a feature extraction algorithm that includes all features.

[0084] Feature map F of the part information;

[0085] S72: At different scales, dimension S i Extract the corresponding scale feature map A from the top. i Scale feature map A i Including scale information, the specific operation is as follows:

[0086]

[0087] in Indicates a step size of S iA convolution operation with a kernel size of 1×1. This represents a convolution operation with a stride of 2 and a kernel size of 2×2;

[0088] S73: Calculate the scale feature map A obtained in step S72. i The location feature map O is obtained after one convolution layer. i Based on each location feature map O i The image to be detected is subjected to defect location detection, and the detection results of all location feature maps are fused to obtain the location information of the defects contained in the image to be detected.

[0089] S74: Calculate the scale feature map A obtained in step S72. i After four convolutional layers, the bounding box feature map L is obtained. i Based on each bounding box feature map L i The image to be detected is subjected to defect border detection. The detection results of all border feature maps are fused and the border information of the defects is obtained by using a non-maximum suppression method.

[0090] Preferably, the preprocessing operations in step S71 include image digitization, image labeling, image smoothing and enhancement, image cropping, data normalization, etc.; the pillar network is an existing feature extraction network model, or a feature extraction network model modified on the existing feature extraction network model.

[0091] Preferably, the scale dimension S in step S72 i It is scale feature map A i The scaling factor relative to the feature map F is given by the formula S. i =2 i+2 (i = 1, 2, 3, ..., n), where n is a positive integer and can be set according to the actual situation.

[0092] Preferably, the step S73 based on each location feature map O i The steps for detecting defect locations and fusing the detection results include:

[0093] S91: Transfer the location feature map O i The position (x) of each location feature point on the map i ,y i All of these are mapped onto F, and the positions of the corresponding feature points on F are obtained as (u,v). i ;

[0094] S92: When the location feature point (x) i ,y i The value of p(x) i ,y i When the value is greater than the detection threshold ε, it indicates that the location feature point (x) is...i ,y i The corresponding feature point (u,v) i Located within the defect region, such that p(u,v) i =1, otherwise p(u,v) i =0;

[0095] S93: For all location feature maps O i Repeat steps S91 and S92, and then fuse all the detection results using the following formula to obtain the location information of the defect:

[0096] p(u,v)=max i∈1～n p(u,v) i

[0097] Where p(u,v)=1 indicates that the feature point (u,v) is located within the defect area, and p(u,v)=0 indicates that the feature point (u,v) is located outside the defect area.

[0098] Preferably, the step S74 based on each bounding box feature map L i The steps for performing defect border detection and fusing the detection results include:

[0099] S101: Transfer the bounding box feature map L i The position of each bounding box feature point (x) i ,y i All of these are mapped onto F, and the positions of the corresponding feature points on F are obtained as (u,v). i Border feature points (x) i ,y i The value of ) is l(x i ,y i )=[l ai ,l bi ,l ci ,l di ], l ai ,l bi ,l ci ,l di Let (u, v) represent feature points respectively. i The distance to the left, top, right, and bottom borders of the defect;

[0100] S102: If 8S i-1 <max[l ai ,l bi ,l ci ,l di ]<8S i (i=1 hour 8S i-1 If replaced with 0), then for feature points (u,v) located within the defect area... iPerform border detection;

[0101] S103: For all bounding box feature maps L i Repeat steps S101 and S102, and then fuse all the detection results using the following formula to obtain the border information of the defects:

[0102]

[0103] Here, Set{l(u,v)} represents the set of l(u,v), where l(u,v) is a four-dimensional vector used to represent the defect border. This indicates that we iterate through all i∈1~n and select all non-zero l(u,v). i NMS[·] is a nonmaximum suppression method.

[0104] Table 2 shows a comparison of the detection results of the defect detection system for arbitrary scale distribution provided by this invention with other different defect detection methods. Our method achieves an AP (mean accuracy) of 53.10, which is higher than ModifiedYOLOv3 (Jing J, Zhuo D, Zhang H, et al. Fabric defect detection using the improved YOLOv3 model[J]. Journal of Engineered Fibers and Fabrics, 2020.) and FSR (Y. Qin, M. Chen, L. Qi and Y. Sun, "Focus Generator with Score Classification on FabricDefect Detection," 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), Portland, OR, USA, 2019.), achieving the best results in the one-stage network. We also compared it with the two-stage detection network, Faster R-CNN+Soft-NMS (Liu X, Liu Z, Li C, et al. Fabric defect detection based on faster R-CNN[C] / / Ninth International Conference on Graphic and Image). The AP (AP and FPS) of the Processing (2018) method is also lower than our method, but its detection speed is 5 frames per second slower. Therefore, our method has superior detection performance. (AP and FPS: Number of images recognized per second)

[0105]

[0106] Table 2

Claims

1. A defect detection system with arbitrary scale distribution, characterized in that: The defect detection system can be used to detect defects distributed at any scale. The arbitrary scale distribution means that the length, width and area of ​​the defects are unlimited. The detection system includes: an image feature extraction module, a scale feature extraction module and a defect detection module. The image feature extraction module inputs the image to be detected into the pillar network after preprocessing to extract features and obtains a feature map containing all information. The pillar network is an existing feature extraction network model or a feature extraction network model modified on the existing feature extraction network model. The scale feature extraction module extracts scale features from the feature map, and obtains corresponding scale feature maps in different scale dimensions according to the size of the defect. The scale dimension is the scale reduction ratio of the scale feature map relative to the feature map, and the scale feature map contains scale information. The defect detection module performs defect location detection and defect border detection on the scale feature map to obtain the defect location information and border information. The defect detection system can be used to detect defects with large differences in scale distribution. The large differences in scale distribution are manifested in the large differences in aspect ratio and area distribution between different defects. The large differences in aspect ratio distribution indicate that the ratio of the maximum aspect ratio to the minimum aspect ratio of the defect is greater than 8, and the large differences in area distribution indicate that the ratio of the maximum area to the minimum area of ​​the defect is greater than 50. The defect detection module includes a defect location detection module and a defect border detection module. The defect location detection module obtains a location feature map by convolving the scale feature map. Based on each location feature map, it performs defect location detection on the image to be detected and fuses the detection results of all location feature maps to obtain the location information of the defects contained in the image to be detected. The defect border detection module obtains a border feature map after the scale feature map is convolved by four layers. Based on each border feature map, defect border detection is performed on the image to be detected. The detection results of all border feature maps are fused and a non-maximum suppression method is used to obtain the border information of the defect. The defect location detection module acquires the defect location information through the following steps: S51: For the scale feature map The location feature map is obtained after one convolution operation. The feature map is defined as follows: , and The feature maps are respectively represented. The height and width, and the kernel size are , ; S52: Define the location feature map The position of each feature point on the surface is: , , ,in, Indicates to Rounding down, Indicates to Rounding down, the value of the location feature point is defined as... ; S53: Define the feature map The position of each feature point on is The value of the feature point is defined as follows: ,Will The position of each feature point on the map is mapped to superior: S54: If , If the detection threshold is used, it indicates the location feature point. Corresponding feature points Located within the defect area, making ,otherwise ; S55: For all the aforementioned location feature maps , Perform the operations described in steps S52 to S54, and fuse the results to obtain the final defect location detection result, i.e.: like , representing the feature point If located within the defect area, , representing the feature point Located outside the defect area, the location information of the defects contained in the image to be detected can be obtained; The defect border detection module performs defect border detection in the following steps: S61: For the scale feature map The bounding box feature map is obtained after four layers of convolution operations. The kernel size is , ; S62: Due to the border feature map and the location feature map The height and width are the same, using to indicate The positions of the bounding box feature points, and the values ​​of the bounding box feature points are four-dimensional vectors. From step S53, we can obtain that... Mapping to feature map The feature points on are Then the Representing feature points respectively The distance to the left, top, right, and bottom borders of the defect; S63: If hour Replacing it with 0 will affect feature points located within the defect area. Perform border detection: S64: For all the aforementioned location feature maps , Perform the operations described in steps S62 to S63, and fuse the results to obtain the final defect border detection result, i.e.: in, express The set, It is a four-dimensional vector used to represent the border of defects. This indicates traversing all Select all non-zero values. , It is a nonmaximum suppression method.

2. The defect detection system according to claim 1, characterized in that, The preprocessing operations include at least one of the following: image digitization, image labeling, image smoothing and enhancement, image cropping, and data normalization; the feature map is defined as... , and The feature maps are respectively represented. The height and width, where R is a real number. This indicates that there are H×W real values ​​in F. and All are positive integers.

3. The defect detection system according to claim 1, characterized in that, The scale feature extraction steps include: S31: Define the scale dimension , , The corresponding scale feature map is , , It is a positive integer; S32: Based on the feature map and the scale dimension The scale feature map can be obtained. : in Indicates step size is The kernel size is Convolution operation, This indicates a stride of 2 and a kernel size of . Convolution operation, Indicates to Rounding down, Indicates to Round down.

4. A method for detecting defects of arbitrary scale distribution, used with the defect detection system according to any one of claims 1-3, characterized in that, The defect detection method is used to detect defects distributed at any scale, where the length, width, and area of ​​the defects are unlimited. The defect detection method includes the following steps: S71: After preprocessing, the image to be detected is input into the pillar network for feature extraction, resulting in a feature map containing all information. ; S72: At different scale dimensions Extract the corresponding scale feature map. Scale feature map Including scale information, the specific operation is as follows: in Indicates step size is The kernel size is Convolution operation, This indicates a stride of 2 and a kernel size of . Convolution operations; S73: Calculate the scale feature map obtained in step S72. The location feature map is obtained after one convolutional layer. Based on each location feature map Defect location detection is performed on the image to be detected, and the detection results of all location feature maps are fused to obtain the location information of the defects contained in the image to be detected. S74: Calculate the scale feature map obtained in step S72. After four convolutional layers, the bounding box feature map is obtained. Based on each bounding box feature map Defect border detection is performed on the image to be detected. The detection results of all border feature maps are fused and non-maximum suppression method is used to obtain the border information of the defects. The preprocessing operations in step S71 include image digitization, image labeling, image smoothing and enhancement, image cropping, and data normalization; the pillar network is an existing feature extraction network model, or a feature extraction network model modified on the existing feature extraction network model. The defect detection method described above can be used to detect defects with large differences in scale distribution. The large differences in scale distribution are manifested in the large differences in aspect ratio and area distribution among different defects. The large differences in aspect ratio distribution indicate that the ratio of the maximum aspect ratio to the minimum aspect ratio of the defect is greater than 8, and the large differences in area distribution indicate that the ratio of the maximum area to the minimum area of ​​the defect is greater than 50.

Images