A deep learning-based ceramic tile flaw detection method

Abstract

The present application relates to a kind of ceramic tile flaw detection methods based on deep learning, method includes: the image of the ceramic tile to be detected is acquired, image is input into the ceramic tile flaw detection model of training completion, and output detection result, wherein, the training process of ceramic tile flaw detection model is specifically as follows: improved YOLOv7 model is established, YOLOv7 model includes backbone module, Neck module, residual convolution module and head module, training set is input into improved YOLOv7 model, is trained by the loss function that weight fusion IoU function and NWD function are combined, and the ceramic tile flaw detection model of training completion is obtained.Compared with prior art, the present application has the advantages such as improving the precision of target detection model to the detection of the flaw of ceramic tile.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection of ceramic tile flaws, and in particular to a ceramic tile flaw detection method based on deep learning. BACKGROUND

[0002] At present, the quality inspection link of most ceramic tile enterprises is mainly completed by quality inspection workers, and 2-6 quality inspection workers are needed on each production line to perform quality inspection work at the same time. Manual detection is not only time-consuming and labor-intensive, but also requires high employment costs. The present application can help enterprises to realize the automation of the quality inspection link, improve the quality inspection efficiency of products and reduce the production cost of products.

[0003] The current mainstream target detection model such as YOLOv7 can already achieve very high recognition accuracy and speed, but the accuracy is not high when identifying small targets in pictures, such as ceramic tile flaws. There are a large number of small targets in ceramic tile flaws, which leads to low recognition accuracy of the existing target detection model, so it is difficult to put the model into practical application. SUMMARY

[0004] The purpose of the present application is to provide a ceramic tile flaw detection method based on deep learning to improve the accuracy of the target detection model in detecting ceramic tile flaws.

[0005] The purpose of the present application can be achieved by the following technical solutions:

[0006] A ceramic tile flaw detection method based on deep learning, the method comprising:

[0007] Obtaining a ceramic tile image to be detected, inputting the image into a trained ceramic tile flaw detection model, and outputting a detection result, wherein the training process of the ceramic tile flaw detection model is specifically as follows:

[0008] An improved YOLOv7 model is established, the YOLOv7 model comprising a backbone module, a Neck module, a residual convolution module and a head module, a training set is input into the improved YOLOv7 model, and the improved YOLOv7 model is trained through a loss function combining an IoU function and an NWD function by weight fusion, to obtain a trained ceramic tile flaw detection model, wherein the loss function is specifically as follows:

[0009] loss=(1-ration)*(1-NWD)+ration*(1-IoU)

[0010] Wherein, ration is the proportion of small targets in the data set, NWD represents the normalized Whistans distance function, and IoU represents the intersection over union function.

[0011] Further, after the training set input improves the YOLOv7 model, the training set first enters the backbone module, and after entering the backbone module, it first passes through multiple CBS modules and efficient aggregation modules to obtain preliminary features. The preliminary features are input into the first downsampling module and the first efficient aggregation module. The output features are used as the first input features of the Neck module. The first input features of the Neck module are input into the next downsampling module and the next efficient aggregation module to output the second input features of the Neck module. The second input features of the Neck module are input into the next downsampling module and the next efficient aggregation module. The output features are input into the spatial pyramid pooling module. The spatial pyramid pooling module outputs the third input features of the Neck module.

[0012] In the Neck module, the first input features and the second input features are processed through the CBS module, and the third input features are processed through the CBS module and the feature content perception reorganization module. The output features are combined with the processed second input features and input into the ELAN-H module to output features S1. S1 is combined with the first input features of the Neck module through the CBS module and the feature content perception reorganization module, and input into the ELAN-H module to obtain an output, which is recorded as S2. S2 is combined with S1 through the downsampling module and input into the ELAN-H to obtain an output, which is recorded as S3. S3 is combined with the third input features of the Neck module through the downsampling module and input into the ELAN-H module to obtain an output, which is recorded as S4.

[0013] The residual convolution module is located between the head module and the backbone module. The first input features of the Neck module are combined with S2 after passing through the residual convolution module, and the combined result is recorded as H1. The second input features of the Neck module are combined with S3 after passing through the residual convolution module, and the combined result is recorded as H2. The third input features of the Neck module are combined with S4 after passing through the residual convolution module, and the combined result is recorded as H3.

[0014] In the head module, H1, H2 and H3 are input into the reparameterization module after being reduced in dimension through the CBS module. The obtained output is input into the convolution layer to reduce the dimension and obtain the corresponding output results head1, head2 and head3. head1, head2 and head3 are integrated and iteratively trained with the loss function.

[0015] Further, the efficient aggregation module has four branches. The first branch is a CBS module, the second branch is also a CBS module, the third branch is stacked with two CBS modules after the second branch, and the fourth branch is stacked with two CBS modules after the third branch. The output results of the four branches are combined as the output of the efficient aggregation module.

[0016] Further, the downsampling module has two branches, one branch is a pooling layer and a CBS module, and the other branch is two CBS modules, and the downsampling module combines the output results of the two branches as the output.

[0017] Further, the spatial pyramid pooling module has two branches, the first branch first passes the input features through three CBS modules, then inputs the output results into four pooling layers to obtain four output results, combines the four output results to obtain an output, and then inputs the output into two CBS modules to obtain the output features of the first branch, and the second branch is composed of a CBS module, and the spatial pyramid pooling module combines the output results of the two branches as the output result.

[0018] Further, the feature content-aware reorganization module is divided into an upsampling kernel prediction module and a feature reorganization module, the upsampling kernel prediction module performs channel compression on the input feature map, then performs content coding and upsampling kernel prediction, and the feature reorganization module performs feature reorganization according to the predicted upsampling kernel.

[0019] Further, the residual convolution module is composed of a convolution layer, a batch normalization layer and a SiLU function.

[0020] Further, the reparameterization module has three branches, for the input features, the first branch directly performs batch normalization, the second branch performs batch normalization after passing through a CBS module, and the third branch performs normalization after passing through a CBS module, and then the reparameterization module combines the output results of the three branches as the output.

[0021] Further, the CBS module is composed of a convolution layer, a batch normalization layer and a SiLU function.

[0022] Further, the normalized Whistans distance function is:

[0023]

[0024] Wherein, C is a constant, determined by the characteristics of the data set, and WD is the Whistans distance between the rectangular frame and the real frame,

[0025] The Whistans distance between the rectangular frame and the real frame is:

[0026]

[0027] Wherein, x1, y1 is the center point coordinate of the real frame, w1, h1 is the width and height of the real frame, x2, y2 is the center point coordinate of the predicted frame, and w2, h2 is the width and height of the predicted frame.

[0028] Compared with the prior art, the present application has the following beneficial effects:

[0029] (1) The detection model of the present application uses the small target ratio of the data set as the weight to fuse the IoU function and the NWD function to replace the original IoU function as a new coordinate regression loss function. Compared with the original function, the loss function of the present application can improve the missed detection problem of small targets and effectively improve the detection accuracy.

[0030] (2) The detection model two of the present application uses a feature content-aware reorganization (CARAFE) operator to replace the nearest neighbor interpolation operator, which can obtain a larger receptive field and effectively utilize the information around the feature map, further improving the recognition accuracy of the model.

[0031] (3) The present application adds a residual convolution (RESCONV) between the Backbone and the Head of the model to obtain more position information. The Neck part of YOLOv7 is a feature pyramid network (FPN), a network structure that fuses high-level features and low-level features. High-level features contain more semantic information, but the resolution is relatively low, and the position information contained is less. The resolution of low-level features is higher, containing more position information, but the semantic information is less, so fusing higher-level features with low-level features can obtain rich semantic information and position information. However, since small targets are small in size, more position information is needed, and when low-level features fuse the information of high-level features, the original position information is lost, causing small targets to be missed. The present application connects the output feature map of the Backbone to the Head through residual convolution, which can not only retain the information after fusing high-level and low-level features, but also obtain more rich position information, improve the missed detection of small targets, and further improve the detection accuracy of small targets. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 The structure diagram of the ceramic tile defect detection model of the present application. DETAILED DESCRIPTION

[0033] The present application will be described in detail below in conjunction with the drawings and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present application, and gives a detailed implementation manner and specific operation process, but the protection scope of the present application is not limited to the following examples.

[0034] The current mainstream target detection model such as YOLOv7 can achieve high recognition accuracy and speed, but the accuracy is not high when identifying small targets in the picture. There are a large number of small targets in the ceramic tile defects, which leads to low recognition accuracy of the existing target detection model, so it is difficult to put the model into practical application. In view of the problem of low recognition accuracy of small targets, the YOLOv7 is improved, including using a new loss function, replacing the upsampling operator and changing the network structure and other ways, the improved model has higher recognition accuracy, and maintains the recognition speed of the original model, which can help enterprises to apply it to the product quality inspection link of the factory.

[0035] The purpose of the present application is to improve the YOLOv7 model to improve the recognition accuracy of small targets in ceramic tile defects, and a ceramic tile defect detection method based on deep learning is proposed, and the specific steps of the method include:

[0036] Obtain the ceramic tile image to be detected, input the image into the trained ceramic tile defect detection model, and output the detection result, wherein the training process of the ceramic tile defect detection model is specifically:

[0037] An improved YOLOv7 model is established, the YOLOv7 model includes a backbone module, a Neck module, a residual convolution module and a head module, the training set is input into the improved YOLOv7 model, and the loss function combined by the weight fusion IoU function and the NWD function is trained to obtain the trained ceramic tile defect detection model, wherein the loss function is specifically:

[0038] Loss=(1-ration)*(1-NWD)+ration(1-IoU)

[0039] Wherein, ration is the proportion of small targets in the data set, NWD represents the normalized Whistons distance function, and IoU represents the intersection over union function.

[0040] The normalized Whistons distance function is:

[0041]

[0042] Wherein, C is a constant, which is determined by the characteristics of the data set, WD is the Whistons distance between the rectangular frame and the real frame,

[0043] The Whistons distance between the rectangular frame and the real frame is:

[0044]

[0045] Wherein, x1, y1 are the center point coordinates of the real frame, w1, h1 are the width and height of the real frame, x2, y2 are the center point coordinates of the predicted frame, and w2, h2 are the width and height of the predicted frame.

[0046] The structure of the improved ceramic tile defect detection model is as shown in Figure 1 The model is composed of four parts, namely Backbone, Neck and Head, and RESCONV between Backbone and Head. The four parts will be introduced respectively as follows:

[0047] Backbone: After the input picture is input, the features are preliminarily extracted through four CBS modules (a CBS module is composed of a convolution layer, a batch normalization layer and a SiLU function in series) and ELAN (ELAN is an efficient aggregation module of YOLOv7, which has four branches, the first branch is a CBS module, the second branch is also a CBS module, the third branch is two CBS modules stacked after the second branch, and the fourth branch is two CBS modules stacked after the third branch, and finally the output results of the four branches are combined to obtain the output of the efficient aggregation module), and then the output results are input into the MP module (the MP module is a down-sampling module of YOLOv7, which has two branches, one branch is a pooling layer and a CBS module, and the other branch is two CBS modules, and the output results of the two branches are combined to obtain the output of the MP module) and the ELAN module to obtain the output results, which are the first input features of the Neck module and the input features of the next MP module and ELAN module. After the next MP module and ELAN module, the output results are obtained, which are the second input features of the Neck module and the input features of the next MP module and ELAN module. After the next MP module and ELAN module, the output results are obtained, which are input into the SPPCSPC module (the SPPCSPC module is a spatial pyramid pooling module of YOLOv7, which has two branches, the first branch will first pass the input features through three CBS modules, then input the output results into four pooling layers to obtain four output results, then combine the four output results to obtain an output, and then input it into two CBS modules to obtain the output features of the first branch, the second branch is composed of a CBS module, and the SPPCSPC combines the output results of the two branches to obtain the final output results. ) to obtain the third input features of the Neck module.

[0048] Neck: The three inputs of the Neck module are generated by the Backbone, wherein the first input feature and the second input feature pass through the CBS module, and the third input feature is not processed. The third input feature of the Neck module passes through the CBS module and the CARAFE module (CARAFE is a feature content-aware reorganization module, which is divided into two main modules, namely the up-sampling kernel prediction module and the feature reorganization module. The up-sampling kernel prediction module will compress the feature map in the channel, then encode the content and perform up-sampling kernel prediction, and the feature reorganization module will reorganize the features according to the predicted up-sampling kernel.) to obtain an output result, which is combined with the second input feature of the Neck module and input into the ELAN-H module (ELAN-H is similar to the ELAN structure, except that the combination method is different.) to obtain an output, which is denoted as S1. S1 passes through the CBS module and the CARAFE module and is combined with the first input feature of the Neck module and input into the ELAN-H module to obtain an output, which is denoted as S2. S2 is combined with S1 by the MP module and input into the ELAN-H to obtain an output, which is denoted as S3. S3 is combined with the third input feature of the Neck module by the MP module and input into the ELAN-H module to obtain an output, which is denoted as S4.

[0049] RESCONV: A RESCONV module (RESCONV module is composed of a convolution layer, a batch normalization layer and a SiLU function) is added between the Head and the Backbone. Specifically, the first input feature of the Neck module is combined with S2 by RESCONV, and the combined result is denoted as H1; the second input feature of the Neck module is combined with S3 by RESCONV, and the combined result is denoted as H2; and the third input feature of the Neck module is combined with S4 by RESCONV, and the combined result is denoted as H3.

[0050] Head: H1 is input into the REP module (REP module is the reparameterization module of YOLOv7. During model training, there are three branches. The first branch directly performs batch normalization, the second branch passes through the CBS module and then performs batch normalization, and the third branch passes through the CBS module and then performs normalization. After that, the output results of the three branches are combined) after dimension reduction to obtain an output, which is input into the CONV (convolution layer) for dimension reduction to obtain Head1. The formation process of Head2 and Head3 is similar to that of Head1.

[0051] The loss functions of the obtained Head1, Head2 and Head3 are iteratively trained to obtain a trained model.

[0052] Compared with the traditional YOLOv7 model, the present application makes the following improvements:

[0053] I. A new coordinate regression loss function is proposed. The original coordinate regression loss function of the model uses the intersection over union (IoU) function, but the IoU function is very sensitive to small targets. When detecting small targets, even if the distance between the predicted box and the real box is very close, the IoU value of them is very low, which leads to a large number of predicted boxes close to the real box being eliminated, thereby causing the problem of missing detection of small targets. The NWD function is a normalized wasserstein distance function, which is obtained by modeling the rectangular box as a Gaussian function and calculating the wasserstein distance between the rectangular boxes.

[0054] Because the NWD function is not sensitive to small targets, using the NWD function to replace the IoU function can improve the problem of missing detection of small targets. However, there are still a certain number of medium and large targets in the ceramic flaw, and experiments have proved that if only the NWD function is used to replace the IoU function, the precision improvement is not obvious enough. Therefore, the present application uses the proportion of small targets in the data set as the weight to fuse the IoU function and the NWD function to replace the original IoU function as a new coordinate regression loss function

[0055] II. The feature content-aware reassembly (CARAFE) operator is used to replace the nearest neighbor interpolation operator. The up-sampling process of the nearest neighbor interpolation operator is essentially a 4-fold replication of the original data, which is a kind of "uniform" up-sampling. The CARAFE operator is divided into an up-sampling kernel prediction module and a feature reassembly module. For the up-sampling kernel prediction module, first, use 1*1 convolution to compress the channel number of the input feature map to reduce the subsequent calculation amount, then use 5*5 convolution to generate a 3*3 up-sampling kernel according to the content of the input feature map, and finally use the softmax function to normalize the up-sampling kernel. For the feature reassembly module, first use the nearest neighbor interpolation operator to perform 2-fold up-sampling on the input feature map, and then use the up-sampling kernel obtained by the up-sampling kernel prediction module to convolve the feature map obtained by 2-fold up-sampling to complete the up-sampling process. Because the up-sampling kernel is obtained by learning, it is related to the semantic information of the input feature map, and using the CARAFE operator to replace the nearest neighbor interpolation operator can obtain a larger receptive field, effectively utilize the information around the feature map, and further improve the recognition accuracy of the model.

[0056] III. Adding residual convolution (RESCONV) between the Backbone and the Head of the model to obtain more location information. The Neck part of YOLOv7 is a feature pyramid network (FPN), a network structure that fuses high-level features with low-level features. High-level features contain more semantic information, but because of the lower resolution, they contain less location information. Low-level features have higher resolution and contain more location information, but they have less semantic information, so fusing high-level features with low-level features can obtain both rich semantic information and location information. However, because small targets are small in size, they require more location information, and when low-level features fuse the information of high-level features, the original location information is lost, causing small targets to be missed. Inspired by the residual idea, the output feature map of the Backbone is connected to the Head through residual convolution, which can both retain the information after fusing high-level and low-level features and obtain more location information, improve the missed detection of small targets, and further improve the detection accuracy of small targets.

[0057] The preferred embodiments of the present application are described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and changes without creative labor based on the concept of the present application. Therefore, any technical solutions obtained by logical analysis, reasoning or limited experiments based on the prior art within the concept of the present application shall be within the protection scope defined by the claims.

Claims

1. A deep learning-based ceramic tile defect detection method, characterized by, The method comprises: An image of a ceramic tile to be detected is acquired, the image is input into a trained ceramic tile defect detection model, and a detection result is output, wherein the training process of the ceramic tile defect detection model is specifically as follows: An improved YOLOv7 model is established, the YOLOv7 model comprises a backbone module, a Neck module, a residual convolution module and a head module, a training set is input into the improved YOLOv7 model, the training set is trained through a loss function in which a weight fusion IoU function and an NWD function are combined, and a trained ceramic tile defect detection model is obtained, wherein the loss function is specifically as follows: loss = (1-ration)*(1-NWD)+ration*(1-IoU) Wherein, ration is the proportion of small targets in the data set, NWD represents the normalized Whistans distance function, and IoU represents the intersection over union function; After the training set is input into the improved YOLOv7 model, the training set first enters the backbone module, then passes through a plurality of CBS modules and an efficient aggregation module to obtain preliminary features, the preliminary features are input into a first downsampling module and a first efficient aggregation module, the output features are used as the first input features of the Neck module, the first input features of the Neck module are input into a next downsampling module and a next efficient aggregation module, the second input features of the Neck module are output, the second input features of the Neck module are input into a further downsampling module and a further efficient aggregation module, and the output features are input into a spatial pyramid pooling module, and the spatial pyramid pooling module outputs the third input features of the Neck module; In the Neck module, the first input features and the second input features are processed through the CBS module, the third input features are processed through the CBS module and a feature content perception reorganization module, the output features are combined with the processed second input features and input into an ELAN-H module, output features S1 are output, S1 is processed through the CBS module and the feature content perception reorganization module, combined with the first input features of the Neck module and input into the ELAN-H module to obtain an output, and this output is recorded as S2, S2 is processed through a downsampling module and combined with S1 and input into the ELAN-H to obtain an output, and this output is recorded as S3, S3 is processed through a downsampling module and combined with the third input features of the Neck module and input into the ELAN-H module to obtain an output, and this output is recorded as S4; The residual convolution module is located between the head module and the backbone module, the first input features of the Neck module are combined with S2 after being processed through the residual convolution module, the combination result is recorded as H1, the second input features of the Neck module are combined with S3 after being processed through the residual convolution module, the combination result is recorded as H2, and the third input features of the Neck module are combined with S4 after being processed through the residual convolution module, and the combination result is recorded as H3. In the head module, H1, H2 and H3 are input into the reparameterization module after dimension reduction by the CBS module, and the output obtained is input into the convolution layer for dimension reduction to obtain the corresponding output results head1, head2 and head3, and the head1, head2 and head3 are iteratively trained together with the loss function. 2.The ceramic tile flaw detection method based on deep learning according to claim 1, characterized in that, The efficient aggregation module has four branches, the first branch is a CBS module, the second branch is also a CBS module, the third branch is two CBS modules stacked after the second branch, and the fourth branch is two CBS modules stacked after the third branch, and the efficient aggregation module combines the output results of the four branches as the output. 3.The ceramic tile flaw detection method based on deep learning according to claim 1, characterized in that, The downsampling module has two branches, one branch is a pooling layer and a CBS module, and the other branch is two CBS modules, and the downsampling module combines the output results of the two branches as the output. 4.The ceramic tile flaw detection method based on deep learning according to claim 1, characterized in that, The spatial pyramid pooling module has two branches, the first branch will first pass the input features through three CBS modules, then input the output results into four pooling layers to obtain four output results, then combine the four output results to obtain an output, and then input it into two CBS modules to obtain the output features of the first branch, and the second branch is composed of a CBS module, and the spatial pyramid pooling module combines the output results of the two branches as the output. 5.The ceramic tile flaw detection method based on deep learning according to claim 1, characterized in that, The feature content-aware reorganization module is divided into an upsampling kernel prediction module and a feature reorganization module, the upsampling kernel prediction module performs channel compression on the input feature map, then performs content encoding and upsampling kernel prediction, and the feature reorganization module performs feature reorganization according to the predicted upsampling kernel. 6.The ceramic tile flaw detection method based on deep learning according to claim 1, characterized in that, The residual convolution module is composed of a convolution layer, a batch normalization layer and a SiLU function. 7.The ceramic tile flaw detection method based on deep learning according to claim 1, characterized in that, The reparameterization module has three branches, for the input features, the first branch directly performs batch normalization, the second branch performs batch normalization after the CBS module, and the third branch performs normalization after the CBS module, and then the reparameterization module combines the output results of the three branches as the output.

8. The deep learning-based ceramic tile defect detection method according to any one of claims 2-7, characterized in that, The CBS module is composed of a convolution layer, a batch normalization layer and a SiLU function. 9.The ceramic tile flaw detection method based on deep learning according to claim 1, characterized in that, The normalized Whistunst distance function is: Where C is a constant determined by the characteristics of the data set, WD is the Whistunst distance between the rectangular frame and the real frame, The Whistunst distance between the rectangular frame and the real frame is: Where x1, y1 are the center point coordinates of the real frame, w1, h1 are the width and height of the real frame, x2, y2 are the center point coordinates of the predicted frame, and w2, h2 are the width and height of the predicted frame.

Images