Deformation defect detection method and device

Abstract

The present invention discloses a method and device for detecting deformation defects, the method comprising: obtaining a target image with a deformation defect image; inputting the target image into a preset deformation defect detection model; outputting the target position of the deformation defect image in the target image and / or the target defect type corresponding to the deformation defect image through the deformation defect detection model; wherein the target position and the target defect type are displayed through a prediction box output by the deformation defect detection model, the target defect type is one of at least one preset defect type, and the loss function of the deformation defect detection model includes a deformation constraint, and the deformation constraint is used to constrain the deformation of the prediction box. The method disclosed by the present invention can solve the problem of low accuracy of low-contrast deformation defect detection in the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and in particular to a method and device for detecting deformation defects. Background Art

[0002] Traditional defect detection technologies such as automatic optical inspection combine electronic images with algorithms to judge the output of defect detection results.

[0003] However, the detection coverage of automatic optical inspection is relatively small, and most of the inspection targets can only be obvious defects with standard and easy-to-identify. The remaining defects, especially low-contrast deformation defects, will make the defect features unclear due to low contrast, and the defect deformation will make the defect difficult to confirm and identify, which is prone to false detection problems. Therefore, in the existing technology, deformation defects can only be inspected manually, and the inspection standards are prone to vary from person to person. In addition, the algorithm has poor portability, and the algorithm needs to be re-debugged for defect detection for different products. In addition, the maintenance cost of automatic optical inspection equipment is high and it cannot cope with sudden new defects.

[0004] In summary, there is an urgent need in the prior art for a method to improve the accuracy of low-contrast deformation defect detection. Summary of the invention

[0005] The present invention provides a method and device for detecting deformation defects, which are used to solve the problem of low detection accuracy of low-contrast deformation defects in the prior art.

[0006] In order to solve the above technical problems, the present invention provides a deformation defect detection method, which is applied to industrial defect detection. The method comprises:

[0007] Obtain a target image with a deformation defect image;

[0008] Inputting the target image into a preset deformation defect detection model;

[0009] Outputting the target position of the deformation defect image in the target image and / or the target defect type corresponding to the deformation defect image through the deformation defect detection model;

[0010] Among them, the target position and the target defect type are displayed by the prediction box output by the deformation defect detection model, the target defect type is one of at least one preset defect type, and the loss function of the deformation defect detection model includes a deformation constraint, and the deformation constraint is used to constrain the deformation of the prediction box.

[0011] Optionally, before inputting the target image into a preset deformation defect detection model, the method further includes:

[0012] Inputting a sample image into a preset deep learning model for training, wherein the sample image has a deformation defect image, and the sample image is annotated with a labeling box indicating the position of the deformation defect image in the sample image and the defect type corresponding to the deformation defect image;

[0013] Updating the weights of the deep learning model until the loss function of the deep learning model outputs a minimum value;

[0014] Determine a target weight that minimizes the output of the loss function;

[0015] The deep learning model corresponding to the target weight is determined as a deformation defect detection model.

[0016] Optionally, the step of inputting the sample image into a preset deep learning model for training includes:

[0017] Inputting a sample image into a composite backbone network, and extracting image features of the sample image through the composite backbone network;

[0018] Inputting the image features into a feature pyramid network, and extracting feature representations of different scales of the image features through the feature pyramid network;

[0019] The feature representations of the image features at different scales are input into a detection head, and the prediction box output by the detection head outputs the predicted position of the deformation defect image in the sample image and the defect type corresponding to the deformation defect image.

[0020] Optionally, before updating the weight of the deep learning model until the loss function of the deep learning model outputs a minimum value, the method further includes:

[0021] Determine the deformation constraint by a first preset formula;

[0022] The first preset formula is:

[0023]

[0024] Wherein, β is a deformation constraint, w is a first width of the prediction box, h is a first height of the prediction box, and w gt is the second width of the label box, and h gt is the second height of the annotation box.

[0025] Optionally, extracting the image features of the sample image by using a composite backbone network includes:

[0026] Extracting a first image feature larger than a first preset scale through a main backbone network;

[0027] A second image feature smaller than a second preset scale is extracted through the auxiliary backbone network.

[0028] Optionally, extracting image features smaller than a second preset scale by using an auxiliary backbone network includes:

[0029] The image features smaller than the second preset scale are extracted by lightweight network using at least one ShuffleNet module.

[0030] Optionally, after extracting the image features smaller than the second preset scale by the auxiliary backbone network, the method further includes:

[0031] The first image feature and the second image feature are fused through an enhanced receptive field module to obtain the image feature of the sample image.

[0032] Optionally, before inputting the sample image into the composite backbone network, the method further includes:

[0033] For each convolutional layer of the composite backbone network, a convolutional channel attention enhancement mechanism is integrated.

[0034] Optionally, before inputting the target image into a preset deformation defect detection model, the method further includes:

[0035] Enhancing the contrast of the target image by using a histogram equalization algorithm;

[0036] Before inputting the feature representations of the image features at different scales into the detection head, the method further includes:

[0037] The anchor box of the detected head is updated by a k-means algorithm, wherein the predicted box is one of the anchor boxes.

[0038] In a second aspect, the present invention provides a deformation defect detection device, the device comprising:

[0039] An acquisition module, used for acquiring a target image with a deformation defect image;

[0040] An input module, used for inputting the target image into a preset deformation defect detection model;

[0041] An output module, configured to output the target position of the deformation defect image in the target image and / or the target defect type corresponding to the deformation defect image through the deformation defect detection model;

[0042] Among them, the target position and the target defect type are displayed by the prediction box output by the deformation defect detection model, the target defect type is one of at least one preset defect type, and the loss function of the deformation defect detection model includes a deformation constraint, and the deformation constraint is used to constrain the deformation of the prediction box.

[0043] Compared with the prior art, the deformation defect detection method provided by the present invention has the following beneficial effects:

[0044] By adding constraints for constraining the deformation of the prediction box in the loss function of the deformable defect detection model, the deformation of the prediction box can be constrained when the loss function outputs a small value. It can be understood that the deformation of the defect image will cause the deformation of the prediction box used to predict the defect image, and the deformation of the prediction box will cause the prediction box to predict the position of the deformed defect image inaccurately. Therefore, by constraining the deformation of the prediction box, the accuracy of detection of low-contrast deformation defects can be improved, thereby solving the problem of low detection accuracy of low-contrast deformation defects in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only part of the embodiments of the present invention, rather than all of the embodiments. For ordinary technicians in this field, without paying creative work, other drawings obtained based on these drawings all fall within the scope of protection of this application.

[0046] Figure 1 is a flow chart of a deformation defect detection method provided in an embodiment of the present application;

[0047] Figure 2 is a flow chart of another deformation defect detection method provided in an embodiment of the present application;

[0048] Figure 3 is a flow chart of another deformation defect detection method provided in an embodiment of the present application;

[0049] Figure 4 is a flow chart of another deformation defect detection method provided in an embodiment of the present application;

[0050] Figure 5 is a flow chart of another deformation defect detection method provided in an embodiment of the present application;

[0051] Figure 6 is a defect detection result diagram output by the deformation defect detection model provided in an embodiment of the present application;

[0052] Figure 7is another defect detection result diagram outputted by the deformation defect detection model provided in the embodiment of the present application;

[0053] Figure 8 It is a structural schematic diagram of a deformation defect detection device provided in an embodiment of the present application;

[0054] Fig. 9 It is a structural schematic diagram of a deformation defect detection device provided in an embodiment of the present application;

[0055] Fig.10 It is a structural schematic diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0057] In order to make the description of the disclosed content more detailed and complete, the following is an illustrative description of the implementation mode and specific examples of the present invention; however, this is not the only form of implementing or applying the specific embodiments of the present invention. The implementation mode covers the features of multiple specific embodiments and the method steps and their sequence for constructing and operating these specific embodiments. However, other specific embodiments can also be used to achieve the same or equal functions and step sequences. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0058] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0059] In the description of the embodiments of the present invention, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two, and other quantifiers are similar. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention, and the embodiments of the present application and the features in the embodiments may be combined with each other without conflict.

[0060] Example 1

[0061] like Figure 1 The flowchart of a deformation defect detection method provided in an embodiment of the present invention includes the following steps.

[0062] Step S102, obtaining a target image with a deformation defect image.

[0063] Specifically, the deformation defect image of the target image may be a deformed defect.

[0064] More specifically, the method of obtaining the target image can be any method that can obtain an image, for example, the defect can be photographed to obtain the target image, or the defect can be photographed to obtain a video, and the target image can be obtained by intercepting the video frame. As a specific example, the target image can be taken by an industrial camera.

[0065] Step S103: input the target image into a preset deformation defect detection model.

[0066] It should be noted that the deformation defect detection model preset in this step can be any algorithm model that can be used for defect detection. For example, the deformation defect detection model can be a deep learning model of the YOLO series for target detection, or it can be a deep learning model of the region-based convolutional neural network (RCNN) series for target detection.

[0067] Optionally, before inputting the target image into the preset deformation defect detection model, the target image may be preprocessed, for example, the target image may be normalized, and the size of the target image may be adjusted so that the size of the target image is suitable for processing by the deformation defect detection model.

[0068] Step S104: outputting the target position of the deformation defect image in the target image and / or the target defect type corresponding to the deformation defect image through the deformation defect detection model.

[0069] Among them, the target position and the target defect type are displayed by the prediction box output by the deformation defect detection model, the target defect type is one of at least one preset defect type, and the loss function of the deformation defect detection model includes a deformation constraint, and the deformation constraint is used to constrain the deformation of the prediction box.

[0070] It can be understood that the prediction box of the deformation defect detection model can be accompanied by the position information of the predicted deformation defect image in the target image and the defect type of the deformation defect image.

[0071] Specifically, the position information of the deformation defect image in the target image can be any information that can represent the position of the deformation defect image. For example, a coordinate system can be established in the target image based on certain rules, and then the coordinates of each vertex of the deformation defect image can be obtained, and the coordinates of these vertices can be used as the position information of the deformation defect image in the target image; or the coordinates of the diagonal vertices of the deformation defect image and the width and height of the deformation defect image can be obtained based on the established coordinate system, and the coordinates, width and height of the diagonal vertices of the deformation defect image can be used as the position information of the deformation defect image in the target image.

[0072] More specifically, the preset at least one defect type can be set according to the industrial field to which the present embodiment is applied. For example, if the present embodiment is applied to the field of mobile terminals, the setting of the defect type in this step can be set based on the defects that may exist in the mobile terminal, such as camera defects, battery defects, and display screen defects. In addition, it can be understood that the preset at least one defect type means that the preset defect type can be one or more. When the preset defect type is one, the target defect type is the preset defect type. When the preset defect type is multiple, the target defect type is one of the multiple preset defect types.

[0073] It can be understood that the loss function in the deep learning model can include several constraints. By controlling the loss function to output a smaller value, the parameters corresponding to the constraints in the loss function can be constrained. On this basis, it can be understood that by adding constraints to constrain the deformation of the prediction box, the deformation of the prediction box can be constrained when the loss function outputs a smaller value. It can be understood that the deformation of the defect image will cause the deformation of the prediction box that predicts it, and the deformation of the prediction box will cause the prediction box to predict the position of the deformed defect image inaccurately. Therefore, by constraining the deformation of the prediction box, the accuracy of the detection of deformed defects can be improved.

[0074] In an optional implementation, Figure 2 FIG. 1 is a flow chart of another deformation defect detection method provided by an embodiment of the present invention. Before step S102, the method further includes:

[0075] Step S1011, inputting the sample image into a preset deep learning model for training;

[0076] The sample image has a deformation defect image, and the sample image is annotated with a labeling box to indicate the position of the deformation defect image in the sample image and the defect type corresponding to the deformation defect image;

[0077] Step S1012, updating the weight of the deep learning model until the loss function of the deep learning model outputs a minimum value;

[0078] Step S1013, determining a target weight that makes the loss function output a minimum value;

[0079] Step S1014: determine the deep learning model corresponding to the target weight as a deformation defect detection model.

[0080] It should be noted that the method of marking defects by using the annotation box may be any method that can be used for image annotation, for example, it may be marked by using image annotation tools such as CVAT, VOTT or labelimg.

[0081] Specifically, the deep learning model in this step can be a deep learning model for target detection, such as a YOLO series model and an RCNN deep learning model.

[0082] It should be noted that the setting of the defect position and defect category in this step can refer to step S103, which will not be repeated here.

[0083] Specifically, the loss function in this step may include classification loss, border loss, and confidence loss. In order to ensure that the value of the loss function output by the deep learning model is minimized, the weights of the deep model may be continuously tentatively changed until the goal is achieved.

[0084] Optionally, for the deep learning model that determines the target weight, the deep model with the corresponding weight as the target weight can be embedded in the equipment in the factory, and defective products can be further input into the equipment. The reliability of the deep learning model corresponding to the target weight can be determined by the output defect detection results. If the reliability of the deep learning model corresponding to the target weight is high, the deep learning model can be used as the defect detection model that is finally put into use. If the reliability of the deep learning model corresponding to the target weight is low, the weight of the deep learning model is further optimized and updated, and the deep learning model is retrained until the deep learning model meets the reliability requirements.

[0085] It can be understood that by training the deep learning model through the present implementation method to obtain a defect detection model, a defect detection model with higher reliability can be obtained, thereby realizing the automation of defect detection.

[0086] In an optional implementation, Figure 3 FIG. 1 is a flow chart of another deformation defect detection method provided by an embodiment of the present invention, wherein step S1011 comprises:

[0087] Step S10111, inputting a sample image into a composite backbone network, and extracting image features of the sample image through the composite backbone network;

[0088] Step S10112, inputting the image features into a feature pyramid network, and extracting feature representations of different scales of the image features through the feature pyramid network;

[0089] Step S10113, inputting feature representations of different scales of the image features into a detection head, and outputting the predicted position of the deformation defect image in the sample image and the defect type corresponding to the deformation defect image through the prediction box output by the detection head.

[0090] It should be noted that the preset deep learning model may include a composite backbone network, a feature pyramid network and a detection head, wherein the composite backbone network can be used to extract image features of a sample image, the feature pyramid network may include several layers, which can be used to input image features output by the composite backbone network and extract feature representations of image features at different scales, and the feature representation of each scale is extracted through a layer of the feature pyramid network. The detection head is used to input feature representations of image features at different scales, and predict the position and defect type of the deformation defect image corresponding to the feature representation of each scale through the prediction box it includes.

[0091] It should be noted that the composite backbone network can be any network structure that can extract image features of sample images, such as SqueezeNet series networks, ShuffleNet series networks, and Resnet networks; the feature pyramid network can be any network structure that can extract feature representations of different scales, such as Single Shot MultiBox Detector (SSD) network, Atrous spatial pyramid pooling (ASPP) network, and recursive feature pyramid network (RFP) network; the detection head can be any network structure that can predict the position and defect type of a deformation defect image through a prediction box, such as YOLO network, RetinaNet network, and Mask RCNN network.

[0092] It can be understood that the feature map of the image can be extracted through the composite backbone network, and then the feature pyramid network can process the feature map, extract the feature representations of different scales corresponding to the feature map, and then the corresponding defect image in the feature representations of different scales of the feature map can be detected by detecting the head. It can be further understood that detecting defects in the feature representations of different scales of the feature map can detect both the contour features and detail features of the defects, thereby improving the accuracy of the deformation defect detection model in detecting deformation defects.

[0093] In an optional implementation, before step S1013, the method further includes:

[0094] Determine the deformation constraint by a first preset formula;

[0095] The first preset formula is:

[0096]

[0097] Wherein, β is a deformation constraint, w is a first width of the prediction box, h is a first height of the prediction box, and w gt is the second width of the label box, and h gt is the second height of the annotation box.

[0098] It should be noted that the prediction box in the present implementation is one of the anchor boxes set in the deformation defect detection model. Several anchor boxes of different scales or the same scale can be set in the deformation defect detection model, where each anchor box is used to perform defect detection on the deformation defect image of the detection area it covers, and output the defect detection result as a prediction box.

[0099] Optionally, the loss function may also include other constraints, such as coverage area constraints, center point distance constraints, and aspect ratio constraints. The calculation methods of the above constraints are already well-known techniques in the art and will not be described in detail here.

[0100] It can be understood that the first preset formula for the deformation constraint in this implementation can be determined according to the second preset formula, where the second preset formula is: Among them, P is the pressure at the deformation defect, F is the pressure at the deformation defect, and S is the deformation area at the deformation defect. Therefore, it can be understood that the deformation of the defect can be reflected by its deformation area. Therefore, the first preset formula can further determine the deformation constraint by determining the ratio of the area of ​​the prediction box to the area of ​​the annotation box.

[0101] It can be understood that the smaller the loss function is, the smaller the deformation constraint is, which corresponds to the first preset formula, that is, the larger the area of ​​the annotation box is, the smaller the area of ​​the prediction box is. The larger the area of ​​the annotation box means that the defect is more likely to be deformed. At this time, reducing the area of ​​the prediction box can effectively avoid the prediction box from deforming with the deformation of the defect, and thus avoid the problem of inaccurate position prediction of the deformed defect image due to the deformation of the prediction box. Therefore, the deformation constraint obtained by the first constraint formula can improve the accuracy of detection of deformation defects by constraining the deformation of the prediction box.

[0102] In an optional implementation, Figure 4 FIG. 1 is a flow chart of another deformation defect detection method provided by an embodiment of the present invention, wherein step S10111 includes:

[0103] Step S101111, extracting first image features larger than a first preset scale through a main backbone network;

[0104] Step S101112: extracting second image features smaller than a second preset scale through an auxiliary backbone network.

[0105] Specifically, the first preset scale and the second preset scale can be set according to specific needs during application. For example, the first preset scale can be equal to the second preset scale, or the first preset scale can be greater than the second preset scale.

[0106] It should be noted that the main backbone network can be any network that can extract image features larger than the first preset scale, such as the SqueezeNet series networks, ShuffleNet series networks and Resnet networks with a larger receptive field; the auxiliary backbone network can be any network that can extract image features smaller than the second preset scale, such as the SqueezeNet series networks, ShuffleNet series networks and Resnet networks with a smaller receptive field.

[0107] It can be understood that the main backbone network can extract the first image features of the sample image with a larger scale, that is, the main backbone network can extract larger features in the sample image, and the auxiliary backbone network can extract the second image features smaller than the second preset scale, that is, the auxiliary backbone network can extract smaller detail features in the sample image, so the main backbone network and the auxiliary backbone network can make the extracted feature map more comprehensive.

[0108] In an optional implementation, step S101112 includes:

[0109] The image features smaller than the second preset scale are extracted by lightweight network using at least one ShuffleNet module.

[0110] Specifically, the ShuffleNet module lightweight network can be any ShuffleNet module lightweight network, such as the ShuffleNetv1 network and the ShuffleNetv2 network.

[0111] It can be understood that the ShuffleNet module lightweight network has a lighter structure than other backbone networks, which can make the backbone network calculate faster, so that the deformation defect detection model calculates faster.

[0112] In an optional implementation, Figure 5 FIG. 1 is a flow chart of another deformation defect detection method provided by an embodiment of the present invention. After step S101112, the method further includes:

[0113] Step S101113: The first image feature and the second image feature are fused by an enhanced receptive field module to obtain the image feature of the sample image.

[0114] It can be understood that the enhanced receptive field module can fuse image features of different scales to obtain fused features, thereby making the features output by the backbone network to the feature pyramid network more comprehensive.

[0115] Specifically, the enhanced receptive field module in the present implementation may be any enhanced receptive field module, such as ASPP and radial basis function (RBF).

[0116] It is understandable that by enhancing the receptive field module, the receptive field of a backbone network can also be increased, so that the backbone network can extract more features and effectively increase the scale of the feature map output by the backbone network. Optionally, the main backbone network can be enabled to extract a first image feature greater than a first preset scale by enhancing the receptive field module, and further, the first image feature and the second image feature can be fused by enhancing the receptive field module to obtain the image feature of the sample image.

[0117] Optionally, the enhanced receptive field module can be an RBF-s network, and the RBF-s neural network has strong nonlinear fitting capabilities, can map arbitrarily complex nonlinear relationships, and has simple learning rules, which is easy to implement on a computer. It also has strong robustness, memory capacity, nonlinear mapping capabilities, and strong self-learning capabilities.

[0118] In an optional implementation, before step S10111, the method further includes:

[0119] For each convolutional layer of the composite backbone network, a convolutional channel attention enhancement mechanism is integrated.

[0120] It is understandable that the backbone network usually has multiple convolutional layers.

[0121] It should be noted that the convolutional channel attention mechanism is an attention mechanism used to enhance the performance of convolutional neural networks. It is used to extract the most representative channel information in the input feature map in order to better capture the important features in the image or specific tasks.

[0122] When performing feature extraction, traditional convolutional neural networks give the same feature weight to each channel, ignoring the correlation between channels. However, in some cases, some channels may contain more representative information, while other channels may contain more noise or irrelevant information. The convolutional channel attention mechanism learns the weight of each channel, allowing the network to automatically select the most useful channel for feature extraction.

[0123] Specifically, the convolutional channel attention mechanism generally consists of two key components: the channel attention module and the feature fusion module.

[0124] The channel attention module is responsible for calculating the weight of each channel. This can be done by reducing the dimension of the feature map to a vector using a global average pooling operation, and then obtaining the weight of each channel through a fully connected layer and an activation function. These weights can be regarded as an indicator of the importance of the channel.

[0125] The feature fusion module multiplies the original feature map with the calculated channel weight to perform weighted fusion of features from different channels, which can be achieved by simple element-by-element multiplication.

[0126] By introducing the convolutional channel attention mechanism, the network can automatically learn the importance of each channel and selectively enhance the representative channels. This helps to improve the expressiveness and generalization performance of the model, thereby achieving better performance in various computer vision tasks.

[0127] In an optional implementation, before step S102, the method further includes:

[0128] The contrast of the target image is enhanced by a histogram equalization algorithm.

[0129] Specifically, the histogram equalization algorithm increases the contrast and the clarity of the image by widening the grayscale values ​​of the image with a large number of pixels (i.e., the grayscale values ​​that play a major role in the picture) and merging the grayscale values ​​of the image with a small number of pixels (i.e., the grayscale values ​​that do not play a major role in the picture), so as to facilitate the next step of image processing.

[0130] Optionally, the contrast of the target image may be enhanced by using other methods for enhancing the contrast of the image. For example, the method for enhancing the contrast of the image may be linear transformation, histogram normalization, gamma transformation, etc.

[0131] In an optional implementation, before step S10113, the method further includes:

[0132] The anchor box of the detected head is updated by the k-means algorithm, wherein the predicted box is one of the anchor boxes.

[0133] It should be noted that the specific process of clustering anchor frames using the K-means algorithm is as follows: the first step is to cluster the sample points (i.e., information used to represent the deformation defect image) to obtain the initialized k cluster centers as anchor frames; the second step is to use a similarity metric (e.g., Euclidean distance) to assign each sample point to the cluster center closest to it; the third step is to calculate the mean of the sample points in each cluster (i.e., the sample point closest to each sample point in the cluster), and update the mean of the sample points to the cluster center of the cluster, and repeat the second and third steps until the cluster center of each cluster no longer changes, or the number of cycles reaches the preset maximum number of cycles. Therefore, it can be understood that updating the anchor frame of the detection head using the k-means algorithm is also the updating of the sample point mean to the cluster center of the sample cluster (i.e., the anchor frame) in the third step above.

[0134] It should be noted that the number K of anchor frames can be set according to specific needs of the application and is not limited in detail here.

[0135] It should be noted that the information used to represent the deformation defect image can be a labeling box that annotates the deformation defect image, or it can be the center point, width and height of the deformation defect image. No specific limitation is made here. The information of the deformation defect image can be used to determine the information of the anchor frame obtained after clustering, and then the anchor frame can be obtained based on the anchor frame information.

[0136] It can be understood that by continuously updating the anchor frame, the sample points distributed around each anchor frame can be made more evenly distributed, so that when the anchor frame detects the sample points, the detection speed is faster and the detection capability is stronger.

[0137] As a specific example, a method for detecting deformation defects includes the following steps.

[0138] Step S201, integrating a convolution channel attention enhancement mechanism for each convolution layer of the composite backbone network;

[0139] Step S202, extracting a first image feature larger than a first preset scale through a main backbone network;

[0140] Step S203, extracting image features smaller than a second preset scale through a lightweight network using at least one ShuffleNet module;

[0141] Step S204, fusing the first image feature and the second image feature through an enhanced receptive field module to obtain an image feature of the sample image;

[0142] Step S205, inputting the image features into a feature pyramid network, and extracting feature representations of different scales of the image features through the feature pyramid network;

[0143] Step S206, inputting the feature representations of the image features at different scales into the detection head, and outputting the predicted position of the deformation defect image in the sample image and the defect type corresponding to the deformation defect image through the prediction box output by the detection head;

[0144] Step S207, updating the weight of the deep learning model until the loss function of the deep learning model outputs a minimum value;

[0145] Step S208, determining a target weight that makes the loss function output a minimum value;

[0146] Step S209, determining the deep learning model corresponding to the target weight as a deformation defect detection model;

[0147] Step S210, obtaining a target image with a deformation defect image;

[0148] Step S211, inputting the target image into a preset deformation defect detection model;

[0149] Step S212, outputting the target position of the deformation defect image in the target image and / or the target defect type corresponding to the deformation defect image through the deformation defect detection model;

[0150] The target position and the target defect type are displayed by a prediction box output by the deformation defect detection model, the target defect type is one of at least one preset defect type, and the loss function of the deformation defect detection model includes a deformation constraint, which is used to constrain the deformation of the prediction box.

[0151] It should be noted that the detection results shown in the third column of Table 1 are obtained by executing steps S201 to S212, and the loss function does not include deformation constraints at this time;

[0152] It should be noted that by executing steps S210 to S212, the detection results shown in the fourth column of Table 1 can be obtained, and at this time, the loss function includes the deformation constraint;

[0153] It should be noted that, by executing all the steps, the test results shown in the fifth column of Table 1 can be obtained. Figure 6 and Figure 7 The following is the defect detection result output by the input target image, where Figure 6 The defect detection result in is that the defect type is knife line, Figure 7 The defect detection result in is that the defect type is bruise.

[0154]

[0155] Table 1

[0156] It should be noted that, based on Table 1, it can be seen that when the model and loss function are not optimized: the defect detection rate is 2.38%, and the false detection rate is 0. When only the model is optimized, 42 products with low-contrast polymorphic defects are put in, 25 are detected, and the detection rate is 59.42%; 4302 products without low-contrast polymorphic defects are put in, 20 are falsely detected, and the false detection rate is 0.46%. When only the loss function is optimized, 42 products with low-contrast polymorphic defects are put in, 30 are detected, and the detection rate is 71.43%; 4302 products without low-contrast polymorphic defects are put in, 15 are falsely detected, and the false detection rate is 0.35%. When the model and loss function are optimized at the same time, 42 products with low-contrast polymorphic defects are put in, 36 are detected, and the detection rate is 85.71%; 4302 products without low-contrast polymorphic defects are put in, 5 are falsely detected, and the detection rate is 0.12%.

[0157] The experimental results show that the proposed model and loss function for low-contrast multi-deformation defects are effective, and have good performance in both false detection rate and missed detection rate.

[0158] Example 2

[0159] Based on the above deformation defect detection method, the embodiment of the present invention provides a deformation defect detection device, such as Figure 8 As shown, the deformation defect detection device includes an acquisition module 810 , an input module 820 and an output module 830 .

[0160] An acquisition module 810 is used to acquire a target image having a deformation defect image;

[0161] An input module 820, used to input the target image into a preset deformation defect detection model;

[0162] An output module 830 is used to output the target position of the deformation defect image in the target image and / or the target defect type corresponding to the deformation defect image through the deformation defect detection model;

[0163] Among them, the target position and the target defect type are displayed by the prediction box output by the deformation defect detection model, the target defect type is one of at least one preset defect type, and the loss function of the deformation defect detection model includes a deformation constraint, and the deformation constraint is used to constrain the deformation of the prediction box.

[0164] For other details about how the above-mentioned technical solution is implemented by each module in the above-mentioned deformation defect detection device, reference may be made to the description of the deformation defect detection method provided in the above-mentioned invention embodiment, which will not be repeated here.

[0165] Based on the above deformation defect detection methods, such as Fig. 9 As shown, an embodiment of the present invention further provides a structural schematic diagram of a deformation defect detection device, the identification device includes a processor 91 and a memory 92 coupled to the processor 91. The memory 92 stores a computer program, and when the computer program is executed by the processor 91, the processor 91 executes the steps of the deformation defect detection method in the above embodiment.

[0166] For other details about how the processor 91 in the above-mentioned deformation defect detection device implements the above-mentioned technical solution, please refer to the description of the deformation defect detection method provided in the above-mentioned invention embodiment, which will not be repeated here.

[0167] Among them, the processor 91 can also be called a CPU (Central Processing Unit), and the processor 91 may be an integrated circuit chip with signal processing capabilities; the processor 91 can also be a general-purpose processor, DSP (Digital Signal Process), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, among which the general-purpose processor can be a microprocessor or the processor 31 can also be any conventional processor, etc.

[0168] like Fig.10 As shown, the embodiment of the present invention also provides a schematic diagram of the structure of a computer-readable storage medium, on which a readable computer program 101 is stored; wherein the computer program 101 can be stored in the above storage medium in the form of a software product, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, magnetic disk or optical disk, ROM (Read-Only Memory), RAM (Random Access Memory) and other media that can store program codes, or terminal devices such as computers, servers, mobile phones, and tablets.

[0169] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0170] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0171] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0172] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0173] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

[0174] The technical solution provided by the present application is introduced in detail above. The principles and implementation methods of the present application are explained by using specific examples in the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0175] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0176] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0177] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0178] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0179] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A deformation defect detection method for industrial defect detection, characterized in that: include: Obtain a target image with a deformation defect image; Inputting the target image into a preset deformation defect detection model; Outputting the target position of the deformation defect image in the target image and / or the target defect type corresponding to the deformation defect image through the deformation defect detection model; The target position and the target defect type are displayed by a prediction box output by the deformation defect detection model, the target defect type is one of at least one preset defect type, and the loss function of the deformation defect detection model includes a deformation constraint, and the deformation constraint is used to constrain the deformation of the prediction box; Before inputting the target image into a preset deformation defect detection model, the method further includes: Inputting a sample image into a preset deep learning model for training, wherein the sample image has a deformation defect image, and the sample image is annotated with a labeling box indicating the position of the deformation defect image in the sample image and the defect type corresponding to the deformation defect image; Updating the weights of the deep learning model until the loss function of the deep learning model outputs a minimum value; Determine a target weight that minimizes the output of the loss function; Determining the deep learning model corresponding to the target weight as a deformation defect detection model; Before the weight of the deep learning model is updated until the loss function of the deep learning model outputs a minimum value, the method further includes: Determine the deformation constraint by a first preset formula; The first preset formula is: Wherein, β is a deformation constraint, w is a first width of the prediction box, h is a first height of the prediction box, and w gt is the second width of the label box, and h gt is the second height of the annotation box.

2. The method for detecting deformation defects according to claim 1, characterized in that: The step of inputting the sample image into a preset deep learning model for training includes: Inputting a sample image into a composite backbone network, and extracting image features of the sample image through the composite backbone network; Inputting the image features into a feature pyramid network, and extracting feature representations of different scales of the image features through the feature pyramid network; The feature representations of the image features at different scales are input into a detection head, and the prediction box output by the detection head outputs the predicted position of the deformation defect image in the sample image and the defect type corresponding to the deformation defect image.

3. The method for detecting deformation defects according to claim 2, characterized in that: The step of extracting the image features of the sample image by using the composite backbone network includes: Extracting a first image feature larger than a first preset scale through a main backbone network; A second image feature smaller than a second preset scale is extracted through the auxiliary backbone network.

4. The method for detecting deformation defects according to claim 3, characterized in that: The extracting of image features smaller than a second preset scale by using an auxiliary backbone network includes: The image features smaller than the second preset scale are extracted by lightweight network using at least one ShuffleNet module.

5. The method for detecting deformation defects according to claim 3, characterized in that: After extracting the image features smaller than the second preset scale by the auxiliary backbone network, the method further includes: The first image feature and the second image feature are fused through an enhanced receptive field module to obtain the image feature of the sample image.

6. The method for detecting deformation defects according to claim 3, characterized in that: Before inputting the sample image into the composite backbone network, the method further includes: For each convolutional layer of the composite backbone network, a convolutional channel attention enhancement mechanism is integrated.

7. The method for detecting deformation defects according to claim 2, characterized in that: Before inputting the target image into a preset deformation defect detection model, the method further includes: Enhancing the contrast of the target image by using a histogram equalization algorithm; Before inputting the feature representations of the image features at different scales into the detection head, the method further includes: The anchor box of the detected head is updated by a k-means algorithm, wherein the predicted box is one of the anchor boxes.

8. A deformation defect detection device, the device is used to implement the steps of the deformation defect detection method according to any one of claims 1 to 7, characterized in that: include: An acquisition module, used for acquiring a target image with a deformation defect image; An input module, used for inputting the target image into a preset deformation defect detection model; An output module, configured to output the target position of the deformation defect image in the target image and / or the target defect type corresponding to the deformation defect image through the deformation defect detection model; Among them, the target position and the target defect type are displayed by the prediction box output by the deformation defect detection model, the target defect type is one of at least one preset defect type, and the loss function of the deformation defect detection model includes a deformation constraint, and the deformation constraint is used to constrain the deformation of the prediction box.

Images