Model building method, device, electronic device and storage medium

Abstract

A method, device, electronic device and storage medium for defect detection model establishment, applied to the field of detection technology, including: establishing an image set, the image set including different types of defects; building a defect detection network model based on the image set and Faster R-CNN; training the defect detection network model to obtain a classification detection model; wherein the classification detection model is used to perform defect detection on an image to be detected, and output the type, shape and position of the defect in the image to be detected. It can be used for defect detection of extreme sizes.

Description

Technical Field

[0001] The present disclosure relates to the field of image detection technology, and in particular to a model building method, device, electronic device and storage medium for defect detection. Background Art

[0002] In recent years, with the development of science and technology, the automation of product quality inspection has become one of the main trends in modern production development. Automated defect inspection is of great significance to reducing labor, improving production efficiency and promoting industry intelligence.

[0003] Traditional manual visual inspection methods have problems such as low inspection efficiency, slow inspection speed, low inspection accuracy, and inconsistent inspection standards. Due to the long inspection time, inspectors are prone to visual fatigue, which not only leads to high false detection and missed detection rates, but also causes great damage to the human eye. Therefore, defect detection methods based on machine vision and deep learning have emerged.

[0004] Deep learning algorithms for defect detection have been widely studied and are mainly used in the detection of surface defects such as fabrics, blankets, and floor tiles. In the existing technology, some use basic networks, region proposal networks, and Fast R-CNN detection networks to establish image classification models based on deep learning. During the training process, features are extracted from the input data at each iteration, without the need to manually design cumbersome image feature extractors. However, they only complete preliminary image data screening for conventional shape defects, without enhancing the features of defects such as fabrics. In addition, the convolution process will also lose smaller defect features, so the detection accuracy is not high. Some use fast cyclic convolutional neural network methods to search and locate defect areas, combine features and detectors into a framework, and automatically detect fabric defects. Among them, the RPN network is used to generate suggestion windows, and 300 suggestion windows are generated for each image. The number of suggestion windows is very large, resulting in a very slow detection speed. Summary of the invention

[0005] The main purpose of the present disclosure is to provide a model building method, device, electronic device and storage medium for defect detection, which have high detection accuracy and can be used for defect detection of extreme sizes.

[0006] To achieve the above objectives, the first aspect of the present disclosure provides a model building method for defect detection, comprising:

[0007] Creating an image set, wherein the image set includes different types of defects;

[0008] Building a defect detection network model based on the image set and Faster R-CNN;

[0009] Training the defect detection network model to obtain a classification detection model;

[0010] The classification detection model is used to perform defect detection on the image to be detected and output the type, shape and position of the defects in the image to be detected.

[0011] Optionally, establishing the image set includes:

[0012] Collecting at least one image, and marking defects in the image with a label to obtain the image set;

[0013] The label is marked with at least one of the type, shape and location information of the defect.

[0014] Optionally, building a defect detection network model based on the image set and Faster R-CNN includes:

[0015] Inputting the image set into a feature pyramid network to obtain a first defect feature map, wherein the feature pyramid network adopts a ResNet-50 feature extraction network;

[0016] The first defect feature map is input into the prior anchor generation network to build a defect detection network model based on Faster R-CNN.

[0017] Optionally, inputting the image set into a feature pyramid network to obtain a first defect feature map includes:

[0018] Inputting the image set into the feature pyramid network;

[0019] Performing convolution on the marked defects in the image set from bottom to top to obtain a preliminary defect feature map with successively decreasing sizes;

[0020] Perform 1*1 convolution dimension reduction on all preliminary defect feature maps to obtain intermediate defect feature maps;

[0021] All intermediate defect feature maps are upsampled from top to bottom, and are fused with adjacent intermediate defect feature maps of the next size to obtain first defect feature maps with successively reduced sizes.

[0022] Optionally, inputting the first defect feature map into a priori anchor generation network to construct a defect detection network model based on Faster R-CNN includes:

[0023] Generating candidate sub-regions in the first defect feature map with successively smaller sizes;

[0024] Get the true value box according to the location of the defect marked by the label in each image;

[0025] Compare the candidate sub-regions of each image with the ground truth box to filter out candidate anchors;

[0026] Adjusting the shape of the candidate anchors by regression so that the candidate anchors of each image are close to the true value frame, and obtaining a priori anchors corresponding to the first defect feature map whose size is successively reduced;

[0027] The first defect feature map whose size is reduced in sequence and the prior anchor corresponding to the first defect feature map are input into a preset classification network to determine the parameters of the defect detection network model.

[0028] Optionally, the classification network includes 1 pooling layer and 4 fully connected layers, and the size is sequentially reduced to input the first defect feature map and the prior anchor corresponding to the first defect feature map into a preset classification network, and determining the parameters of the defect detection network model includes:

[0029] Inputting the first defect feature map whose size is reduced in sequence and the prior anchor corresponding to the first defect feature map into the pooling layer to obtain feature maps with the same size and the prior anchor corresponding to the feature map;

[0030] Taking the feature maps of the same size and the prior anchors corresponding to the feature maps as inputs of a first fully connected layer, and obtaining outputs of the first fully connected layer;

[0031] Using the output of the first fully connected layer as the input of the second fully connected layer to obtain the output of the second fully connected layer;

[0032] Using the output of the second fully connected layer as the input of the third fully connected layer, so that the third fully connected layer outputs the position and shape information of the defect in each image through regression;

[0033] The output of the second fully connected layer is used as the input of the fourth fully connected layer, so that the fourth fully connected layer outputs the type information of the defect in each image through Softmax classification.

[0034] Optionally, during the process of training the defect detection network model, a cross entropy loss function and a shape prediction loss function of a priori anchor are used to constrain the training process of the defect detection network model.

[0035] A second aspect of an embodiment of the present disclosure provides a model building device for defect detection, comprising:

[0036] A building module, used for building an image set, wherein the image set includes different types of defects;

[0037] A building module for building a defect detection network model based on the image set and Faster R-CNN;

[0038] A training module, used for training the defect detection network model to obtain a classification detection model;

[0039] The classification detection model is used to perform defect detection on the image to be detected and output the type, shape and position of the defects in the image to be detected.

[0040] A third aspect of the present disclosure provides an electronic device, including:

[0041] A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the model building method for defect detection provided in the first aspect of the embodiment of the present disclosure is implemented.

[0042] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the model building method for defect detection provided in the first aspect of an embodiment of the present disclosure is implemented.

[0043] From the above embodiments of the present disclosure, it can be seen that the model establishment method, device, electronic device and storage medium for defect detection provided by the present disclosure fuse the features of the current layer with the features of the adjacent next layer through the multi-size feature fusion algorithm of the feature pyramid network, thereby solving the problem of defect feature information loss caused by only selecting the top-level feature map when extracting features through multi-layer convolution in traditional neural networks, enhancing the detail features of small defects, making defect feature extraction more accurate, and improving the sensitivity and detection ability of extreme shape defects such as small defects and slender defects. By selecting candidate sub-regions on the first defect feature map of each size, a smaller number of candidate anchors are obtained, and then the size of the candidate anchor is adjusted by regression, and finally the prior anchor closest to the defect feature size is obtained. The prior anchor determined by this method is not only suitable for defect detection of extreme sizes, but also solves the problem of fixed shape of anchors in the Faster R-CNN network, and effectively avoids the situation where the shape of the anchor is greatly different from the defect size. At the same time, the process of determining the prior anchor also greatly reduces the number of candidate anchors, effectively improving the detection speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 A schematic diagram of a flow chart of a method for establishing a model for defect detection provided by an embodiment of the present disclosure;

[0046] Figure 2 A schematic diagram of a feature pyramid network provided by an embodiment of the present disclosure;

[0047] Figure 3 A schematic diagram of a priori anchor generation network provided in one embodiment of the present disclosure;

[0048] Figure 4 A training flow chart of a classification detection model provided in one embodiment of the present disclosure;

[0049] Figure 5 A schematic diagram of the structure of a model building device for defect detection provided by an embodiment of the present disclosure;

[0050] Figure 6 A schematic diagram of the hardware structure of an electronic device is shown. DETAILED DESCRIPTION

[0051] In order to make the disclosure purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0052] The disclosed classification detection model includes a feature extraction network, a region of interest generation network, an R-CNN classification and positioning network, wherein the feature extraction network is a feature pyramid network, which not only avoids the loss of small feature information, but also supplements and enhances the defect features, thereby extracting the defect features more accurately. The region of interest generation network adopts a priori anchor generation network, and determines the position and shape of the priori anchor by comparing the candidate anchors with the true value boxes on feature maps of different sizes, thereby providing a more accurate position and shape of the priori anchor for training, and greatly reducing the number of candidate anchors at the same time, effectively improving the positioning accuracy and detection speed, and still having a significant effect on defects of extreme shapes.

[0053] See also Figure 1 , Figure 1 A flow chart of a method for establishing a model for defect detection provided by an embodiment of the present disclosure, the method mainly comprises the following steps:

[0054] S101, establishing an image set, wherein the image set includes different types of defects;

[0055] Specifically, there should be multiple images of each defect type, for example, 100, 200, etc., and the image size is not required. Because fabric defects cover more extreme shapes, the present disclosure takes fabric defects as an example. The types of fabric defects are divided into no defects, holes, stains, snagging, knots, pattern board jumps, hundred feet, coarse warp, coarse weft, broken warp, broken weft, sparse and dense gaps, wear marks, rolling marks, dead wrinkles, double weft, double warp, reed path, bad weft yarn, etc.

[0056] S102, building a defect detection network model based on the image set and Faster R-CNN;

[0057] S103, training the defect detection network model to obtain a classification detection model;

[0058] The classification detection model is used to perform defect detection on the image to be detected and output the type, shape and position of the defects in the image to be detected.

[0059] In one embodiment of the present disclosure, step S101 includes:

[0060] Collect at least one image, and mark defects in the image with a label to obtain the image set;

[0061] The label is marked with at least one of the type, shape and location information of the defect.

[0062] Optionally, a label library may be established based on labels. The label files in the label library are established in the COCO format. Taking fabric defects as an example, all collected fabric image data with defects constitute an image set.

[0063] In this embodiment, a defect detection network model is built based on an image set and Faster R-CNN, and then the defect detection network model is trained to obtain a classification detection model, which can effectively improve positioning accuracy and detection speed, and still has a significant effect on defects of extreme shapes.

[0064] In one embodiment of the present disclosure, step S102 includes:

[0065] S1021, inputting the image set into a feature pyramid network to obtain a first defect feature map, wherein the feature pyramid network adopts a ResNet-50 feature extraction network;

[0066] S1022: Input the first defect feature map into a priori anchor generation network to construct a defect detection network model based on Faster R-CNN.

[0067] Specifically, the ResNet-50 feature extraction network contains 49 convolutional layers and one fully connected layer. The collected fabric images are convolved from bottom to top to extract defect features with decreasing sizes. Except for the first layer which is a 7*7 convolution, the rest are 1*1 convolution and 3*3 convolution, and a series of feature maps of different sizes (in decreasing sizes) are obtained. For example, Figure 2 As shown, the preliminary defect feature maps C1, C2, and C3 of the 10th, 40th, and 49th layers can be selected for 1*1 convolution dimensionality reduction and / or upsampling. It should be noted that the present disclosure does not limit the number of layers and quantity of the selected preliminary defect feature maps, and the above is only an exemplary description.

[0068] In this embodiment, the feature extraction network is selected as a feature pyramid network, which not only avoids the loss of small feature information, but also supplements and enhances the defect features, so as to more accurately extract the defect features. The region of interest generation network adopts the prior anchor generation network, which can provide more accurate prior anchor position and shape for the training of the defect detection network, and greatly reduces the number of candidate anchors, effectively improves the positioning accuracy and detection speed, and still has a significant effect on defects of extreme shapes.

[0069] In one embodiment of the present disclosure, see Figure 2 , step S1021 includes:

[0070] Inputting the image set into the feature pyramid network;

[0071] Perform convolution on the marked defects in the image set from bottom to top to obtain preliminary defect feature maps with successively decreasing sizes;

[0072] Perform 1*1 convolution dimension reduction on all preliminary defect feature maps to obtain intermediate defect feature maps;

[0073] All intermediate defect feature maps are upsampled from top to bottom, and are fused with adjacent intermediate defect feature maps of the next size to obtain first defect feature maps with successively reduced sizes.

[0074] Understandable, see Figure 2 , Figure 2 C1, C2 and C3 are preliminary defect feature maps with decreasing sizes, and F1, F2 and F3 are first defect feature maps with decreasing sizes, wherein the corresponding preliminary defect feature map, intermediate defect feature map and first defect feature map have the same size ( Figure 2 The intermediate defect feature diagram is not shown in the figure), for example, the size of C1 is the same as F1, the size of C2 is the same as F2, and the size of C3 is the same as F3.

[0075] It can be understood that the first defect feature map includes all defect feature information in the preliminary defect feature map and the preliminary defect feature map of the next size adjacent to the preliminary defect feature map. Figure 2 , the preliminary defect feature map of the next size adjacent to C1 is C2, and the preliminary defect feature map of the next size adjacent to C2 is C3. F1 includes the defect feature information of C1 and C2, and F2 includes the feature information of C2 and C3. If C3 is the last preliminary defect feature map, since the last size C3 does not have an adjacent preliminary defect feature map of the next size, only 1*1 convolution dimensionality reduction is performed on C3 without the subsequent top-down upsampling. Therefore, F3 only includes all defect feature information of C3. If C3 has an adjacent preliminary defect feature map of the next size, normal 1*1 convolution dimensionality reduction and top-down upsampling are performed. It should be noted that Figure 2 This is only an illustrative description, and the present disclosure does not limit the number of preliminary defect feature maps of successively reduced sizes obtained by convolution, which may be 1, 2, 3, 4, and so on.

[0076] In this embodiment, the features of this layer are fused with the features of the adjacent next layer, which solves the problem of defect feature information loss caused by only selecting the top-level feature map when extracting features by multi-layer convolution in traditional neural networks, enhances the detail features of small defects, makes defect feature extraction more accurate, and improves the sensitivity and detection capability of extreme shape defects such as small defects and slender defects.

[0077] In one embodiment of the present disclosure, see Figure 3 , step S1022 includes:

[0078] Generating candidate sub-regions in the first defect feature map with successively smaller sizes;

[0079] Get the true value box according to the location of the defect marked by the label in each image;

[0080] Compare the candidate sub-regions of each image with the ground truth box to filter out candidate anchors;

[0081] Adjust the shape of the candidate anchor by regression, so that the candidate anchor of each image approaches the true value box, and obtain the prior anchor corresponding to the first defect feature map whose size is reduced in sequence;

[0082] The first defect feature map with the size reduced in sequence and the prior anchor corresponding to the first defect feature map are input into a preset classification network to determine the parameters of the defect detection network model.

[0083] Specifically, the prior anchor generation network includes two steps: position prediction and shape prediction. Position prediction is to predict the center point of the candidate anchor. The image of each channel (the number of channels can be set by yourself) in the first defect feature map with decreasing size is respectively subtracted from the image mean of all channels corresponding to the first defect feature map, and a probability map with the same size as the first defect feature map is obtained through 1*1 convolution and sigmoid function (the probability map includes the probability that each pixel in the first defect feature map is a defect). Compare the predefined threshold ΔL with the probability that each pixel is a defect, determine the pixels exceeding the threshold, and obtain several candidate sub-regions. The candidate sub-region containing the center point of the true value box is selected as the candidate anchor, and the center point of the candidate sub-region is the center point of the candidate anchor. Shape prediction is to predict the shape size of the prior anchor, map the defect truth box in the original image to the first defect feature map with decreasing size, and obtain the truth box on the first defect feature map. Through regression, adjust the width and height of the candidate anchor on the first defect feature map with decreasing size, and finally make the intersection and union ratio of the adjusted candidate anchor and the corresponding truth box on the first defect feature map as large as possible, which is the prior anchor corresponding to the first defect feature map of this size.

[0084] The threshold ΔL can be set according to actual conditions, such as 0.5, 0.6, .065, etc. The present disclosure does not limit this.

[0085] In this embodiment, the position and shape of the prior anchor are determined by comparing the candidate anchors with the true value box on the first defect feature map with successively decreasing sizes, so as to provide a more accurate position and shape of the prior anchor for training the defect detection network model, and at the same time greatly reduce the number of candidate anchors, effectively improve the positioning accuracy and detection speed, and still have a significant effect on defects of extreme shapes, and also solve the problem of fixed shape of the anchor in the Faster R-CNN network, and effectively avoid the situation where the shape of the anchor is greatly different from the defect size.

[0086] In one embodiment of the present disclosure, the classification network includes 1 pooling layer and 4 fully connected layers, the size is sequentially reduced, the first defect feature map and the prior anchor corresponding to the first defect feature map are input into the preset classification network, and the parameters of the defect detection network model are determined, including:

[0087] The first defect feature map whose size is reduced in sequence and the prior anchor corresponding to the first defect feature map are input into the pooling layer to obtain feature maps with the same size and the prior anchor corresponding to the feature map;

[0088] Using the feature maps of the same size and the prior anchors corresponding to the feature maps as inputs of a first fully connected layer, and obtaining outputs of the first fully connected layer;

[0089] The output of the first fully connected layer is used as the input of the second fully connected layer to obtain the output of the second fully connected layer;

[0090] The output of the second fully connected layer is used as the input of the third fully connected layer, so that the third fully connected layer outputs the location and shape information of the defect in each image through regression;

[0091] The output of the second fully connected layer is used as the input of the fourth fully connected layer, so that the fourth fully connected layer outputs the type information of defects in each image through Softmax classification.

[0092] In one of the embodiments of the present disclosure, during the process of training the defect detection network model, a cross entropy loss function and a shape prediction loss function of a priori anchor are used to constrain the training process of the defect detection network model.

[0093] Specifically, the image set in S101 can be divided into a training set and a test set, and the training set is input into the defect detection network model for training. In the process of training the defect detection network model, the cross entropy loss function and the shape prediction loss function of the prior anchor are used to constrain the training process of the defect detection network model. After the training is completed, the defect detection network model is tested using the test set.

[0094] The training set and the test set are divided according to a ratio of 9:1, 8:2 or 7:3, which is not limited in the present disclosure.

[0095] In one example, the initialization model used for training is a model trained on Microsoft's COCO target detection dataset. The parameter update method of the model is Momentum, the initial learning rate is 0.001, and after 30,000 iterations, the learning rate is 0.0001, the momentum coefficient is 0.9, the batch size is 16, and the IoU threshold for non-maximum suppression is 0.7.

[0096] Please refer to the above Figure 4 , Figure 4A training flow chart of a classification detection model provided for an embodiment of the present disclosure. The advantages of the present disclosure over the prior art are: through the multi-size feature fusion algorithm of the feature pyramid network, the features of the current layer are fused with the features of the adjacent next layer, which solves the problem of defect feature information loss caused by only selecting the top-level feature map when extracting features by multi-layer convolution in the traditional neural network, enhances the detail features of small defects, makes the defect feature extraction more accurate, and improves the sensitivity and detection ability of extreme shape defects such as small defects and slender defects. By selecting candidate sub-regions on the first defect feature map of each size, a smaller number of candidate anchors are obtained, and then the size of the candidate anchor is adjusted by regression, and finally the prior anchor closest to the defect feature size is obtained. The prior anchor determined by this method is not only suitable for defect detection of extreme sizes, but also solves the problem of fixed shape of the anchor in the Faster R-CNN network, and effectively avoids the situation where the shape of the anchor is greatly different from the defect size. At the same time, the process of determining the prior anchor also greatly reduces the number of candidate anchors, effectively improving the detection speed.

[0097] See also Figure 5 , Figure 5 : is a schematic diagram of a structure of a model building device for defect detection provided by an embodiment of the present disclosure. The device can be built into an electronic device. The device mainly includes:

[0098] A building module 501 is used to build an image set, where the image set includes different types of defects;

[0099] A building module 502 is used to build a defect detection network model based on the image set and Faster R-CNN;

[0100] A training module 503 is used to train the defect detection network model to obtain a classification detection model;

[0101] The classification detection model is used to perform defect detection on the image to be detected and output the type, shape and position of the defects in the image to be detected.

[0102] The establishment module 501 includes:

[0103] The acquisition submodule acquires at least one image;

[0104] A labeling submodule is used to label the defects in the image using labels to obtain the image set;

[0105] The label is marked with at least one of the type, shape and location information of the defect.

[0106] In one embodiment of the present disclosure, the building module 502 includes:

[0107] A first input submodule is used to input the image set into a feature pyramid network to obtain a first defect feature map, where the feature pyramid network adopts a ResNet-50 feature extraction network;

[0108] The second input submodule is used to input the first defect feature map into the prior anchor generation network to build a defect detection network model based on Faster R-CNN.

[0109] In one of the embodiments of the present disclosure, the first input submodule is specifically used to: input the image set into the feature pyramid network; perform convolution on the marked defects in the image set from bottom to top to obtain preliminary defect feature maps with successively decreasing sizes; perform 1*1 convolution dimensionality reduction on all preliminary defect feature maps to obtain intermediate defect feature maps; upsample all intermediate defect feature maps from top to bottom, and fuse them with adjacent intermediate defect feature maps of the next size to obtain first defect feature maps with successively decreasing sizes.

[0110] In one of the embodiments of the present disclosure, the second input submodule is specifically used to: generate candidate sub-regions in the first defect feature map with successively reduced sizes; obtain a true value frame according to the position of the defect marked by the label in each image; compare the candidate sub-region of each image with the true value frame to screen out candidate anchors; adjust the shape of the candidate anchor by regression so that the candidate anchor of each image approaches the true value frame to obtain a priori anchor corresponding to the first defect feature map with successively reduced sizes; input the first defect feature map with successively reduced sizes and the priori anchor corresponding to the first defect feature map into a preset classification network to determine the parameters of the defect detection network model.

[0111] In one embodiment of the present disclosure, the classification network includes 1 pooling layer and 4 fully connected layers, and the first defect feature map whose size is reduced in sequence and the prior anchor corresponding to the first defect feature map are input into a preset classification network, and the parameters of the defect detection network model are determined including: the first defect feature map whose size is reduced in sequence and the prior anchor corresponding to the first defect feature map are input into the pooling layer to obtain feature maps with the same size and the prior anchors corresponding to the feature maps; the feature maps with the same size and the prior anchors corresponding to the feature maps are used as inputs of the first fully connected layer to obtain the output of the first fully connected layer; the output of the first fully connected layer is used as inputs of the second fully connected layer to obtain the output of the second fully connected layer; the output of the second fully connected layer is used as inputs of the third fully connected layer, so that the third fully connected layer outputs the position and shape information of the defect in each image through regression; the output of the second fully connected layer is used as inputs of the fourth fully connected layer, so that the fourth fully connected layer outputs the type information of the defect in each image through Softmax classification.

[0112] In one of the embodiments of the present disclosure, during the process of training the defect detection network model, a cross entropy loss function and a shape prediction loss function of a priori anchor are used to constrain the training process of the defect detection network model.

[0113] For details not provided in the above embodiments, please refer to Figures 1 to 4 The relevant description of the illustrated embodiment will not be repeated here.

[0114] See also Figure 6 , Figure 6 A hardware structure diagram of an electronic device is shown.

[0115] The electronic device described in this embodiment includes:

[0116] The memory 61, the processor 62 and the computer program stored in the memory 61 and executable on the processor, when the processor executes the program, the aforementioned Figure 1 The model building method for defect detection is described in the illustrated embodiment.

[0117] Furthermore, the electronic device also includes:

[0118] At least one input device 63 ; at least one output device 64 .

[0119] The memory 61 , processor 62 , input device 63 and output device 64 are connected via a bus 65 .

[0120] The input device 63 may be a camera, a touch panel, a physical button or a mouse, etc. The output device 64 may be a display screen.

[0121] The memory 61 may be a high-speed random access memory (RAM) memory or a non-volatile memory, such as a disk memory. The memory 61 is used to store a set of executable program codes, and the processor 62 is coupled to the memory 61 .

[0122] Furthermore, the present disclosure also provides a computer-readable storage medium, which may be disposed in the electronic device in the above embodiments. Figure 5 The electronic device in the embodiment shown. The computer readable storage medium stores a computer program, which, when executed by the processor, implements the aforementioned Figure 1The model building method for defect detection described in the illustrated embodiment. Further, the computer storable medium may also be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0123] It should be noted that the functional modules in the various embodiments of the present disclosure may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0124] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product.

[0125] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0126] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0127] The above is a description of a model building method, device, electronic device and storage medium for defect detection provided by the present invention. For those skilled in the art, according to the ideas of the embodiments of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A model building method for defect detection, characterized in that: include: Creating an image set, wherein the image set includes different types of defects; Build a defect detection network model based on the image set and Faster R-CNN, The step of building a defect detection network model based on the image set and Faster R-CNN includes: inputting the image set into a feature pyramid network to obtain a first defect feature map, wherein the step of inputting the image set into a feature pyramid network to obtain a first defect feature map includes: Inputting the image set into the feature pyramid network; Performing convolution on the marked defects in the image set from bottom to top to obtain a preliminary defect feature map with successively decreasing sizes; Perform 1*1 convolution dimension reduction on all preliminary defect feature maps to obtain intermediate defect feature maps; All intermediate defect feature maps are upsampled from top to bottom, and are fused with adjacent intermediate defect feature maps of the next size to obtain first defect feature maps with successively smaller sizes; The defect detection network model based on the image set and Faster R-CNN also includes: Inputting the first defect feature map into a priori anchor generation network to construct a defect detection network model based on Faster R-CNN, wherein the priori anchor generation network includes two steps: position prediction and shape prediction; Training the defect detection network model to obtain a classification detection model; The classification detection model is used to perform defect detection on the image to be detected and output the type, shape and position of the defects in the image to be detected.

2. The model building method according to claim 1, characterized in that: The establishing of the image set comprises: Collecting at least one image, and marking defects in the image with a label to obtain the image set; The label is marked with at least one of the type, shape and location information of the defect.

3. The model building method according to claim 1, characterized in that: The defect detection network model based on the image set and FasterR-CNN also includes: The feature pyramid network adopts the ResNet-50 feature extraction network.

4. The model building method according to claim 3, characterized in that: The step of inputting the first defect feature map into a priori anchor generation network to construct a defect detection network model based on Faster R-CNN includes: Generating candidate sub-regions in the first defect feature map with successively smaller sizes; Get the true value box according to the location of the defect marked by the label in each image; Compare the candidate sub-regions of each image with the ground truth box to filter out candidate anchors; Adjusting the shape of the candidate anchors by regression so that the candidate anchors of each image are close to the true value frame, and obtaining a priori anchors corresponding to the first defect feature map whose size is successively reduced; The first defect feature map whose size is reduced in sequence and the prior anchor corresponding to the first defect feature map are input into a preset classification network to determine the parameters of the defect detection network model.

5. The model building method according to claim 4, characterized in that: The classification network includes 1 pooling layer and 4 fully connected layers, the size is sequentially reduced, the first defect feature map and the prior anchor corresponding to the first defect feature map are input into the preset classification network, and the parameters of the defect detection network model are determined, including: Inputting the first defect feature map whose size is reduced in sequence and the prior anchor corresponding to the first defect feature map into the pooling layer to obtain feature maps with the same size and the prior anchor corresponding to the feature map; Taking the feature maps of the same size and the prior anchors corresponding to the feature maps as inputs of a first fully connected layer, and obtaining outputs of the first fully connected layer; Using the output of the first fully connected layer as the input of the second fully connected layer to obtain the output of the second fully connected layer; Using the output of the second fully connected layer as the input of the third fully connected layer, so that the third fully connected layer outputs the position and shape information of the defect in each image through regression; The output of the second fully connected layer is used as the input of the fourth fully connected layer, so that the fourth fully connected layer outputs the type information of the defect in each image through Softmax classification.

6. The model building method according to any one of claims 1 to 5, characterized in that: In the process of training the defect detection network model, a cross entropy loss function and a shape prediction loss function of a priori anchor are used to constrain the training process of the defect detection network model.

7. A model building device for defect detection, characterized in that: include: A building module, used for building an image set, wherein the image set includes different types of defects; A building module for building a defect detection network model based on the image set and Faster R-CNN, The step of building a defect detection network model based on the image set and Faster R-CNN includes: inputting the image set into a feature pyramid network to obtain a first defect feature map, wherein the step of inputting the image set into a feature pyramid network to obtain a first defect feature map includes: Inputting the image set into the feature pyramid network; Performing convolution on the marked defects in the image set from bottom to top to obtain a preliminary defect feature map with successively decreasing sizes; Perform 1*1 convolution dimension reduction on all preliminary defect feature maps to obtain intermediate defect feature maps; All intermediate defect feature maps are upsampled from top to bottom, and are fused with adjacent intermediate defect feature maps of the next size to obtain first defect feature maps with successively smaller sizes; The defect detection network model based on the image set and Faster R-CNN also includes: Inputting the first defect feature map into a priori anchor generation network to construct a defect detection network model based on Faster R-CNN, wherein the priori anchor generation network includes two steps: position prediction and shape prediction; A training module, used for training the defect detection network model to obtain a classification detection model; The classification detection model is used to perform defect detection on the image to be detected and output the type, shape and position of the defects in the image to be detected.

8. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, each step in the model building method for defect detection described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the model building method for defect detection according to any one of claims 1 to 6 is implemented.

Images