A component defect detection method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of defect detection, and provides a component defect detection method and device, electronic equipment and a storage medium, the method comprising: determining a target component region image in a to-be-detected image; extracting a plurality of feature vectors from the target component region image; inputting the plurality of feature vectors into a pre-constructed multi-cluster center feature distribution model, determining a plurality of cluster center vectors, and classifying each feature vector based on the plurality of cluster center vectors; and determining whether the target component has defects based on the classified plurality of feature vectors. By using different cluster center vectors to classify each feature vector, more texture, illumination and other feature information is retained to represent the distribution of the features while ensuring the consistency of the semantic information, so that the defect detection can be realized only by using positive samples, the problem that there are few defect samples of key components in industrial equipment is solved, and the accuracy of defect detection is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of defect detection, in particular to a component defect detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] Component defect detection is a classic problem of image processing, which exists in many industrial applications. The main idea to solve this kind of problem is to extract the features of the defect area and the normal area, input them into the classifier for training and classification, and finally judge the data according to the decision value given by the classifier.

[0003] At present, the current non-negative sample defect detection method constructs a feature vector space, constructs a Gaussian distribution model or directly uses the entire feature vector set as the overall feature distribution model of the same local area in the normal image, and then sets a distribution threshold to divide the range of the overall feature distribution of the image. Finally, the distance between the local area features of the detection image and the feature distribution model is calculated to distinguish between normal and defects. However, the texture, color and other changes of the same area of the industrial key component image are not uniform, and contain rich feature information of different dimensions of the same category, which may cause the technical problem of inaccurate defect detection in the defect detection stage. Therefore, how to improve the accuracy of component defect detection has become a technical problem that cannot be ignored. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a component defect detection method and device, electronic equipment and storage medium, which classifies each feature vector by using different cluster center vectors, ensures the consistency of semantic information, retains more texture, illumination and other feature information to represent the distribution of features, thereby realizing defect detection only relying on positive samples, solving the problem of few defect samples and many defect types of key components in industrial equipment, and improving the accuracy of defect detection.

[0005] The embodiment of the present application provides a component defect detection method, which comprises the following steps:

[0006] An image to be detected is acquired, and a target component area image in the image to be detected is determined;

[0007] A plurality of feature vectors are extracted from the target component area image;

[0008] The plurality of feature vectors are input into a pre-constructed multi-cluster center feature distribution model, a plurality of cluster center vectors are determined, and each feature vector is classified based on the plurality of cluster center vectors;

[0009] Based on the plurality of classified feature vectors, it is determined whether the target component has defects.

[0010] In one possible implementation, acquiring the image to be detected and determining the target component region image in the image to be detected includes:

[0011] Obtain the component template image;

[0012] The target component in the image to be detected is initially located using the component template image, thereby determining the preliminary region image of the target component in the image to be detected;

[0013] The component template image and the preliminary region image of the target component are accurately matched using a feature detection algorithm to determine the region image of the target component.

[0014] In one possible implementation, the step of using the component template image to initially locate the target component in the image to be detected, and determining a preliminary region image of the target component in the image to be detected, includes:

[0015] The image to be detected and the component template image are input into a pre-trained neural network model to determine the similarity value between each image region in the image to be detected and the component template image.

[0016] The maximum similarity value is selected from among the multiple similarity values, and the image region corresponding to the maximum similarity value is determined as the preliminary region image of the target component.

[0017] In one possible implementation, the multi-cluster center feature distribution model is determined through the following steps:

[0018] Obtain the reference feature vector set corresponding to the standard component image;

[0019] A reference feature vector is selected from the set of reference feature vectors as the first cluster center vector, and the first distance from each reference feature vector in the set of reference feature vectors to the first cluster center vector is calculated.

[0020] The largest first distance is selected from multiple first distances, and the reference feature vector corresponding to the largest first distance is used as the second cluster center vector;

[0021] An initial multi-cluster center feature distribution model is constructed based on the first cluster center vector and the second cluster center vector;

[0022] The initial multi-cluster center feature distribution model is updated with cluster center vectors, and the multi-cluster center feature distribution model is determined using the multiple updated cluster center vectors each time.

[0023] In a possible implementation, the cluster center vector updating of the initial multi-cluster center feature distribution model, the determination of the multi-cluster center feature distribution model by using the plurality of updated cluster center vectors, comprises:

[0024] determining a target Euclidean distance of the first cluster center vector and the second cluster center vector;

[0025] determining a distance threshold based on a product of the target Euclidean distance and a scale factor;

[0026] calculating a second distance between each cluster center vector and a reference feature vector corresponding to a non-cluster center in the reference feature vector set, and selecting a maximum second distance from the plurality of second distances;

[0027] comparing the maximum second distance with the distance threshold;

[0028] if the maximum second distance is greater than the distance threshold, determining the reference feature vector corresponding to the maximum second distance as a new cluster center vector, and performing the cluster center vector updating of the initial multi-cluster center feature distribution model until the multi-cluster center feature distribution model is determined by the cluster center vector updating.

[0029] In a possible implementation, the input of the plurality of feature vectors into the pre-constructed multi-cluster center feature distribution model, the determination of the plurality of cluster center vectors, and the classification of each feature vector based on the plurality of cluster center vectors, comprises:

[0030] for each feature vector, calculating a third distance between the feature vector and each cluster center vector, determining a cluster center vector corresponding to a minimum third distance of the plurality of third distances of the feature vector as a target cluster center vector, and classifying the feature vector into a target category corresponding to the target cluster center vector.

[0031] In a possible implementation, the determination of whether the target component has a defect based on the classified plurality of feature vectors, comprises:

[0032] determining a target distance between each classified feature vector and a corresponding cluster center vector;

[0033] for each classified feature vector, determining whether the target distance of the classified feature vector is greater than a target feature distribution model boundary threshold; wherein the target feature distribution model boundary threshold is a threshold of a target cluster center vector corresponding to the classified feature vector;

[0034] if yes, the target component has a defect.

[0035] This application embodiment also provides a defect detection device for a component, the defect detection device comprising:

[0036] The target component region determination module is used to acquire the image to be detected and determine the target component region image in the image to be detected;

[0037] The feature extraction module is used to extract multiple feature vectors from the target component region image;

[0038] The feature classification module is used to input multiple feature vectors into a pre-constructed multi-cluster center feature distribution model, determine multiple cluster center vectors, and classify each feature vector based on the multiple cluster center vectors;

[0039] The defect determination module is used to determine whether the target component has a defect based on multiple feature vectors after classification.

[0040] This application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the defect detection method for the components described above are performed.

[0041] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the defect detection method for the component described above.

[0042] This application provides a method, apparatus, electronic device, and storage medium for defect detection of components. The defect detection method includes: acquiring an image to be detected and determining a target component region image in the image to be detected; extracting multiple feature vectors from the target component region image; inputting the multiple feature vectors into a pre-constructed multi-cluster center feature distribution model to determine multiple cluster center vectors, and classifying each feature vector based on the multiple cluster center vectors; and determining whether the target component has a defect based on the classified feature vectors. By using different cluster center vectors to classify each feature vector, while ensuring semantic information consistency, more feature information such as texture and lighting is retained to characterize the feature distribution, thereby achieving defect detection based solely on positive samples. This solves the problem of few defect samples and many defect types in key components of industrial equipment and improves the accuracy of defect detection.

[0043] In order to make the above objectives, characteristics and advantages of the present application more apparent, more comprehensible, the following preferred embodiments are specifically described in detail below, together with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments, and it should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0045] Figure 1 A flow chart of a component defect detection method provided by the embodiments of the present application;

[0046] Figure 2 A schematic diagram of a component defect detection method provided by the embodiments of the present application;

[0047] Figure 3 A structural schematic diagram of a component defect detection device provided by the embodiments of the present application;

[0048] Figure 4 A structural schematic diagram of a component defect detection device provided by the embodiments of the present application;

[0049] Figure 5 A structural schematic diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical solutions and advantages of the embodiments of the present application more apparent, the following will clearly and completely describe the technical solutions of the embodiments of the present application in combination with the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application only play a purpose of illustration and description, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn according to the actual proportion. The flow chart shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow chart can not be implemented in sequence, and the steps without logical context relationship can be reversed in sequence or implemented simultaneously. In addition, under the guidance of the content of the present application, one or more other operations can be added to the flow chart, or one or more operations can be removed from the flow chart.

[0051] In addition, the described embodiments are only some of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0052] In order to enable those skilled in the art to use the content of the present application, the following implementation is given in combination with a specific application scenario "detecting defects of components", and those skilled in the art can apply the general principles defined herein to other embodiments and application scenarios without departing from the spirit and scope of the present application.

[0053] The method, device, electronic equipment or computer readable storage medium of the embodiments of the present application can be applied to any scene requiring detection of defects of components, and the embodiments of the present application do not limit the specific application scenario. Any solution using the method, device, electronic equipment and storage medium provided by the embodiments of the present application for detecting defects of components is within the scope of protection of the present application.

[0054] First, the application scenarios applicable to the present application are introduced. The present application can be applied to the field of defect detection technology.

[0055] The current mainstream convolutional neural network-based defect detection algorithm first needs to collect the image of the detection object, and construct a defect detection data set through manual labeling and other methods. Then, a suitable convolutional neural network model is built, and the corresponding defect detection data set is used for training. Finally, the data to be detected is transmitted into the trained defect detection model, which can quickly realize accurate defect detection. The effect of the trained defect detection model depends on the quantity and quality of the defect data. However, in the field of industrial defects, the defect samples of some key components are few and the defect types are many, which has become a big problem that needs to be solved by the current mainstream defect detection algorithm.

[0056] It is found through research that, at the present stage, the current non-negative sample type defect detection method constructs a feature vector space, constructs a Gaussian distribution model or directly uses the entire feature vector set as the overall feature distribution model of the same local area in the normal image, then sets a distribution threshold to divide the range of the overall feature distribution of the image, and finally calculates the distance between the local area features of the detection image and the feature distribution model to distinguish between normal and defects. However, the texture, color and other changes of the same area of the industrial key component image are not uniform, and contain rich feature information of different dimensions of the same category, which can cause the technical problem of inaccurate defect detection in the defect detection stage. Therefore, how to improve the accuracy of component defect detection has become a technical problem that cannot be underestimated.

[0057] Based on this, the embodiment of the application provides a component defect detection method, which classifies each feature vector by using different cluster center vectors, ensures the consistency of semantic information, retains more texture, illumination and other feature information to represent the distribution of features, thereby realizing defect detection only relying on positive samples, solving the problem of few defect samples and many defect types of key components in industrial equipment, and improving the accuracy of defect detection.

[0058] Please refer to Figure 1 , Figure 1 A flowchart of a component defect detection method provided by the embodiment of the application is shown in FIG. 1. Figure 1 As shown in FIG. 1, the detection method provided by the embodiment of the application includes the following steps.

[0059] S101: Obtain a detection image, and determine a target component area image in the detection image.

[0060] In this step, the detection image is obtained, and the target component area image in the detection image is determined.

[0061] Here, the detection image can be an industrial inspection image. Because the components such as insulators, wire clamps, anti-vibration hammers, grading rings and fan-shaped adjusting plates account for a small proportion in the image, the target component area image in the detection image needs to be determined.

[0062] Here, the target component can be an insulator, a wire clamp, an anti-vibration hammer, a grading ring, a fan-shaped adjusting plate or other components.

[0063] In one possible implementation, the obtaining of the detection image and the determination of the target component area image in the detection image include:

[0064] Step A: Obtain a component template image.

[0065] Here, the component template image is obtained, and the component template image is an image of a defect-free component and is pre-set.

[0066] Step B: preliminarily positioning the target component in the to-be-detected image by using the component template image to determine a preliminary region image of the target component in the to-be-detected image.

[0067] Here, the target component in the to-be-detected image is coarsely positioned according to the component template image to determine a preliminary region image of the target component in the to-be-detected image.

[0068] In a possible implementation, the preliminarily positioning the target component in the to-be-detected image by using the component template image to determine a preliminary region image of the target component in the to-be-detected image includes:

[0069] Step a: inputting the to-be-detected image and the component template image into a pre-trained neural network model to determine a similarity value of each image region in the to-be-detected image and the component template image.

[0070] Here, the to-be-detected image and the component template image are input into the neural network model to determine a similarity value of each image region in the to-be-detected image and the component template image.

[0071] The neural network model is composed of two weight-shared convolutional neural networks, maps the input to-be-detected image and component template image to a high-dimensional feature space, then calculates the similarity of the to-be-detected image and the component template image using a contrast loss, and finally obtains the similarity value of the to-be-detected image and the component template image.

[0072] Since the input component template image and the to-be-detected image are of different scales, the similarity value of each image region in the to-be-detected image and the component template image can be obtained after being input into the neural network model.

[0073] Step b: screening a maximum similarity value from the plurality of similarity values, and determining an image region corresponding to the maximum similarity value as the preliminary region image of the target component.

[0074] Here, the maximum similarity value is screened from the plurality of determined similarity values, and an image region corresponding to the maximum similarity value is determined as the preliminary region image of the target component.

[0075] Since the input component template image and the to-be-detected image are of different scales, the greater the similarity value of each image region in the to-be-detected image and the component template image obtained after being input into the neural network model, the greater the probability that the region contains the component.

[0076] In specific embodiments, small sample target detection is performed according to a pre-trained neural network model, and manual annotation of the component template image is required to complete the coarse positioning of the target component in the image to be detected. The neural network model maps the input two images to be detected and the component template image into a high-dimensional feature space, then calculates the similarity of the two using a contrast loss, and finally obtains the similarity relationship of the two images to be detected and the component template image. Since the scale of the input component template image and the image to be detected is different, the similarity relationship between each image region in the detection image and the component template image can be obtained after inputting the neural network model, and the higher the similarity, the greater the probability that the region contains the component. Thus, small sample target positioning can be realized by annotating a small number of component template images to achieve target positioning, which not only improves the detection speed, but also meets the demand for coarse positioning of component regions in industrial inspection images.

[0077] Step C: using a feature detection algorithm to accurately match the component template image and the preliminary region image of the target component to determine the target component region image.

[0078] Here, the component template image and the preliminary region image of the target component are accurately matched using a point, line, and surface or other feature detection algorithm to determine the target component region image.

[0079] Wherein, the component template image and the preliminary region image of the target component are accurately matched using a feature point, feature line, and feature surface feature detection algorithm image to obtain the target component region image.

[0080] Wherein, the target component region image is the accurate region of the target component in the image to be detected, and the preliminary region image of the target component is the coarse positioning region of the target component in the image to be detected.

[0081] In specific embodiments, in industrial inspection images, key components such as insulators, wire clamps, anti-vibration hammers, grading rings, and fan-shaped adjustment plates have a small proportion in the image and their positions and backgrounds change, so direct image registration using algorithms has poor results. In this scheme, the determination of the target component region image in the image to be detected is completed by using two parts of the similar region registration small sample target detection and the feature point, line, surface or other modal type feature detection algorithm registration. Using small sample target detection, coarse positioning of the target component region is realized, and then the feature point, line, surface or other modal type feature detection algorithm is used to finely register the target component region image to determine the target component region image.

[0082] S102: Extract a plurality of feature vectors from the target component region image.

[0083] In this step, a plurality of feature vectors are extracted from the target component region image.

[0084] In specific embodiments, due to the influence of factors such as viewing angle and illumination, each image region has the characteristics of rich texture and variable features. The present scheme uses a convolutional neural network with strong feature extraction capability for pre-training, thereby realizing feature extraction of the target component region image. The obtained features can be divided into three levels according to the feature level, i.e., low-level features, middle-level features, and deep-level features. Since the feature dimension of the low-level features represents too much information, dimension disaster problem is easily caused when performing feature distribution learning. The deep-level features are affected by the pre-training classification task, and have less semantic information and are more inclined to represent the classification semantic information of the detection object. Therefore, the middle-level features extracted by the pre-training convolutional neural network are finally selected as the feature vectors.

[0085] S103: inputting the plurality of feature vectors into a pre-constructed multi-cluster center feature distribution model, determining a plurality of cluster center vectors, and classifying each feature vector based on the plurality of cluster center vectors.

[0086] In this step, the feature vectors are input into the multi-cluster center feature distribution model, a plurality of cluster center vectors are determined, and each feature vector is classified according to the plurality of cluster center vectors. Different cluster center vectors are used to classify the feature vectors, which not only ensures the consistency of semantic information but also retains more texture, illumination, and other feature information to represent the feature distribution.

[0087] In one possible implementation, the multi-cluster center feature distribution model is determined by the following steps:

[0088] (1) obtaining a reference feature vector set corresponding to a standard component image.

[0089] Here, the reference feature vector set corresponding to the standard component image is obtained.

[0090] (2) selecting a reference feature vector in the reference feature vector set as a first cluster center vector, and calculating a first distance from each reference feature vector in the reference feature vector set to the first cluster center vector.

[0091] Here, a reference feature vector in the reference feature vector set is selected as a first cluster center vector, and a first distance from each reference feature vector in the reference feature vector set to the first cluster center vector is calculated.

[0092] Here, the first distance from each reference feature vector in the reference feature vector set except the reference feature vector corresponding to the first cluster center vector to the first cluster center vector can be calculated by using the Euclidean distance formula.

[0093] (3):In a plurality of the first distance screening out the maximum first distance, and the maximum first distance corresponding to the reference feature vector as a second clustering center vector.

[0094] Here, in a plurality of the first distance screening out the maximum first distance, and the maximum first distance corresponding to the reference feature vector as a second clustering center vector.

[0095] (4):Based on the first clustering center vector and the second clustering center vector to build an initial multi clustering center feature distribution model.

[0096] Here, the first clustering center vector C1 and the second clustering center vector C2 to build an initial multi clustering center feature distribution model.

[0097] (5):The initial multi clustering center feature distribution model is updated for clustering center vector, using each updated multiple clustering center vector to determine the multi clustering center feature distribution model.

[0098] Here, the initial multi clustering center feature distribution model is updated for clustering center vector, using each updated multiple clustering center vector to determine the multi clustering center feature distribution model.

[0099] Here, the updated multiple clustering center vector contains the clustering center vector determined before.

[0100] Here, the scheme proposes a multi clustering center feature distribution learning strategy. In the normal industrial key components data set, the feature category of the same image region may belong to the component, and may belong to the background. After using the feature extraction strategy, the consistency of the feature category of the same image region is ensured to a certain extent.

[0101] S1031: determine the target Euclidean distance of the first clustering center vector and the second clustering center vector.

[0102] Here, the target Euclidean distance of the first clustering center vector and the second clustering center vector is determined.

[0103] In which, the target Euclidean distance of the first clustering center vector and the second clustering center vector is determined by the following formula:

[0104] D = ||c1-c2||;

[0105] In which, c1 is the first clustering center vector, c2 is the second clustering center vector, and D is the target Euclidean distance of the first clustering center vector and the second clustering center vector.

[0106] S1032: Determine a distance threshold based on the product of the target Euclidean distance and the scale factor.

[0107] Here, the distance threshold is determined based on the product of the target Euclidean distance and the scale factor.

[0108] Here, the scale factor can be set to 0.1, and the setting method of the scale factor is not limited.

[0109] S1033: Calculate the second distance between each cluster center vector and the reference feature vector corresponding to the non-cluster center in the reference feature vector set, and select the maximum second distance from the plurality of second distances.

[0110] Here, the second distance between each cluster center vector and the reference feature vector corresponding to the non-cluster center in the reference feature vector set is calculated, and the maximum second distance is selected from the plurality of second distances.

[0111] Here, the second distance is the minimum distance between each cluster center vector and the non-cluster center vector.

[0112] Here, the second distance between the first cluster center vector and the second cluster center vector and the reference feature vector corresponding to the non-cluster center in the reference feature vector set is calculated, a cluster center minimum distance set is constructed, and the maximum second distance in the cluster center minimum distance set is selected.

[0113] S1034: Compare the maximum second distance with the distance threshold.

[0114] Here, the maximum second distance is compared with the distance threshold.

[0115] S1035: If greater, the reference feature vector corresponding to the maximum second distance is determined as a new cluster center vector, and the cluster center vector update of the initial multi-cluster center feature distribution model is performed until the multi-cluster center feature distribution model is determined.

[0116] Here, if the maximum second distance is greater than the distance threshold, the reference feature vector corresponding to the maximum second distance is determined as a new cluster center vector, and the cluster center vector update of the initial multi-cluster center feature distribution model is performed, and the distance between the new cluster center vector, the first cluster center vector and the second cluster center vector and the reference feature vector corresponding to the non-cluster center in the reference feature vector set is calculated, the maximum distance is selected, if the maximum distance is greater than the distance threshold, the reference feature vector corresponding to the maximum distance is determined as a new distance center vector, the above calculation is repeated, and when the maximum distance is less than the distance threshold, the multi-cluster center feature distribution model is determined.

[0117] In a specific embodiment, the Euclidean distance between each cluster center vector and non-cluster center vectors is calculated, and the minimum distances calculated for each cluster center vector are summarized to construct a set of minimum distances for cluster center vectors. Then, the maximum distance in the set of minimum distances for cluster centers is selected. If this maximum distance is greater than a distance threshold, its corresponding reference feature vector is set as the new cluster center, thereby completing the construction of the multi-cluster center feature distribution model.

[0118] In one possible implementation, the step of inputting the multiple feature vectors into a pre-constructed multi-cluster center feature distribution model to determine multiple cluster center vectors, and classifying each feature vector based on the multiple cluster center vectors, includes:

[0119] For each feature vector, the third distance from the feature vector to each cluster center vector is calculated. The cluster center corresponding to the smallest third distance among the multiple third distances corresponding to the feature vector is determined as the target cluster center vector. The feature vector is then classified into the target category corresponding to the target cluster center vector.

[0120] Here, the third distance from each feature vector to each cluster center vector is calculated. The cluster center vector corresponding to the smallest third distance among the multiple third distances corresponding to the feature vector is determined as the target cluster center vector, and the feature vector is classified into the target category corresponding to the target cluster center vector.

[0121] Here, the third distance can be determined using the Euclidean distance formula.

[0122] This process involves calculating the distance between the feature vector and each cluster center vector, and then using the minimum distance principle to classify each feature vector into different cluster center categories, thus completing the inter-class classification task. Using different cluster center vectors to classify feature vectors preserves more texture and lighting information to characterize the feature distribution while maintaining semantic consistency.

[0123] S104: Based on the multiple feature vectors after classification, determine whether the target component has defects.

[0124] In this step, based on the multiple feature vectors after classification, it is determined whether the target component has defects.

[0125] In one possible implementation, determining whether the target component has a defect based on multiple classified feature vectors includes:

[0126] S1041: Determine the target distance from each classified feature vector to the corresponding cluster center vector.

[0127] Here, the target distance of each classified feature vector to the corresponding cluster center vector is determined.

[0128] S1042: For each classified feature vector, it is determined whether the target distance of the classified feature vector is greater than the target feature distribution model boundary threshold value; wherein the target feature distribution model boundary threshold value is a threshold value of the target cluster center vector corresponding to the classified feature vector.

[0129] Here, for each classified feature vector, it is determined whether the target distance of the target distance of the classified feature vector to the corresponding cluster center vector is greater than the target feature distribution model boundary threshold value.

[0130] Wherein, the target feature distribution model boundary threshold value is a threshold value of the target cluster center vector corresponding to the classified feature vector.

[0131] Wherein, the target feature distribution model boundary threshold value is determined by the following formula:

[0132]

[0133] Wherein, S is the target feature distribution model boundary threshold value, i is the reference feature vector corresponding to the maximum distance between the target cluster center vector and the reference feature vector in the corresponding category, f(xi) is the i-th reference feature vector extracted by the convolutional neural network, a is the feature distribution model boundary threshold tolerance, and c is the target cluster center vector corresponding to the classified feature vector.

[0134] Wherein, the target feature distribution model boundary threshold value corresponding to different classified feature vectors is different and needs to be determined according to the above formula.

[0135] S1043: If yes, the target component has defects.

[0136] Here, if the target distance of any one of the classified feature vectors is greater than the target feature distribution model boundary threshold value, the target component has defects, and if the target distance of the classified feature vectors is greater than the target feature distribution model boundary threshold value, the target component has no defects.

[0137] Here, the target component can also be determined to have defects by the following method: after the feature distribution model is constructed, most of the current negative sample defect detection algorithms calculate the distance between each local region feature in the detection image and the corresponding region feature distribution model as the final defect score. In the defect score map, the higher the score of a region, the greater the possibility that the region is defective.

[0138] In specific embodiments, in the field of industrial defect detection, the regions in the images of industrial key components are different, the corresponding feature categories are quite different, and the defect score thresholds calculated by the ROC curves of different categories of defects are also different. Even with prior information of defect regions, using the defect score thresholds calculated by the ROC curves, the effect of defect detection of industrial key components is still poor. Therefore, a multi-cluster center feature distribution model boundary threshold determination strategy is proposed for the final defect detection. A tolerance α is set, the maximum value of the distance between the target cluster center vector corresponding to the feature vector and the reference feature vector in the corresponding category is selected, and the corresponding feature distribution model boundary threshold S can be obtained by the formula. In the defect detection stage, if the Euclidean distance between the feature vector of the detection object and the corresponding target cluster center vector is greater than the boundary threshold S, it is judged as a defect. The scheme provides a multi-cluster center feature distribution model boundary threshold determination strategy, which uses different defect score thresholds for detection images and can adapt to the feature difference transformation of industrial components.

[0139] In specific embodiments, please refer to Figure 2 , Figure 2 is a schematic diagram of a component defect detection method provided by the embodiments of the present application. As Figure 2 shown, the image to be detected and the component template image are input into the neural network model, the similarity values of each image region in the image to be detected and the component template image are determined, the maximum similarity value is selected from the determined multiple similarity values, and the image region corresponding to the maximum similarity value is determined as the preliminary region image of the target component. The point, line and surface feature detection algorithm is used to accurately match the component template image and the preliminary region image of the target component, the target component region image is determined, and multiple feature vectors are extracted from the target component region image by using the convolutional neural network. The feature vectors are input into the multi-cluster center feature distribution model, multiple cluster center vectors are determined, and each feature vector is classified according to the multiple cluster center vectors. According to the classified multiple feature vectors, it is determined whether the target component has defects.

[0140] The defect detection method of the component provided in the embodiment of the present application comprises: acquiring a to-be-detected image, and determining a target component region image in the to-be-detected image; extracting a plurality of feature vectors from the target component region image; inputting the plurality of feature vectors into a pre-constructed multi-cluster center feature distribution model, determining a plurality of cluster center vectors, and classifying each feature vector based on the plurality of cluster center vectors; and determining whether the target component has a defect based on the classified plurality of feature vectors. By using different cluster center vectors to classify each feature vector, more texture, illumination and other feature information are retained to represent the distribution of the feature while ensuring the consistency of the semantic information, so that the defect detection can be realized only by using a positive sample, the problem of few defect samples and many defect types of a key component in an industrial equipment is solved, and the accuracy of the defect detection is improved.

[0141] Please refer to Figure 3 、 Figure 4 , Figure 3 Figure 1 is a structural schematic diagram of a defect detection device of a component provided in the embodiment of the present application; Figure 4 Figure 2 is another structural schematic diagram of the defect detection device of the component provided in the embodiment of the present application. As shown in Figure 3 The defect detection device 300 of the component comprises:

[0142] A target component region determination module 310 is configured to acquire a to-be-detected image, and determine a target component region image in the to-be-detected image.

[0143] A feature extraction module 320 is configured to extract a plurality of feature vectors from the target component region image.

[0144] A feature classification module 330 is configured to input the plurality of feature vectors into a pre-constructed multi-cluster center feature distribution model, determine a plurality of cluster center vectors, and classify each feature vector based on the plurality of cluster center vectors.

[0145] A defect determination module 340 is configured to determine whether the target component has a defect based on the classified plurality of feature vectors.

[0146] Further, when the target component region determination module 310 is used to acquire a to-be-detected image and determine a target component region image in the to-be-detected image, the target component region determination module 310 is specifically configured to:

[0147] acquire a component template image;

[0148] The component template image is used to preliminarily locate the target component in the to-be-detected image, and a preliminary region image of the target component in the to-be-detected image is determined.

[0149] The feature detection algorithm is used to accurately match the component template image and the preliminary region image of the target component, and a target component region image is determined.

[0150] Further, when the component template image is used to preliminarily locate the target component in the to-be-detected image, and a preliminary region image of the target component in the to-be-detected image is determined, the target component region determination module 310 is specifically configured to:

[0151] The to-be-detected image and the component template image are input into a pre-trained neural network model to determine a similarity value of each image region in the to-be-detected image and the component template image.

[0152] Among a plurality of similarity values, a maximum similarity value is screened out, and an image region corresponding to the maximum similarity value is determined as the preliminary region image of the target component.

[0153] Further, as shown in Figure 4 The component defect detection device 300 further includes a model training module 350, and the model training module 350 is configured to:

[0154] A reference feature vector set corresponding to a standard industrial component image is acquired.

[0155] A first feature vector in the reference feature vector set is screened out as a first clustering center vector, and a first distance from each reference feature vector in the reference feature vector set to the first clustering center vector is calculated.

[0156] Among a plurality of first distances, a maximum first distance is screened out, and a reference feature vector corresponding to the maximum first distance is taken as a second clustering center vector.

[0157] Based on the first clustering center vector and the second clustering center vector, an initial multi-clustering center feature distribution model is constructed.

[0158] The clustering center vector of the initial multi-clustering center feature distribution model is updated, and the multi-clustering center feature distribution model is determined by using a plurality of clustering center vectors after each update.

[0159] Further, the model training module 350 is specifically configured to:

[0160] determine the target Euclidean distance of the first cluster center vector and the second cluster center vector;

[0161] determine the distance threshold based on the product of the target Euclidean distance and the scale factor;

[0162] calculate the second distance between each cluster center vector and the reference feature vector corresponding to the non-cluster center in the reference feature vector set, and select the maximum second distance from the plurality of second distances;

[0163] compare the maximum second distance with the distance threshold;

[0164] if the maximum second distance is greater than the distance threshold, determine the reference feature vector corresponding to the maximum second distance as a new cluster center vector, and perform the cluster center vector update on the initial multi-cluster center feature distribution model until the multi-cluster center feature distribution model is determined.

[0165] Further, the feature classification module 330 is specifically configured to:

[0166] for each feature vector, calculate the third distance between the feature vector and each cluster center vector, determine the cluster center vector corresponding to the minimum third distance of the plurality of third distances of the feature vector as a target cluster center vector, and classify the feature vector into a target category corresponding to the target cluster center vector.

[0167] Further, the defect determination module 340 is specifically configured to:

[0168] determine the target distance between each classified feature vector and the corresponding cluster center vector;

[0169] for each classified feature vector, determine whether the target distance of the classified feature vector is greater than a target feature distribution model boundary threshold; wherein the target feature distribution model boundary threshold is a threshold of the target cluster center vector corresponding to the classified feature vector.

[0170] If so, then the target component has a defect.

[0171] The defect detection device for components provided in this application includes: a target component region determination module, used to acquire an image to be detected and determine a target component region image in the image to be detected; a feature extraction module, used to extract multiple feature vectors from the target component region image; a feature classification module, used to input the multiple feature vectors into a pre-constructed multi-cluster center feature distribution model, determine multiple cluster center vectors, and classify each feature vector based on the multiple cluster center vectors; and a defect determination module, used to determine whether the target component has a defect based on the classified multiple feature vectors. By classifying each feature vector using different cluster center vectors, while ensuring semantic information consistency, more feature information such as texture and lighting is retained to characterize the feature distribution, thereby achieving defect detection based solely on the sample. This solves the problem of few defect samples and many defect types in key components of industrial equipment and improves the accuracy of defect detection.

[0172] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 500 includes a processor 510, a memory 520, and a bus 530.

[0173] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 and the memory 520 communicate via the bus 530. When the machine-readable instructions are executed by the processor 510, they can perform the operations described above. Figure 1 The steps of the defect detection method for the component in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0174] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the defect detection method for the component in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0175] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0176] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. The described device embodiments are merely schematic, for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, or the among different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0177] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0178] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit.

[0179] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application or the part of the technical solutions that make essential contributions to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0180] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit the same. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that any skilled person in the art can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some of the technical features, within the technical scope disclosed by the present application. The modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of defect detection of a component, characterized by, The defect detection method comprises: acquiring a to-be-detected image and determining a target component region image in the to-be-detected image; extracting a plurality of feature vectors from the target component region image; inputting the plurality of feature vectors into a pre-constructed multi-cluster center feature distribution model, determining a plurality of cluster center vectors, and classifying each of the feature vectors based on the plurality of cluster center vectors; determining whether the target component has a defect based on the classified plurality of feature vectors; The multi-cluster center feature distribution model is determined by the following steps: acquiring a reference feature vector set corresponding to a standard industrial component image; filtering out a first feature vector as a first cluster center vector in the reference feature vector set, and calculating a first distance from each reference feature vector in the reference feature vector set to the first cluster center vector; filtering out a maximum first distance from the plurality of first distances, and taking the reference feature vector corresponding to the maximum first distance as a second cluster center vector; constructing an initial multi-cluster center feature distribution model based on the first cluster center vector and the second cluster center vector; updating the cluster center vectors of the initial multi-cluster center feature distribution model, and determining the multi-cluster center feature distribution model by using the plurality of cluster center vectors updated each time, comprising: determining a target Euclidean distance of the first cluster center vector and the second cluster center vector; determining a distance threshold based on the product of the target Euclidean distance and a proportionality coefficient; calculating a second distance of each cluster center vector to a reference feature vector corresponding to a non-cluster center in the reference feature vector set, and filtering out a maximum second distance from the plurality of second distances; comparing the maximum second distance with the distance threshold; if greater, determining the reference feature vector corresponding to the maximum second distance as a new cluster center vector, and updating the cluster center vectors of the initial multi-cluster center feature distribution model until the cluster center vector updating is completed to determine the multi-cluster center feature distribution model.

2. The defect detection method according to claim 1, wherein The acquisition of the to-be-detected image and the determination of the target component region image in the to-be-detected image comprise: acquiring a component template image; preliminarily positioning the target component in the to-be-detected image by using the component template image, and determining a preliminary region image of the target component in the to-be-detected image; precisely matching the component template image and the preliminary region image of the target component by using a feature detection algorithm, and determining the target component region image.

3. The defect detection method according to claim 2, characterized by, The preliminary positioning of the target component in the to-be-detected image by using the component template image and the determination of the preliminary region image of the target component in the to-be-detected image comprise: inputting the to-be-detected image and the component template image into a pre-trained neural network model to determine a similarity value of each image region in the to-be-detected image and the component template image; Screening out a maximum similarity value from the plurality of similarity values, and determining an image region corresponding to the maximum similarity value as a preliminary region image of the target component.

4. The defect detection method of claim 1, wherein The inputting of the plurality of feature vectors into the pre-constructed multi-cluster center feature distribution model, the determination of a plurality of cluster center vectors, and the classification of each of the feature vectors based on the plurality of cluster center vectors include: For each of the feature vectors, a third distance of the feature vector to each of the cluster center vectors is calculated, a cluster center vector corresponding to a minimum third distance of the plurality of third distances of the feature vector is determined as a target cluster center vector, and the feature vector is classified into a target category corresponding to the target cluster center vector.

5. The defect detection method of claim 1, wherein The determination of whether the target component has a defect based on the classified plurality of feature vectors includes: A target distance of each of the classified feature vectors to a corresponding cluster center vector is determined. For each of the classified feature vectors, it is determined whether the target distance of the classified feature vector is greater than a target feature distribution model boundary threshold value; the target feature distribution model boundary threshold value is a threshold value of the target cluster center vector corresponding to the classified feature vector. If yes, the target component has a defect.

6. A defect detection apparatus of a component, characterized by, The defect detection device includes: A target component region determination module configured to acquire a to-be-detected image and determine a target component region image in the to-be-detected image; A feature extraction module configured to extract a plurality of feature vectors from the target component region image; A feature classification module configured to input the plurality of feature vectors into a pre-constructed multi-cluster center feature distribution model, determine a plurality of cluster center vectors, and classify each of the feature vectors based on the plurality of cluster center vectors; A defect determination module configured to determine whether the target component has a defect based on the classified plurality of feature vectors. The defect detection device of the component further includes a model training module configured to determine the multi-cluster center feature distribution model by the following steps: A reference feature vector set corresponding to a standard industrial component image is acquired. A first feature vector is screened out from the reference feature vector set as a first cluster center vector, and a first distance of each reference feature vector in the reference feature vector set to the first cluster center vector is calculated. A maximum first distance is screened out from the plurality of first distances, and a reference feature vector corresponding to the maximum first distance is taken as a second cluster center vector. An initial multi-cluster center feature distribution model is constructed based on the first cluster center vector and the second cluster center vector. The cluster center vector of the initial multi-cluster center feature distribution model is updated, and the multi-cluster center feature distribution model is determined by using the plurality of cluster center vectors after each update, including: A target Euclidean distance of the first cluster center vector and the second cluster center vector is determined. A distance threshold value is determined based on a product of the target Euclidean distance and a proportionality coefficient. calculating a second distance between each cluster center vector and a reference feature vector corresponding to a non-cluster center in the set of reference feature vectors, screening a maximum second distance from a plurality of the second distances; comparing the maximum second distance with the distance threshold value; if greater, determining the reference feature vector corresponding to the maximum second distance as a new cluster center vector, and performing cluster center vector updating on the initial multi-cluster center feature distribution model until the cluster center vector updating ends to determine the multi-cluster center feature distribution model.

7. An electronic device, comprising: comprise: a processor, a memory and a bus, the memory storing machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to perform the steps of the defect detection method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of the defect detection method of any one of claims 1 to 5.

Images