Defect detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a defect detection method and device, electronic equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining feature image data of a to-be-detected object; performing feature extraction on the feature image data through a pre-trained feature extraction network to obtain a first feature histogram in the to-be-detected object; and comparing the first feature histogram with a preset sample feature histogram, and determining a defect detection result of the to-be-detected object. According to the embodiment of the invention, the accuracy of defect detection is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a defect detection method, device, electronic device and storage medium. Background Art

[0002] Industrial manufacturing often requires quality inspection of semi-finished and finished products, ensuring they are free of defects before proceeding to the next step. Related technologies often only detect local pixel variations in textures, meaning they can detect structural defects but not logical ones. This results in missed defects and inaccurate overall defect detection. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a defect detection method, device, electronic device, and storage medium to improve the accuracy of defect detection. The specific technical solutions are as follows:

[0004] In a first aspect, an embodiment of the present application provides a defect detection method, comprising:

[0005] Acquire characteristic image data of the object to be detected;

[0006] Performing feature extraction on the feature image data using a pre-trained feature extraction network to obtain a first feature histogram of the object to be detected;

[0007] The first feature histogram is compared with a preset sample feature histogram to determine a defect detection result of the object to be detected.

[0008] In one embodiment of the present application, extracting features from the feature image data using a pre-trained feature extraction network to obtain a first feature histogram of the object to be detected includes:

[0009] Performing feature extraction on the feature image data using a pre-trained feature extraction network to obtain first feature channel data of the object to be detected;

[0010] According to the preset feature channel weights, the first feature channel data of the object to be detected is counted to obtain a first feature histogram of the object to be detected.

[0011] In one embodiment of the present application, before acquiring the characteristic image data of the object to be detected, the method further includes:

[0012] Obtaining sample image data of a plurality of sample objects;

[0013] Inputting the sample image data into an image recognition network to be trained, obtaining image features of the sample image data through a feature extraction network in the image recognition network, and obtaining prediction results of the sample image data through a feature classification network in the image recognition network;

[0014] According to the labeled true value of the sample image data and the prediction result, the parameters of the image recognition network to be trained are adjusted to obtain a trained image recognition network, wherein the feature extraction network in the trained image recognition network is the pre-trained feature extraction network.

[0015] In one embodiment of the present application, after obtaining the pre-trained feature extraction network, the method further includes:

[0016] Performing feature extraction on each of the sample image data using the pre-trained feature extraction network to obtain sample feature channel data of each of the sample image data;

[0017] Obtaining a contribution of each of the characteristic channels according to fluctuations of each of the characteristic channels in the sample characteristic channel data;

[0018] Calculating the feature channel weight of each feature channel based on the contribution of each feature channel;

[0019] The preset sample feature histogram is obtained according to the feature channel weights and the sample feature channel data.

[0020] In one embodiment of the present application, obtaining the contribution of each feature channel according to the fluctuation of each feature channel in the sample feature channel data includes:

[0021] Calculating the variance of each characteristic channel in the sample characteristic channel data, and sorting the fluctuation of each characteristic channel according to the size of the variance to obtain the fluctuation order of each characteristic channel;

[0022] According to the fluctuation order, the contribution of each characteristic channel is obtained.

[0023] In one embodiment of the present application, comparing the first feature histogram with a preset sample feature histogram to determine the defect detection result of the object to be inspected includes:

[0024] Extract features from sample objects without defects and perform statistics to obtain a preset sample feature histogram;

[0025] Calculating a difference between the first feature histogram and the sample feature histogram to obtain an anomaly score of the first feature histogram;

[0026] When the anomaly score exceeds a preset anomaly score threshold, it is determined that the object to be detected has a defect.

[0027] In one embodiment of the present application, calculating the difference between the first feature histogram and the sample feature histogram to obtain an anomaly score of the first feature histogram includes:

[0028] Calculating the Euclidean distance between each vector in the first feature histogram and each vector in the sample feature histogram;

[0029] An anomaly score of the first feature histogram is determined based on the Euclidean distance.

[0030] In a second aspect, an embodiment of the present application provides a defect detection device, comprising:

[0031] An image acquisition module is used to acquire characteristic image data of the object to be detected;

[0032] a histogram acquisition module, configured to extract features from the feature image data using a pre-trained feature extraction network to obtain a first feature histogram of the object to be detected;

[0033] The defect detection module is used to compare the first feature histogram with a preset sample feature histogram to determine a defect detection result of the object to be detected.

[0034] In one embodiment of the present application, the histogram obtaining module is specifically configured to:

[0035] Performing feature extraction on the feature image data using a pre-trained feature extraction network to obtain first feature channel data of the object to be detected;

[0036] According to the preset feature channel weights, the first feature channel data of the object to be detected is counted to obtain a first feature histogram of the object to be detected.

[0037] In one embodiment of the present application, the device further comprises:

[0038] A data acquisition module, configured to acquire sample image data of a plurality of sample objects;

[0039] a data prediction module, configured to input the sample image data into an image recognition network to be trained, obtain image features of the sample image data through a feature extraction network in the image recognition network, and obtain a prediction result of the sample image data through a feature classification network in the image recognition network;

[0040] An image recognition network training module is used to adjust the parameters of the image recognition network to be trained according to the true value of the annotation of the sample image data and the prediction result to obtain a trained image recognition network, wherein the feature extraction network in the trained image recognition network is the pre-trained feature extraction network.

[0041] In one embodiment of the present application, the device further comprises:

[0042] A sample feature extraction module, configured to extract features from each of the sample image data using the pre-trained feature extraction network to obtain sample feature channel data for each of the sample image data;

[0043] a contribution degree acquisition module, configured to obtain the contribution degree of each characteristic channel according to the fluctuation of each characteristic channel in the sample characteristic channel data;

[0044] A weight calculation module, configured to calculate the feature channel weight of each feature channel based on the contribution of each feature channel;

[0045] The sample histogram obtaining module is used to obtain the preset sample feature histogram according to the feature channel weights and the sample feature channel data.

[0046] In one embodiment of the present application, the contribution acquisition module is specifically configured to:

[0047] Calculating the variance of each characteristic channel in the sample characteristic channel data, and sorting the fluctuation of each characteristic channel according to the size of the variance to obtain the fluctuation order of each characteristic channel;

[0048] According to the fluctuation order, the contribution of each characteristic channel is obtained.

[0049] In one embodiment of the present application, the defect detection module includes:

[0050] a score calculation submodule, configured to calculate the difference between the first feature histogram and the sample feature histogram to obtain an anomaly score of the first feature histogram;

[0051] The defect determination submodule is configured to determine that a defect exists in the object to be detected when the anomaly score exceeds a preset anomaly score threshold.

[0052] In one embodiment of the present application, the score calculation submodule is specifically used to:

[0053] Calculating the Euclidean distance between each vector in the first feature histogram and each vector in the sample feature histogram;

[0054] An anomaly score of the first feature histogram is determined based on the Euclidean distance.

[0055] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0056] Memory for storing computer programs;

[0057] The processor is configured to implement any of the above-mentioned defect detection methods when executing a program stored in the memory.

[0058] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements any of the above-mentioned defect detection methods.

[0059] An embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-described defect detection methods.

[0060] Beneficial effects of the embodiments of the present application:

[0061] The defect detection method provided in the embodiment of the present application, after acquiring the feature image data of the object to be detected, performs feature extraction on the feature image data through a pre-trained feature extraction network to obtain a first feature histogram of the object to be detected. The first feature histogram can count and display the specific information of each feature in the object to be detected. Since the visual features of logical defects already exist in normal images without defects, the difference in visual features can only be used to judge structural defects and cannot judge logical defects. Compared with visual features, the first feature histogram has more advanced semantic information and can be used to judge whether the object to be detected has logical features when there are no abnormalities in the visual features. By comparing the first feature histogram with the preset sample feature histogram, the defect detection result of the object to be detected is determined, which can achieve overall detection of structural defects and logical defects, avoid defect omissions as much as possible, and improve the accuracy of defect detection.

[0062] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0064] Figure 1-1An example diagram of a structural defect provided in an embodiment of the present application;

[0065] Figure 1-2 An example diagram of a logic defect provided in an embodiment of the present application;

[0066] Figure 2 A schematic diagram of a process flow of a first defect detection method provided in an embodiment of the present application;

[0067] Figure 3 A possible implementation of step S102 provided in an embodiment of the present application;

[0068] Figure 4 A schematic diagram of a second defect detection method provided in an embodiment of the present application;

[0069] Figure 5-1 A schematic diagram of a process flow of a third defect detection method provided in an embodiment of the present application;

[0070] Figure 5-2 An example diagram of a sample object provided in an embodiment of the present application;

[0071] Figure 5-3 This is an example diagram of a defective object provided in an embodiment of the present application;

[0072] Figure 5-4 A comparison diagram of a sample object and multiple defective objects provided in an embodiment of the present application;

[0073] Figure 5-5 This is an example diagram of a sample feature histogram provided in an embodiment of the present application;

[0074] Figure 5-6 This is an example diagram of a first feature histogram of a defective object provided in an embodiment of the present application;

[0075] Figure 6 A possible implementation of step S402 provided in an embodiment of the present application;

[0076] Figure 7 A possible implementation of step S103 provided in an embodiment of the present application;

[0077] Figure 8-1 A possible implementation of step S602 provided in an embodiment of the present application;

[0078] Figure 8-2 A flowchart illustrating a defect detection method according to an embodiment of the present invention;

[0079] Figure 9 A schematic structural diagram of a defect detection device provided in an embodiment of the present application;

[0080] Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0081] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of this application.

[0082] Since defect detection in related technologies may suffer from missed detection, resulting in inaccurate detection results, in order to solve this problem, an embodiment of the present application provides a defect detection method, which can be executed by an electronic device. Specifically, the electronic device can be a personal terminal, a server, etc.

[0083] First, the following professional terms that may appear in the embodiments of this application are explained:

[0084] Structural defects: There are characteristic defects on the structure of the object to be inspected that are inconsistent with normal characteristics, such as Figure 1-1 The workpiece surface cracks, toothbrush bristles scattered, electronic device surface scratches, etc. The image of the object to be inspected has local pixel variations in texture;

[0085] Logical defects: The image structure of the object to be detected has no defects, there is no local pixel variation in texture, and there are logical problems such as object missing and position error, e.g. Figure 1-2 As shown, a short screw is missing from the screw bag compared to normal products, nuts and fruits are present in the lunch box but placed in abnormal positions, and ingredients are missing from the instant noodle production process.

[0086] Good image: a normal image without defects.

[0087] The following is a detailed description of the defect detection method provided by the embodiment of the present application. Figure 2 , Figure 2 A schematic flow chart of a defect detection method is provided for an embodiment of the present application, including:

[0088] Step S101, acquiring characteristic image data of the object to be detected;

[0089] Step S102, performing feature extraction on the feature image data using a pre-trained feature extraction network to obtain a first feature histogram of the object to be detected;

[0090] Step S103 : comparing the first feature histogram with a preset sample feature histogram to determine a defect detection result of the object to be detected.

[0091] The object to be inspected is an object to be inspected for defects. Specifically, in the industrial manufacturing industry, it can be a semi-finished product or finished product in the production process, for example, a screw package containing multiple screws and nuts, the contents of instant noodles, a fruit lunch box, etc.

[0092] The characteristic image data of the object to be detected refers to image data that can clearly display the characteristics of the object to be detected. Specifically, it can be obtained by any image acquisition device for acquiring image data. For example, by acquiring images through a camera located above the object to be detected, a top-view image that can clearly display the characteristics of the object to be detected is obtained; or by acquiring images through a camera located directly in front of the object to be detected, a front-view image that can clearly display the characteristics of the object to be detected is obtained.

[0093] A pre-trained feature extraction network is used to extract features from the feature image data of the object to be detected, thereby obtaining a first feature histogram of the object to be detected. For example, the trained feature extraction network can be any network with feature extraction capabilities, such as ResNet (residual network), DenseNet (Densely Connected Convolutional Networks, a network with dense connections), SqueezeNet (a lightweight convolutional network), etc. The feature extraction network is pre-trained using normal feature image data (good images) of sample objects of the same type as the object to be detected.

[0094] The first feature histogram can count and display the specific information of each feature in the object to be detected, such as the number, proportion, color, location, etc. of each feature in the object to be detected.

[0095] Feature extraction and statistics are performed on sample objects without defects in advance to obtain a preset sample feature histogram, in which the information of each feature is normal. When the information of each feature in the first feature histogram is consistent with the information of each feature in the sample feature histogram, or the degree of similarity reaches a similarity threshold, it indicates that the object to be inspected is free of defects. When the information of each feature in the first feature histogram is inconsistent with the information of each feature in the sample feature histogram, or the degree of similarity does not reach the similarity threshold, it indicates that the object to be inspected is defective. Exemplarily, the similarity threshold is a pre-set value.

[0096] By comparing the first feature histogram with a preset sample feature histogram, a defect detection result of the object to be detected is determined.

[0097] As can be seen from the above, the defect detection method provided in the embodiment of the present application, after acquiring the feature image data of the object to be detected, performs feature extraction on the feature image data through a pre-trained feature extraction network to obtain a first feature histogram of the object to be detected. The first feature histogram can count and display the specific information of each feature in the object to be detected. Since the visual features of logical defects already exist in normal images without defects, the difference in visual features can only be used to judge structural defects and cannot judge logical defects. Compared with visual features, the first feature histogram has more advanced semantic information and can be used to judge whether the object to be detected has logical features when there are no abnormalities in the visual features. By comparing the first feature histogram with the preset sample feature histogram, the defect detection result of the object to be detected is determined, which can achieve overall detection of structural defects and logical defects, avoid defect omissions as much as possible, and improve the accuracy of defect detection.

[0098] In one embodiment of the present application, Figure 3 As shown, the above step S102 performs feature extraction on the feature image data through a pre-trained feature extraction network to obtain a first feature histogram of the object to be detected, including:

[0099] Step S201, performing feature extraction on the feature image data using a pre-trained feature extraction network to obtain first feature channel data of the object to be detected;

[0100] Step S202: According to preset feature channel weights, first feature channel data of the object to be detected are counted to obtain a first feature histogram of the object to be detected.

[0101] Each feature channel represents a type of feature. For example, a feature channel can represent the color feature, texture feature, shape feature, quantity feature, etc. of the object to be detected. Each feature channel data represents the specific information of the feature channel, for example, color information corresponding to the color feature channel, texture information corresponding to the texture feature channel, shape information corresponding to the shape feature channel, quantity information corresponding to the quantity feature channel, etc.

[0102] The pre-trained feature extraction network extracts features from the feature image data of the object to be detected, generating first feature channel data of the object to be detected. The first feature channel data includes specific information about each feature channel in the feature image data of the object to be detected, such as color information, texture information, shape information, and quantity information of the object to be detected.

[0103] Based on preset feature channel weights, weighted statistics are performed on each feature channel in the first feature channel data to obtain a first feature histogram of the object to be inspected. Specifically, the preset feature channel weights are pre-set effective proportions of each feature channel, with feature channels that are more helpful for defect detection in the object to be inspected being given greater weights. The first feature histogram can represent the weighted first feature channel data of the object to be inspected.

[0104] As can be seen from the above, the defect detection method provided in the embodiment of the present application extracts features from the feature image data of the object to be detected through a pre-trained feature extraction network to obtain first feature channel data of the object to be detected, and the first feature channel data includes specific information of each feature channel of the object to be detected. According to the preset feature channel weights, the first feature channel data of the object to be detected are counted to obtain a first feature histogram of the object to be detected, which can represent the weighted first feature channel data of the object to be detected, and will help to visually visualize the features of distinguishing whether the object to be detected has defects, further improving the accuracy of defect detection.

[0105] In one embodiment of the present application, Figure 4 As shown, before the above step S101 acquires the characteristic image data of the object to be detected, the method further includes:

[0106] Step S301, obtaining sample image data of multiple sample objects;

[0107] Step S302: inputting the sample image data into an image recognition network to be trained, obtaining image features of the sample image data through a feature extraction network in the image recognition network, and obtaining prediction results of the sample image data through a feature classification network in the image recognition network;

[0108] Step S303: adjusting the parameters of the image recognition network to be trained according to the true value of the annotation of the sample image data and the prediction result to obtain a trained image recognition network.

[0109] The feature extraction network in the trained image recognition network is the pre-trained feature extraction network.

[0110] The sample object is a non-defective object of the same type as the object to be inspected, with normal characteristics. For example, if the object to be inspected is a screw pack, the sample object is a standard screw pack with normal characteristics such as the type and quantity of each screw. If the object to be inspected is a fruit platter, the sample object is a standard fruit platter with normal characteristics such as the type, quantity, and placement of each fruit.

[0111] The sample image data of the sample object is a normal image without defects, known as a "good" image. The sample image data of the sample object is input into the image recognition network to be trained. The feature extraction network in the image recognition network extracts the image features of the sample image data. The feature classification network in the image recognition network then generates a prediction result for the sample image data. The prediction result is the image recognition network's identification and annotation of the sample object, such as the classification and information annotation of each feature in the sample object.

[0112] The true value of the annotation of the sample image data is a predetermined true annotation, which can be obtained manually in advance. Based on the difference between the true value of the annotation of the sample image data and the predicted result, the parameters of the image recognition network to be trained are adjusted, and the image recognition network is continuously trained to obtain a trained image recognition network. The feature extraction network in the trained image recognition network is the pre-trained feature extraction network.

[0113] In one example, the image recognition network can be any image recognition network in related technologies, such as ResNet, VGG, DenseNet (all of which are a type of image recognition network), etc.

[0114] As can be seen from the above, the defect detection method provided in the embodiment of the present application uses sample image data without defects and their annotated true values ​​to pre-train the image recognition network, and uses the feature extraction network in the trained image recognition network as the above-mentioned pre-trained feature extraction network to perform feature extraction on the feature image data of the object to be detected, thereby being able to more accurately realize the feature extraction of the object to be detected, thereby improving the accuracy of defect detection of the object to be detected.

[0115] In one embodiment of the present application, Figure 5-1 As shown, after obtaining the pre-trained feature extraction network in step S303, the method further includes:

[0116] Step S401, performing feature extraction on each sample image data using a pre-trained feature extraction network to obtain sample feature channel data of each sample image data;

[0117] Step S402, obtaining the contribution of each characteristic channel according to the fluctuation of each characteristic channel in the sample characteristic channel data;

[0118] Step S403, calculating the feature channel weight of each feature channel based on the contribution of each feature channel;

[0119] Step S404: obtaining the preset sample feature histogram according to the feature channel weights and the sample feature channel data.

[0120] The features of each sample image data are extracted through a pre-trained feature extraction network to obtain sample feature channel data of each sample image data, which includes specific information of each feature channel in the sample image data.

[0121] The fluctuation of each feature channel in the sample feature channel data indicates the contribution of each feature channel to defect detection. The larger the fluctuation, the more different the feature channel is in the good image. The feature is a difference feature in the good image, and the resolution between defect features and normal features is low. Therefore, when there is a difference in the feature channel, it should not be used as a key defect feature detection, which also indicates that the contribution of the feature channel to defect detection is lower. The smaller the fluctuation, the lower the difference of the feature channel in the good image. The feature is a common feature in the good image, the resolution between defect features and normal features is high, and the contribution to defect detection is higher. Specifically, the fluctuation of each feature channel can be represented by statistical values ​​that can indicate data stability, such as the variance, mean, and median of each feature channel.

[0122] For example, the color information of the color feature channel in sample image data A is silver, and the color information in sample image data B is black and silver. The color feature channel has differences, which means that the normal features of the sample object include multiple colors. The difference in color features is not used as a key defect feature detection, and the contribution of the color feature channel to defect detection is low.

[0123] According to the fluctuation of each feature channel in the sample feature channel data, the contribution of each feature channel is obtained, and then the feature channel weight of each feature channel is calculated, which represents the proportion of each feature channel in defect detection. According to the feature channel weight, the sample feature channel data is weighted and statistically analyzed to obtain the preset sample feature histogram.

[0124] In one example, a preset sample feature histogram may be stored in a memory bank (a type of storage space). When defect detection is required for an object to be inspected, the sample feature histogram is retrieved and compared with a first feature histogram of the object to be inspected.

[0125] For example, Figure 5-2 An example diagram showing a sample object, Figure 5-3 An example diagram showing a defect object, Figure 5-4A comparison diagram of a sample object and multiple defective objects is shown. The sample object is a standard screw pack containing varying numbers of long screws, short screws, nuts, and washers. The defective object is also a screw pack containing varying numbers of long screws, short screws, nuts, and washers. The similar shape features have low resolution and cannot be used as distinguishing features for defect detection. However, the number of long screws, short screws, nuts, and washers has high resolution and can be used as a distinguishing feature for defect detection.

[0126] Figure 5-5 An example diagram of a sample feature histogram is shown. Figure 5-6 This diagram shows an example of a first feature histogram for a defective object. It shows the number of different shape features, corresponding to the number of long screws, short screws, nuts, and washers. Based on the different quantities in the histogram, it can be determined that the object is defective.

[0127] As can be seen from the above, the defect detection method provided in the embodiments of the present application obtains the contribution of each feature channel based on the fluctuations of each feature channel in the sample feature channel data, thereby calculating the feature channel weight of each feature channel. Based on the feature channel weights and the sample feature channel data, a preset sample feature histogram is obtained. The differences and contributions of different feature channels are distinguished by weights, thereby making the sample feature histogram have higher resolution and can be used for comparison with the first feature histogram to more accurately detect defects in the object to be inspected.

[0128] In one embodiment of the present application, Figure 6 As shown, the above step S402 obtains the contribution of each characteristic channel according to the fluctuation of each characteristic channel in the sample characteristic channel data, including:

[0129] Step S501, calculating the variance of each feature channel in the sample feature channel data, and sorting the fluctuation of each feature channel according to the size of the variance to obtain the fluctuation order of each feature channel;

[0130] Step S502: Obtain the contribution of each of the characteristic channels according to the fluctuation order.

[0131] Calculate the variance of each feature channel. The smaller the variance, the more common features of good images it contains, the smaller the fluctuation of the feature channel, the greater the contribution of the feature channel, and the greater the proportion that should be given in the feature comparison.

[0132] As can be seen from the above, the defect detection method provided in the embodiment of the present application determines the fluctuation of each feature channel by calculating the variance of each feature channel, and determines the contribution of the feature channel based on this, thereby determining more discriminative features for defect detection, thereby improving the accuracy of defect detection.

[0133] In one embodiment of the present application, Figure 7 As shown, the above step S103 compares the first feature histogram with a preset sample feature histogram to determine the defect detection result of the object to be detected, including:

[0134] Step S601, calculating the difference between the first feature histogram and the sample feature histogram to obtain an anomaly score of the first feature histogram;

[0135] Step S602: When the anomaly score exceeds a preset anomaly score threshold, it is determined that the object to be detected has a defect.

[0136] Feature extraction and statistics are performed on sample objects without defects to obtain a preset sample feature histogram.

[0137] The difference between the first feature histogram and the sample feature histogram is calculated to obtain the anomaly score of the first feature histogram. Specifically, the difference between the variance of the first feature histogram and the variance of the sample feature histogram, the vector distance between the vector of the first feature histogram and the vector of the sample feature histogram, etc. can be calculated to obtain the anomaly score of the first feature histogram.

[0138] The preset anomaly score threshold is a pre-set threshold, which means that when the anomaly score exceeds the threshold, the difference between the first feature histogram and the sample feature histogram is large, based on which it can be determined that the object to be detected has defects.

[0139] As can be seen from the above, the defect detection method provided in the embodiment of the present application scores the difference between the first feature histogram and the sample feature histogram, thereby reducing the probability of defect detection errors and improving the accuracy of defect detection.

[0140] In one embodiment of the present application, Figure 8-1 As shown, the above step S602 calculates the difference between the first feature histogram and the sample feature histogram to obtain the anomaly score of the first feature histogram, including:

[0141] Step S701, calculating the Euclidean distance between each vector in the first feature histogram and each vector in the sample feature histogram;

[0142] Step S702: Determine an anomaly score of the first feature histogram based on the Euclidean distance.

[0143] The Euclidean distance between each vector in the first feature histogram and each vector in the sample feature histogram is calculated to determine the difference between each vector in the first feature histogram and each vector in the sample feature histogram. Specifically, the Euclidean distance can be implemented using any relevant Euclidean distance algorithm. Based on this, an anomaly score for the first feature histogram is determined, which can indicate the degree of anomaly of the first feature histogram relative to the sample feature histogram.

[0144] As can be seen from the above, the defect detection method provided in the embodiment of the present application determines the degree of abnormality of the first feature histogram relative to the sample feature histogram by calculating the Euclidean distance between each vector in the first feature histogram and each vector in the sample feature histogram, quantifies the abnormality degree score, realizes abnormality judgment of the first feature histogram, and then determines the defect detection result of the object to be detected, thereby improving the accuracy of defect detection.

[0145] In one embodiment of the present application, Figure 8-2 As shown, a defect detection process example diagram is provided, wherein the imagenet pre-trained network represents a pre-trained feature extraction network, 1*C*H*W is the size of the feature, and compared with the unsupervised comparison of features in the related art (only comparing visual features), to address the problem of poor logic defect detection, the embodiment of the present application calculates a histogram of the features, 1*C*bin is the data of the histogram, and using the histogram to compare features can better detect logic defects and obtain more accurate defect detection results.

[0146] Second, as Figure 9 As shown, the embodiment of the present application also provides a structural schematic diagram of a defect detection device, including:

[0147] The image acquisition module 801 is used to acquire characteristic image data of the object to be detected;

[0148] A histogram acquisition module 802 is configured to perform feature extraction on the feature image data using a pre-trained feature extraction network to obtain a first feature histogram of the object to be detected;

[0149] The defect detection module 803 is configured to compare the first feature histogram with a preset sample feature histogram to determine a defect detection result of the object to be detected.

[0150] As can be seen from the above, the defect detection device provided in the embodiment of the present application, after acquiring the feature image data of the object to be detected, performs feature extraction on the feature image data through a pre-trained feature extraction network to obtain a first feature histogram of the object to be detected. The first feature histogram can count and display the specific information of each feature in the object to be detected. Since the visual features of logical defects already exist in normal images without defects, the difference in visual features can only be used to judge structural defects and cannot judge logical defects. Compared with visual features, the first feature histogram has more advanced semantic information and can be used to judge whether the object to be detected has logical features when there are no abnormalities in the visual features. By comparing the first feature histogram with the preset sample feature histogram, the defect detection result of the object to be detected is determined, which can achieve overall detection of structural defects and logical defects, avoid missed defects as much as possible, and improve the accuracy of defect detection.

[0151] In one embodiment of the present application, the histogram obtaining module 802 is specifically configured to:

[0152] Performing feature extraction on the feature image data using a pre-trained feature extraction network to obtain first feature channel data of the object to be detected;

[0153] According to the preset feature channel weights, the first feature channel data of the object to be detected is counted to obtain a first feature histogram of the object to be detected.

[0154] As can be seen from the above, the defect detection device provided in the embodiment of the present application extracts features from the feature image data of the object to be detected through a pre-trained feature extraction network to obtain first feature channel data of the object to be detected, and the first feature channel data includes specific information of each feature channel of the object to be detected. According to the preset feature channel weights, the first feature channel data of the object to be detected are counted to obtain a first feature histogram of the object to be detected, which can represent the weighted first feature channel data of the object to be detected, and will help to visually visualize the features of distinguishing whether the object to be detected has defects, further improving the accuracy of defect detection.

[0155] In one embodiment of the present application, the device further comprises:

[0156] A data acquisition module, configured to acquire sample image data of a plurality of sample objects;

[0157] a data prediction module, configured to input the sample image data into an image recognition network to be trained, obtain image features of the sample image data through a feature extraction network in the image recognition network, and obtain a prediction result of the sample image data through a feature classification network in the image recognition network;

[0158] An image recognition network training module is used to adjust the parameters of the image recognition network to be trained according to the true value of the annotation of the sample image data and the prediction result to obtain a trained image recognition network, wherein the feature extraction network in the trained image recognition network is the pre-trained feature extraction network.

[0159] As can be seen from the above, the defect detection device provided in the embodiment of the present application uses sample image data without defects and their annotated true values ​​to pre-train an image recognition network, and uses the feature extraction network in the trained image recognition network as the above-mentioned pre-trained feature extraction network to perform feature extraction on the feature image data of the object to be detected, thereby being able to more accurately realize the feature extraction of the object to be detected, thereby improving the accuracy of defect detection of the object to be detected.

[0160] In one embodiment of the present application, the device further comprises:

[0161] A sample feature extraction module, configured to extract features from each of the sample image data using the pre-trained feature extraction network to obtain sample feature channel data for each of the sample image data;

[0162] a contribution degree acquisition module, configured to obtain the contribution degree of each characteristic channel according to the fluctuation of each characteristic channel in the sample characteristic channel data;

[0163] A weight calculation module, configured to calculate the feature channel weight of each feature channel based on the contribution of each feature channel;

[0164] The sample histogram obtaining module is used to obtain the preset sample feature histogram according to the feature channel weights and the sample feature channel data.

[0165] As can be seen from the above, the defect detection device provided in the embodiment of the present application obtains the contribution of each feature channel based on the fluctuation of each feature channel in the sample feature channel data, thereby calculating the feature channel weight of each feature channel. Based on the feature channel weights and the sample feature channel data, a preset sample feature histogram is obtained. The differences and contributions of different feature channels are distinguished by weights, so that the sample feature histogram has higher resolution and can be used for comparison with the first feature histogram to more accurately detect defects in the object to be inspected.

[0166] In one embodiment of the present application, the contribution acquisition module is specifically configured to:

[0167] Calculating the variance of each characteristic channel in the sample characteristic channel data, and sorting the fluctuation of each characteristic channel according to the size of the variance to obtain the fluctuation order of each characteristic channel;

[0168] According to the fluctuation order, the contribution of each characteristic channel is obtained.

[0169] As can be seen from the above, the defect detection device provided in the embodiment of the present application determines the fluctuation of each characteristic channel by calculating the variance of each characteristic channel, and determines the contribution of the characteristic channel based on this, thereby determining more discriminative features for defect detection, thereby improving the accuracy of defect detection.

[0170] In one embodiment of the present application, the defect detection module 803 includes:

[0171] a score calculation submodule, configured to calculate the difference between the first feature histogram and the sample feature histogram to obtain an anomaly score of the first feature histogram;

[0172] The defect determination submodule is configured to determine that a defect exists in the object to be detected when the anomaly score exceeds a preset anomaly score threshold.

[0173] For example, feature extraction and statistics can be performed on sample objects without defects to obtain a preset sample feature histogram.

[0174] As can be seen from the above, the defect detection method provided in the embodiment of the present application scores the difference between the first feature histogram and the sample feature histogram, thereby reducing the probability of defect detection errors and improving the accuracy of defect detection.

[0175] In one embodiment of the present application, the score calculation submodule is specifically used to:

[0176] Calculating the Euclidean distance between each vector in the first feature histogram and each vector in the sample feature histogram;

[0177] An anomaly score of the first feature histogram is determined based on the Euclidean distance.

[0178] As can be seen from the above, the defect detection method provided in the embodiment of the present application determines the degree of abnormality of the first feature histogram relative to the sample feature histogram by calculating the Euclidean distance between each vector in the first feature histogram and each vector in the sample feature histogram, quantifies the abnormality degree score, realizes abnormality judgment of the first feature histogram, and then determines the defect detection result of the object to be detected, thereby improving the accuracy of defect detection.

[0179] The present application also provides an electronic device, such as Figure 10 Shown, including:

[0180] Memory 901, used for storing computer programs;

[0181] The processor 902 is configured to implement any of the above-mentioned defect detection methods when executing the program stored in the memory 901 .

[0182] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 902 , the communication interface, and the memory 901 communicate with each other via the communication bus.

[0183] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0184] The communication interface is used for communication between the above electronic device and other devices.

[0185] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0186] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0187] In another embodiment provided in the present application, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the steps of any of the above-mentioned defect detection methods are implemented.

[0188] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any defect detection method in the above embodiments.

[0189] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state drive (SSD).

[0190] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0191] Each embodiment in this specification is described in a related manner. Similar portions between the embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so their description is relatively simple. For related portions, refer to the description of the method embodiments.

[0192] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A defect detection method, characterized in that: include: Acquire characteristic image data of the object to be detected; Performing feature extraction on the feature image data using a pre-trained feature extraction network to obtain a first feature histogram of the object to be detected; The first feature histogram is compared with a preset sample feature histogram to determine a defect detection result of the object to be detected.

2. The method according to claim 1, characterized in that The step of extracting features from the feature image data using a pre-trained feature extraction network to obtain a first feature histogram of the object to be detected includes: Performing feature extraction on the feature image data using a pre-trained feature extraction network to obtain first feature channel data of the object to be detected; According to the preset feature channel weights, the first feature channel data of the object to be detected is counted to obtain a first feature histogram of the object to be detected.

3. The method according to claim 1, characterized in that Before acquiring the characteristic image data of the object to be detected, the method further includes: Obtaining sample image data of a plurality of sample objects; Inputting the sample image data into an image recognition network to be trained, obtaining image features of the sample image data through a feature extraction network in the image recognition network, and obtaining prediction results of the sample image data through a feature classification network in the image recognition network; According to the labeled true value of the sample image data and the prediction result, the parameters of the image recognition network to be trained are adjusted to obtain a trained image recognition network, wherein the feature extraction network in the trained image recognition network is the pre-trained feature extraction network.

4. The method according to claim 3, characterized in that After obtaining the pre-trained feature extraction network, the method further includes: Performing feature extraction on each of the sample image data using the pre-trained feature extraction network to obtain sample feature channel data of each of the sample image data; Obtaining a contribution of each of the characteristic channels according to fluctuations of each of the characteristic channels in the sample characteristic channel data; Calculating the feature channel weight of each feature channel based on the contribution of each feature channel; The preset sample feature histogram is obtained according to the feature channel weights and the sample feature channel data.

5. The method according to claim 4, characterized in that Obtaining the contribution of each characteristic channel according to the fluctuation of each characteristic channel in the sample characteristic channel data includes: Calculating the variance of each characteristic channel in the sample characteristic channel data, and sorting the fluctuation of each characteristic channel according to the size of the variance to obtain the fluctuation order of each characteristic channel; According to the fluctuation order, the contribution of each characteristic channel is obtained.

6. The method according to claim 1, characterized in that The comparing the first feature histogram with a preset sample feature histogram to determine the defect detection result of the object to be inspected includes: Extract features from sample objects without defects and perform statistics to obtain a preset sample feature histogram; Calculating a difference between the first feature histogram and the sample feature histogram to obtain an anomaly score of the first feature histogram; When the anomaly score exceeds a preset anomaly score threshold, it is determined that the object to be detected has a defect.

7. The method according to claim 6, characterized in that Calculating the difference between the first feature histogram and the sample feature histogram to obtain an anomaly score of the first feature histogram includes: Calculating the Euclidean distance between each vector in the first feature histogram and each vector in the sample feature histogram; An anomaly score of the first feature histogram is determined based on the Euclidean distance.

8. A defect detection device, characterized in that: include: An image acquisition module is used to acquire characteristic image data of the object to be detected; a histogram acquisition module, configured to extract features from the feature image data using a pre-trained feature extraction network to obtain a first feature histogram of the object to be detected; The defect detection module is used to compare the first feature histogram with a preset sample feature histogram to determine a defect detection result of the object to be detected.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images