Industrial product surface anomaly detection method based on image patch classification

Abstract

The invention relates to an industrial product surface anomaly detection method based on image patch classification. The method comprises the following steps: acquiring an industrial product surface image; using the trained feature extractor to perform feature extraction and cutting on the industrial product surface image to obtain a plurality of patch blocks, and using the trained binary classifier to detect the plurality of patch blocks to obtain an abnormal score of each patch block, the trained binary classifier is obtained by training the binary classifier by using a plurality of abnormal patch blocks of an abnormal sample containing an abnormal region and a plurality of normal patch blocks of a normal sample, and the plurality of patch blocks of the abnormal region cover the surface anomaly type of the industrial product; and rearranging the patch blocks with the abnormal scores and adjusting the patch blocks back to the shape and the size of the industrial product surface image to obtain an abnormal score graph, and judging whether the industrial product surface image is abnormal according to the abnormal score graph. According to the method, dependence of a data set can be avoided, the model can be conveniently and rapidly trained, and the speed and accuracy of industrial product surface anomaly detection are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of surface detection, and in particular relates to a method for detecting surface anomalies of industrial products based on image patch classification. Background Art

[0002] The application of artificial intelligence in actual production practice has become a general trend. Using deep learning technology to detect surface anomalies of industrial products is an important practice, and some industrial product surface anomaly detection methods based on deep learning technology have emerged.

[0003] However, the existing methods for detecting surface anomalies of industrial products have the following pain points: First, the existing algorithms require a large number of normal samples and abnormal samples for model training, but in the actual production environment, it is very difficult to obtain sufficient data sets, so the existing algorithms for detecting surface anomalies of industrial products using deep learning technology are too dependent on data sets; Second, the existing algorithms often cannot distinguish between abnormal samples and normal samples that are visually extremely similar to tiny defect areas, and cannot accurately detect and locate defects, resulting in missed detection, and cannot meet the needs of the actual production environment; Third, the structures of most existing deep learning models are extremely large and complex. In the actual production environment, the deployment and application of such models require a certain amount of time and money costs; In addition, the update and adjustment of the model is also closely related to the structural complexity of the original model. With the addition of new training data, the adjustment of the structurally complex model also requires a lot of time and cost.

[0004] Therefore, it is imperative to find an effective deep learning method for industrial product surface anomaly detection that is independent of data sets and can train models quickly and easily. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a method for detecting surface anomalies of industrial products based on image patch classification. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] The embodiment of the present invention provides a method for detecting surface anomalies of industrial products based on image patch classification, comprising the steps of:

[0007] Acquire surface images of industrial products;

[0008] Using the trained feature extractor to extract features and cut the surface image of the industrial product to obtain a number of patches;

[0009] The plurality of patch blocks are detected by using a trained binary classifier to obtain an abnormal score for each patch block; wherein the trained binary classifier is obtained by training the binary classifier using a plurality of abnormal patch blocks in an abnormal area included in an abnormal sample and a plurality of normal patch blocks in a normal sample, and the plurality of patch blocks in the abnormal area cover an abnormal type of the surface of an industrial product;

[0010] The patches with the anomaly scores are rearranged and adjusted back to the shape and size of the industrial product surface image to obtain an anomaly score map, and whether the industrial product surface image has an anomaly is determined based on the anomaly score map.

[0011] In one embodiment of the present invention, the training method of the feature extractor and the binary classifier comprises the steps of:

[0012] Acquire a training data set including an industrial product surface image and an abnormal area mask image corresponding to the industrial product surface image;

[0013] Initializing parameters of the feature extractor and the binary classifier;

[0014] Perform feature extraction and segmentation on the images in the training data set to obtain abnormal patches of several abnormal samples and normal patches of several normal samples;

[0015] According to the abnormal area marked in the abnormal area mask map, a number of abnormal patch blocks that actually contain abnormalities are selected from a number of abnormal patch blocks as abnormal data, and a number of normal patch blocks are randomly selected as normal data according to the number of abnormal patch blocks;

[0016] Using the abnormal data and the normal data to train the binary classifier to optimize the classification loss function of the binary classifier, and outputting the abnormal scores of each abnormal patch block and each normal patch block;

[0017] Rearranging and adjusting the patches with the anomaly scores back to the shape and size of the industrial product surface image to obtain an anomaly score map;

[0018] Calculating an image loss function between the anomaly score map and the anomaly region mask map;

[0019] Using the classification loss function and the image loss function to update the network parameters of the feature extractor and the binary classifier;

[0020] When the iteration conditions are met, a trained feature extractor and a trained binary classifier are obtained.

[0021] In one embodiment of the present invention, obtaining a training data set including an industrial product surface image and an abnormal region mask image corresponding to the industrial product surface image includes:

[0022] Collect image data of industrial product surfaces;

[0023] Marking all abnormal regions of abnormal samples in the industrial product surface image data to obtain an abnormal region mask map, wherein all abnormal regions cover the abnormal types of the industrial product surface;

[0024] The abnormal samples corresponding to the abnormal region mask image and the same number of normal samples are used as the training data set.

[0025] In one embodiment of the present invention, feature extraction and segmentation are performed on images in a training data set to obtain abnormal patches of a number of abnormal samples and normal patches of a number of normal samples, including:

[0026] Scaling and cropping the images in the training data set to obtain a number of images of the same size;

[0027] Performing feature extraction on the plurality of images of the same size to obtain a plurality of feature maps;

[0028] The plurality of feature maps are cut to obtain abnormal patch blocks of a plurality of abnormal samples and normal patch blocks of a plurality of normal samples.

[0029] In one embodiment of the present invention, feature extraction is performed on the plurality of images of the same size to obtain a plurality of feature maps, including:

[0030] A pre-trained residual network is used to extract features from the plurality of images of the same size to obtain a plurality of feature maps.

[0031] In one embodiment of the present invention, the number of the normal patch blocks is 90%-110% of the number of the abnormal patch blocks.

[0032] In one embodiment of the present invention, the binary classifier includes a fully connected neural network model.

[0033] In one embodiment of the present invention, the fully connected neural network model includes a first fully connected layer, a second fully connected layer and a third fully connected layer, wherein the first fully connected layer, the second fully connected layer and the third fully connected layer are connected in sequence, the first fully connected layer adopts a ReLU activation function, the second fully connected layer adopts a ReLU activation function, and the third fully connected layer adopts a Sigmoid activation function.

[0034] In one embodiment of the present invention, the classification loss function includes a binary cross entropy loss function, which is expressed as:

[0035]

[0036] Among them, N batchis the number of patches in a batch, x i is the feature vector of the ith patch, is a collection of abnormal patch blocks, Indicator function, It means that when the ith patch block is abnormal, the value is 1, otherwise it is 0. i ; θ) is the output of the fully connected neural network model for the i-th patch block, Represents the predicted probability of the output of the i-th patch block by the fully connected neural network model after the Sigmoid activation function.

[0037] In one embodiment of the present invention, judging whether the surface image of the industrial product has an abnormality according to the abnormality score map includes:

[0038] When there is a patch block in the anomaly score map whose anomaly score is greater than or equal to a preset threshold, the industrial product surface image is abnormal, and the abnormal position is located;

[0039] When there is a patch block in the anomaly score map whose anomaly score is less than a preset threshold, there is no anomaly in the surface image of the industrial product.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The method of the present invention uses a number of abnormal patch blocks in the abnormal area that the abnormal sample truly contains and a number of normal patch blocks in the normal sample to train the binary classifier. The number of patch blocks in the abnormal area only needs to cover the abnormal type of the industrial product surface. Therefore, only a small number of normal samples and abnormal samples need to be obtained to train the binary classifier, which not only greatly reduces the training time, but also saves a lot of data collection and data annotation costs; at the same time, the present invention forms a patch classification detector by a feature extractor and a binary classifier, and the structure is extremely lightweight and effective, which greatly reduces the deployment time and operation cost of the model in an actual production environment, and the subsequent update of the model is extremely convenient and fast, and only a small number of samples need to be obtained to update, avoiding the defect that the model cannot be updated immediately and effectively because it is too large and complex; therefore, the industrial product surface anomaly detection method of the present invention has the advantages of being independent of the data set and conveniently and quickly training the model, thereby improving the speed and accuracy of industrial product surface anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic flow chart of a method for detecting anomalies on the surface of industrial products based on image patch classification provided by an embodiment of the present invention;

[0043] Figure 2 A flowchart of a training method for a patch classification detector provided by an embodiment of the present invention;

[0044] Figure 3 A test flow chart for performing anomaly detection on a trained patch classification detector provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0046] Embodiment 1

[0047] See also Figure 1 , Figure 1 A flow chart of a method for detecting anomalies on the surface of industrial products based on image patch classification provided by an embodiment of the present invention. The method for detecting anomalies on the surface of industrial products comprises the following steps:

[0048] S1. Acquire the surface image of industrial products.

[0049] Specifically, an industrial camera is used to perform high-quality and complete imaging of the surface of the industrial product, and the image is stored to obtain an image of the surface of the industrial product.

[0050] S2. Use the trained feature extractor to extract features and cut the industrial product surface image to obtain a number of patch blocks, and use the trained binary classifier to detect the number of patch blocks to obtain the abnormality score of each patch block; wherein the trained binary classifier is trained using the abnormal samples containing a number of abnormal patch blocks in the abnormal area and a number of normal patch blocks in the normal samples, and the several patch blocks in the abnormal area cover the abnormal type of the industrial product surface.

[0051] In the embodiment of the present invention, a deep learning algorithm (deep neural network) and a binary classifier are combined to jointly generate a patch classification (PatchClassifier) ​​detection method including three parts: extraction and cutting of feature patches, training of binary classifiers based on patches, and detection and positioning of abnormal areas. It can be understood that the patch classification detection method consists of image input, feature map extraction, feature block cutting, binary classifier abnormality detection and positioning, wherein feature extractor is used to extract feature maps, and binary classifier is used to detect patch abnormalities, and the feature extractor and binary classifier together constitute a patch classification detector.

[0052] In a specific embodiment, the feature extractor includes but is not limited to a pre-trained residual network. Exemplarily, a pre-trained 50-layer residual network is used.

[0053] In a specific embodiment, the binary classifier includes a fully connected neural network model. The fully connected neural network model includes a first fully connected layer, a second fully connected layer, and a third fully connected layer, wherein the first fully connected layer, the second fully connected layer, and the third fully connected layer are connected in sequence, the first fully connected layer uses a ReLU activation function, the second fully connected layer uses a ReLU activation function, and the third fully connected layer uses a Sigmoid activation function. The loss function of the fully connected neural network model selects a binary cross entropy loss function. The optimizer selects an Adam optimizer.

[0054] It should be noted that the binary classifier may also use a neural network model of other structures, and is not limited to a fully connected neural network model. When the binary classifier uses a fully connected neural network model, the number of fully connected layers in the fully connected neural network model is not limited to three layers, and may also be 1 layer, 2 layers, or more layers.

[0055] See also Figure 2 , Figure 2 A flowchart of a method for training a feature extractor and a binary classifier provided by an embodiment of the present invention, in which a small number of normal and abnormal samples are input, a pre-trained residual network is used to extract a feature map, and then the feature map is cut and divided into blocks, and then the patches that truly contain abnormal areas are selected from all the patches of the abnormal samples, and normal patches whose number is approximately equal to the number of patches that truly contain abnormal areas are randomly selected from the patches of the normal samples, and finally a binary classifier is obtained by training these two patches, and the parameters of the feature extractor are updated during each training process. The training method of the feature extractor and the binary classifier specifically includes the following steps:

[0056] S21, obtaining a training data set including an industrial product surface image and an abnormal area mask image corresponding to the industrial product surface image. Specifically including:

[0057] S211. Collecting surface image data of industrial products.

[0058] Specifically, an industrial camera is used to perform high-quality, complete imaging of the surface of industrial products, and the images are stored as raw image data for detecting abnormalities on the surface of industrial products.

[0059] S212, marking all abnormal regions of abnormal samples in the industrial product surface image data to obtain an abnormal region mask map, wherein all abnormal regions cover the abnormal types of the industrial product surface.

[0060] Specifically, the ImageJ dataset annotation tool is used to manually annotate all abnormal areas in a small number of abnormal samples using polygonal marking boxes. The annotation result is a mask map, and its name corresponds to the name of the abnormal sample one by one.

[0061] It should be noted that a small number of abnormal samples refers to samples that can cover the abnormal types of industrial product surfaces. For example, for a certain type of industrial product, if there are 5 types of surface abnormalities, then a small number of abnormal samples can be 50 samples; that is, for this type of industrial product, no matter the number of industrial product surface image data collected is 200 or 1000, 50 samples can cover all types of surface abnormalities, then the number of a small number of abnormal samples is 50 abnormal samples.

[0062] S213, taking the abnormal samples corresponding to the abnormal region mask image and the same number of normal samples as the industrial product surface image training set.

[0063] Specifically, all abnormal area mask images obtained in step S212 are stored in the ground_truth folder, and the abnormal samples corresponding to the mask images and the same number of normal samples are stored in the train folder as the industrial product surface image training set. Further, all the remaining abnormal samples and normal sample images are stored in the test folder as the industrial product surface image test set. Through this step, an industrial product surface image dataset containing three folders (ground_truth, train and test) is finally generated.

[0064] S22. Train the feature extractor and the binary classifier.

[0065] S221. Initialize the parameters of the feature extractor and the binary classifier.

[0066] In this embodiment, the initialization parameters mainly include three parts: the first is to initialize the parameters of the feature extractor, and a pre-trained 50-layer residual network is exemplarily used as the feature extraction network. This deep neural network is widely used in the extraction of image data features, and the initialization parameters are the pre-trained model parameters trained on a public data set; the second is to initialize the parameters of the binary classifier, and a three-layer fully connected neural network model is exemplarily used as the binary classifier. The initialization parameters are the number of neurons in the three layers and the learning rate parameters of the optimizer (exemplarily using the Adam optimizer); the third is to initialize the parameters of the input data preprocessing, including the scaling and cropping parameters of the training set and test set data.

[0067] S222, performing feature extraction and segmentation on the images in the training data set to obtain abnormal patches of a number of abnormal samples and normal patches of a number of normal samples.

[0068] First, image data mask maps, image names, and category names are obtained from the training dataset according to the batch size.

[0069] Then, the images in the training dataset are scaled and cropped to obtain several images of the same size.

[0070] Next, feature extraction is performed on a plurality of images of the same size to obtain a plurality of feature maps. Exemplarily, a plurality of images of the same size are input into a pre-trained residual network to perform feature extraction to obtain a plurality of feature maps.

[0071] Finally, several feature maps are cut to obtain abnormal patches of several abnormal samples and normal patches of several normal samples, which are used as raw data for subsequent training of the binary classifier.

[0072] S223. According to the abnormal area marked in the abnormal area mask map, select a number of abnormal patch blocks that actually contain abnormalities from the number of abnormal patch blocks as abnormal data for training the binary classifier, and randomly select a number of normal patch blocks according to the number of abnormal patch blocks as normal data for training the binary classifier. Specifically, the number of normal patch blocks is 90%-110% of the number of abnormal patch blocks.

[0073] This embodiment uses the normal and abnormal patch blocks obtained by cutting to train the binary classifier, and the patch blocks that truly contain the abnormal area, rather than using the entire feature map for training or using all the normal and abnormal patch blocks for training. This can more effectively amplify the difference between normal and abnormal, because compared with the entire feature map, the patch blocks obtained by cutting have higher resolution and more obvious details. In addition, using the patch blocks that truly contain defective areas is equivalent to screening out those areas that do not contain defects in advance from the feature map corresponding to the abnormal sample, which makes subsequent training and detection more accurate and effective.

[0074] In this embodiment, the number of normal and abnormal patch blocks is approximately equal and has the same order of magnitude. Since the number of abnormal patch blocks is small, the number of normal patch blocks is also small. Therefore, only a small number of normal samples and abnormal samples need to be obtained to train a binary classifier, which saves a lot of data collection and data labeling costs and greatly reduces training time.

[0075] S224. Train a binary classifier using abnormal data and normal data to optimize the classification loss function of the binary classifier, and output anomaly scores for each abnormal patch block and each normal patch block.

[0076] Specifically, the three-layer fully connected neural network model is trained to obtain a binary classifier. The training process can be regarded as an optimization process of a loss function. The classification loss function includes a binary cross entropy loss function, and its formula is as follows:

[0077]

[0078] Among them, N batch is the number of patches in a batch, representing the number of patches involved in a loss calculation; x iis the feature vector of the ith patch, is a collection of abnormal patch blocks, indicator function; It means that when the ith patch block is abnormal, the value is 1, otherwise it is normal, the value is 0; f(x i ; θ) is the output of the fully connected neural network model for the i-th patch block, Represents the predicted probability of the output of the i-th patch block by the fully connected neural network model after the Sigmoid activation function.

[0079] The meaning of the above formula is: for each patch block x i , the cross entropy loss consists of two parts, if x i is an abnormal patch block, and the loss is If x i is a normal patch block, and the loss is In summary, the loss function sums the losses of all patches in a batch and takes the average.

[0080] This embodiment selects a binary cross entropy loss function to update the binary classifier. The binary cross entropy loss function converts the output of the binary classifier model into a probability value between 0.0 and 1.0 through a Sigmoid function. This probability interpretation makes the output value intuitive and easy to understand, and the output of the model can be directly interpreted as the probability that the patch block is predicted to be abnormal. At the same time, the binary cross entropy loss function accurately reflects the difference between the predicted value and the true value. When the predicted value of the model is close to the true label, the loss value is small. When the predicted value of the model is greatly different from the true label, the loss value is large. This design allows the binary cross entropy loss function to effectively reward correct classifications while punishing incorrect classifications, thereby guiding the parameter update of the three-layer fully connected neural network model to improve in the direction of correct classification.

[0081] S225, rearrange and adjust the patches with anomaly scores back to the shape and size of the industrial product surface image to obtain an anomaly score map.

[0082] S226. Calculate the image loss function between the abnormal score map and the abnormal samples.

[0083] Specifically, the loss function between the anomaly score map and the abnormal samples is obtained by calculating the similarity between the two. This calculation process belongs to the prior art and will not be described in detail in this embodiment.

[0084] S227. Use the classification loss function and the image loss function to update the network parameters of the binary classifier and the feature extractor.

[0085] Specifically, the classification loss function is used to update the network parameters of the binary classifier, and the image loss function is used to update the network parameters of the feature extractor.

[0086] S228. When the iteration condition is met, a trained feature extractor and a trained binary classifier are obtained.

[0087] Specifically, when a preset number of iterations is reached, the iteration is stopped to obtain a trained feature extractor and a trained binary classifier, and the trained feature extractor and the trained binary classifier constitute a trained patch classification detector.

[0088] Furthermore, the trained patch classification detector is tested for anomaly detection using the industrial product surface image test set. Figure 3 , Figure 3 A test flow chart for abnormality detection of a trained patch classification detector provided in an embodiment of the present invention. For the test process, a large number of remaining normal and abnormal samples are input, and a pre-trained residual network is used to extract feature maps, which are then cut and divided into blocks. All the patch blocks are then input into the trained binary classifier together. The binary classifier generates an abnormality score between the interval [0.0, 1.0] for each patch block, and then rearranges and adjusts the patch blocks with abnormal scores back to the shape and size of the original image. Finally, it is determined whether the test image has an abnormality and the location of the abnormality is located.

[0089] The testing process includes the following steps:

[0090] S231, obtaining data. Obtain image data, mask images, image names, category names, and other data from the industrial product surface image test set according to the batch size.

[0091] S232. Scale and crop the images in the industrial product surface image test set to obtain a number of images of the same size; use a pre-trained residual network to extract features from the number of images of the same size to obtain a number of feature maps; then, cut the number of feature maps to obtain patches of the test image.

[0092] S233. Use the trained binary classifier to detect the patch blocks of the test image and generate an anomaly score between [0.0, 1.0], so as to determine whether there is an anomaly in the test image and generate a corresponding heat map.

[0093] S3. Rearrange and adjust the patches with anomaly scores back to the shape and size of the industrial product surface image to obtain an anomaly score map, and determine whether the industrial product surface image has an anomaly based on the anomaly score map.

[0094] Specifically, when there is a patch block with an anomaly score greater than or equal to a preset threshold in the anomaly score map, the industrial product surface image is abnormal and the abnormal position is located; when there is a patch block with an anomaly score less than a preset threshold in the anomaly score map, the industrial product surface image is not abnormal. Exemplarily, the value of the anomaly score is [0.0, 1.0], and the value of the preset threshold is 0.5.

[0095] Furthermore, when the industrial product surface anomaly detection method based on image patch classification of this embodiment is used for detection in an actual production environment, for products of the same type, the trained feature extraction can be directly used to detect the surface image of the industrial product; for different types of products, the patch classification detector is first trained using the surface image data of the product of that type before detection. Since the number of abnormal samples and normal samples required for training is small, the training time is short and the training efficiency is high.

[0096] Based on the technical problem of improving the detection speed and detection accuracy of the existing industrial product surface anomaly detection method based on deep learning models, this embodiment of the industrial product surface anomaly detection method based on image patch classification divides the optical images of a small number of normal samples and abnormal samples into patches, selects normal patches and abnormal patches from the large number of patch blocks after segmentation to construct a data set for deep learning model training, uses a deep learning model based on patch block classification (i.e., residual network ResNet) and a multi-layer perceptron MLP (i.e., binary classifier), and uses the trained patch classification detector to effectively detect subtle abnormal areas and give abnormal detection results, thereby improving the speed and accuracy of industrial product surface anomaly detection.

[0097] The present embodiment only needs to obtain a small number of normal samples and abnormal samples to train the patch classification detector, which not only greatly reduces the training time, but also saves a lot of data collection and data labeling costs; at the same time, the patch classification detector structure of the present embodiment is extremely lightweight and effective, which greatly reduces the deployment time and operation cost of the model in the actual production environment, and the subsequent update of the model is extremely convenient and fast, and only a small number of samples need to be obtained to update, avoiding the defect that the model cannot be updated immediately and effectively due to being too large and complex; therefore, the industrial product surface anomaly detection method of the present embodiment has the advantages of being independent of the data set and conveniently and quickly training the model, thereby improving the speed and accuracy of industrial product surface anomaly detection.

[0098] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for detecting surface anomalies of industrial products based on image patch classification, characterized in that: Includes steps: Acquire surface images of industrial products; The trained feature extractor is used to extract features and cut the industrial product surface image to obtain a number of patches, and the trained binary classifier is used to detect the number of patches to obtain an abnormality score for each patch; wherein the trained binary classifier is obtained by training the binary classifier using a number of abnormal patches in an abnormal area included in the abnormal sample and a number of normal patches in the normal sample, and the number of patches in the abnormal area covers the abnormal type of the industrial product surface; The patches with the anomaly scores are rearranged and adjusted back to the shape and size of the industrial product surface image to obtain an anomaly score map, and whether the industrial product surface image has an anomaly is determined based on the anomaly score map.

2. The method for detecting anomalies on the surface of industrial products based on image patch classification according to claim 1, characterized in that: The training method of the feature extractor and the binary classifier comprises the steps of: Acquire a training data set including an industrial product surface image and an abnormal area mask image corresponding to the industrial product surface image; Initializing parameters of the feature extractor and the binary classifier; Perform feature extraction and segmentation on the images in the training data set to obtain abnormal patches of several abnormal samples and normal patches of several normal samples; According to the abnormal area marked in the abnormal area mask map, a number of abnormal patch blocks that actually contain abnormalities are selected from a number of abnormal patch blocks as abnormal data, and a number of normal patch blocks are randomly selected as normal data according to the number of abnormal patch blocks; Using the abnormal data and the normal data to train the binary classifier to optimize the classification loss function of the binary classifier, and outputting the abnormal scores of each abnormal patch block and each normal patch block; Rearranging and adjusting the patches with the anomaly scores back to the shape and size of the industrial product surface image to obtain an anomaly score map; Calculating an image loss function between the anomaly score map and the anomaly region mask map; Using the classification loss function and the image loss function to update the network parameters of the binary classifier and the feature extractor; When the iteration conditions are met, a trained feature extractor and a trained binary classifier are obtained.

3. The method for detecting anomalies on the surface of industrial products based on image patch classification according to claim 2, characterized in that: Acquiring a training data set including an industrial product surface image and an abnormal area mask map corresponding to the industrial product surface image, including: Collect image data of industrial product surfaces; Marking all abnormal regions of abnormal samples in the industrial product surface image data to obtain an abnormal region mask map, wherein all abnormal regions cover the abnormal types of the industrial product surface; The abnormal samples corresponding to the abnormal region mask image and the same number of normal samples are used as the training data set.

4. The method for detecting anomalies on the surface of industrial products based on image patch classification according to claim 2, characterized in that: Perform feature extraction and segmentation on the images in the training data set to obtain abnormal patches of several abnormal samples and normal patches of several normal samples, including: Scaling and cropping the images in the training data set to obtain a number of images of the same size; Performing feature extraction on the plurality of images of the same size to obtain a plurality of feature maps; The plurality of feature maps are cut to obtain abnormal patch blocks of a plurality of abnormal samples and normal patch blocks of a plurality of normal samples.

5. The method for detecting anomalies on the surface of industrial products based on image patch classification according to claim 4, characterized in that: Feature extraction is performed on the plurality of images of the same size to obtain a plurality of feature maps, including: A pre-trained residual network is used to extract features from the plurality of images of the same size to obtain a plurality of feature maps.

6. The method for detecting anomalies on the surface of industrial products based on image patch classification according to claim 2, characterized in that: The number of the normal patches is 90%-110% of the number of the abnormal patches.

7. The method for detecting anomalies on the surface of industrial products based on image patch classification according to claim 2, characterized in that: The binary classifier includes a fully connected neural network model.

8. The method for detecting anomalies on the surface of industrial products based on image patch classification according to claim 7, characterized in that: The fully connected neural network model includes a first fully connected layer, a second fully connected layer and a third fully connected layer, wherein: The first fully connected layer, the second fully connected layer, and the third fully connected layer are connected in sequence, the first fully connected layer adopts a ReLU activation function, the second fully connected layer adopts a ReLU activation function, and the third fully connected layer adopts a Sigmoid activation function.

9. The method for detecting anomalies on the surface of industrial products based on image patch classification according to claim 8, characterized in that: The classification loss function includes a binary cross entropy loss function, which is expressed as: Among them, N batch is the number of patches in a batch, x i is the feature vector of the ith patch, is a collection of abnormal patch blocks, Indicator function, It means that when the ith patch block is abnormal, the value is 1, otherwise it is 0. i ; θ) is the output of the fully connected neural network model for the i-th patch block, Represents the predicted probability of the output of the i-th patch block by the fully connected neural network model after the Sigmoid activation function.

10. The method for detecting anomalies on the surface of industrial products based on image patch classification according to claim 1, characterized in that: Judging whether the industrial product surface image has an abnormality according to the abnormality score map includes: When there is a patch block in the anomaly score map whose anomaly score is greater than or equal to a preset threshold, the industrial product surface image is abnormal, and the abnormal position is located; When there is a patch block in the anomaly score map whose anomaly score is less than a preset threshold, there is no anomaly in the surface image of the industrial product.

Images