Defect detection method and device based on small normal sample data set

Abstract

A defect detection method and device based on a small amount of normal sample data set, wherein the defect detection method steps include obtaining multiple high-resolution normal sample images to make a normal sample data set; constructing an image feature and edge feature extraction network model; inputting the processed normal sample images into the above network model to train the feature extraction capability; applying the pre-trained above network model to the processed multiple normal sample images for feature extraction and splicing; applying a multi-head self-attention module to the splicing result for feature fusion and obtaining a normal sample low-rank feature matrix through multi-element Gaussian distribution calculation; processing a high-resolution product image through the above network model for feature extraction, splicing and feature fusion to obtain a feature vector thereof; calculating the Mahalanobis distance of the product image feature vector and the normal sample low-rank feature matrix to obtain an abnormal distribution heat map and display. Thus, the rapid detection and accurate positioning of the surface defects of an object with complex structure are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of product surface defect detection, and in particular to a defect detection method and device based on a small amount of normal sample data set training. BACKGROUND

[0002] With the development of productivity and the progress of society, at present, in the production field of 3C products and high-end toys, etc., the production characteristics of high mix low volume have begun to appear, that is, the types of products produced on the production line will change frequently, and the yield of each batch of products will not be large. Under this condition, the existing industrial automated visual defect detection system is facing great challenges. First, the existing industrial automated visual defect detection system is mostly based on hand crafted features, and uses template matching method to compare the differences between normal sample and defect sample features to determine the location and type of defects. However, the hand crafted features can only be used for one object, and a large amount of time needs to be invested by professional personnel for development, usually more than 3 months, and a new algorithm needs to be designed after a new product is replaced. Therefore, hand crafted features are time-consuming and expensive, and are difficult to adapt to high-frequency iterative production lines. Second, the defect detection algorithm based on artificial intelligence usually needs a large amount of labeled sample data for training, but it is difficult to collect enough sample data for small batch production lines, and usually the probability of occurrence of defective products is very low, so it is difficult to collect enough sample data. Third, in order to balance the speed and accuracy, the existing industrial automated visual defect detection method usually uses low resolution images for analysis, which may result in some subtle defects being unable to be accurately detected.

[0003] Therefore, how to develop a product surface defect detection technology based on a small amount of sample data and suitable for production lines with high mix low volume production characteristics, and capable of realizing fast speed and high accuracy detection, has become one of the technical problems to be solved in the field. SUMMARY

[0004] The technical problem to be solved by the present technical solution is to provide a product structure surface anomaly detection technology, which can train a stable and accurate AI model in a short time by using a small amount of normal sample data, so as to realize fast and accurate defect detection of products with complex structure surface on production lines with high mix low volume production characteristics, thereby meeting the demand for product update on production lines, and reducing the investment of manpower, time and funds in product quality supervision links.

[0005] To solve the above technical problems, the technical scheme provides a defect detection method based on a small amount of normal sample data set, and the steps of the defect detection method include:

[0006] Making a normal sample data set: obtaining a plurality of high-resolution normal sample images containing object structure surfaces to form a normal sample data set;

[0007] Building a feature extraction model: applying a convolutional neural network technology to build an image feature extraction network model and an edge feature extraction network model;

[0008] Pre-training of the feature extraction model: performing image size standardization processing on the plurality of normal sample images in the normal sample data set, dividing the normal sample images after image size standardization processing into a plurality of small image blocks with equal image size, and combining the plurality of small image blocks of the normal sample images into a tensor with a size, and by inputting the tensors with a size of the plurality of normal sample images into the image feature extraction network model and the edge feature extraction network model respectively to extract features, reconstruct images, compare before and after reconstruction, and correct model parameters, to train the feature extraction capability of the image feature extraction network model and the edge feature extraction network model;

[0009] Normal sample feature extraction and splicing: inputting the tensors with a size of the plurality of normal sample images into the pre-trained image feature extraction network model and edge feature extraction network model to extract semantic feature vectors and edge feature vectors of the plurality of small image blocks of the plurality of normal sample images respectively, and simultaneously using a cosine position encoding method to assign a corresponding position vector to each small image block according to the position of each small image block in the normal sample image, and using a parallel connection method to splice the semantic feature vectors, edge feature vectors and position vectors of the plurality of small image blocks of each normal sample image in the same dimension to obtain a tensor with another size;

[0010] Creating a normal sample feature distribution: applying a multi-head self-attention module to perform feature fusion on the tensors with another size of the plurality of normal sample images to obtain discriminative feature vectors of the plurality of normal sample images, and then performing multivariate Gaussian distribution calculation on the discriminative feature vectors of the plurality of normal sample images to obtain a normal sample low-rank feature matrix;

[0011] Product image feature acquisition: high-resolution product images containing object structure surfaces are obtained, and after image size standardization processing, the product images are segmented into a plurality of image blocks of equal size, and the plurality of small image blocks of the product image are combined into a tensor with a size, the tensor with a size of the product image is input into a pre-trained image feature extraction network model and an edge feature extraction network model to extract semantic feature vectors and edge feature vectors of the plurality of small image blocks of the product image, respectively., and according to the position of each small image block of the product image in the product image, a corresponding position vector is given to each small image block of the product image using the cosine position coding method, and the semantic feature vectors, edge feature vectors and position vectors of the plurality of small image blocks of the product image are spliced in the same dimension using a parallel connection method to obtain a tensor with another size, and then a multi-head self-attention module is applied to the tensor with another size of the product image. Feature fusion to obtain a feature vector of the product image;

[0012] Product image defect detection: the Mahalanobis distance between the normal sample low-rank feature matrix and each vector element of the feature vector of the product image is calculated to obtain an abnormal distribution heat map of the product image.

[0013] Defect display: the heat area of the abnormal distribution heat map is extracted and displayed on the corresponding area of the product image.

[0014] As a further implementation of the defect detection method, the image size standardization processing is to fill black pixels at the edges of the image if the length and / or width of the image is less than the preset size, so that the length and / or width of the image after filling black pixels is equal to the preset size.

[0015] As a further implementation of the defect detection method, in the feature extraction model pre-training step and the product image feature acquisition step, the normal sample images and the product images are all segmented into 16 small image blocks of equal size.

[0016] As a further implementation of the defect detection method, the normal sample images and the product images are images with a resolution greater than 20 megapixels taken by an industrial camera.

[0017] As a further implementation of the defect detection method, in the step of creating a normal sample feature distribution, the discriminant feature vectors of a plurality of normal sample images are calculated by a multivariate Gaussian distribution, and the specific method for obtaining the normal sample low-rank feature matrix is:

[0018] Calculate the average value μ of the discriminant feature vectors of the plurality of normal sample images, N is the number of normal sample images, V ia discriminative feature vector of the ith normal sample image;

[0019] a multivariate Gaussian distribution of each vector element of the normal sample is calculated to obtain a low-rank feature matrix (σ, μ) of the normal sample, σ is a covariance matrix.

[0020] As a further implementation of the present defect detection method, in the product image defect detection step, the specific method for calculating the Mahalanobis distance D between the low-rank feature matrix of the normal sample and each vector element of the feature vector of the product image is: x is the feature vector of the product image.

[0021] To solve the above technical problems, the technical scheme also provides a defect detection device trained based on a small amount of normal sample data sets, which comprises:

[0022] An image acquisition module is configured to acquire high-resolution product images containing object structure surfaces to make normal sample data sets or perform defect detection.

[0023] An image processing module is configured to perform image size standardization processing on the product images, and divide the product images after the image size standardization processing into a plurality of small image blocks with equal image sizes, and combine the plurality of small image blocks of the product images into a tensor with a size.

[0024] A model pre-training module is configured to input the tensors with the size of the normal sample images in the normal sample data sets processed by the image processing module into an image feature extraction network model and an edge feature extraction network model, respectively, to train the feature extraction capabilities of the image feature extraction network model and the edge feature extraction network model.

[0025] An image feature extraction module is configured to input the tensors with the size of the product images processed by the image processing module into the pre-trained image feature extraction network model and edge feature extraction network model, respectively, to extract semantic feature vectors and edge feature vectors of the product images, respectively, and use a cosine position encoding method to assign position vectors to the product images, and use a parallel connection method to splice the semantic feature vectors, edge feature vectors and position vectors of the product images in the same dimension to obtain a tensor with another size, and then apply a multi-head self-attention module to perform feature fusion on the tensor with another size of the product images to obtain discriminative feature vectors of the product images.

[0026] A normal sample feature distribution creation module is configured to perform multivariate Gaussian distribution calculation on the discriminative feature vectors of a plurality of normal sample images in the normal sample data sets processed by the image processing module and the image feature extraction module to obtain a low-rank feature matrix of the normal sample.

[0027] a defect detection module for calculating Mahalanobis distance between each vector element of the discriminative feature vector of the product image and the normal sample low-rank feature matrix to obtain an abnormal distribution heat map of the product image;

[0028] a display module for extracting the heat area of the abnormal distribution heat map to overlay and display on the corresponding area of the product image.

[0029] As a further implementation of the defect detection device, the image acquisition module is an industrial camera with an image resolution greater than 20 megapixels.

[0030] As a further implementation of the defect detection device, the normal sample feature distribution creation module is to first calculate the average value μ of the discriminative feature vector of multiple normal sample images, N is the number of normal sample images, V i is the discriminative feature vector of the i-th normal sample image; and then calculate the multivariate Gaussian distribution of each vector element of the normal sample to obtain the normal sample low-rank feature matrix (σ, μ), σ is the covariance matrix.

[0031] As a further implementation of the defect detection device, the defect detection module is to calculate the Mahalanobis distance D between each vector element of the normal sample low-rank feature matrix (σ, μ) and the feature vector x of the product image,

[0032]

[0033] Accordingly, the defect detection method and the defect detection device of the technical solution are applied to the features of the hidden layer output of the convolutional neural network, rather than the output results of the convolutional neural network, and the feature extraction capability of the image feature extraction network model and the edge feature extraction network model based on the convolutional neural network technology is trained only through a small amount (100 to 200) of normal sample images. Moreover, the technical solution uses high-resolution images (including normal sample images and product images), and does not perform down-sampling compression on the size to maintain the original resolution of the images, so that subtle defects (less than 3 mm) on the images can be accurately detected. However, due to the use of high-resolution images, the difficulty of feature extraction of the above network model is increased and the efficiency of feature extraction is reduced, so the technical solution is to first divide the high-resolution images into a plurality of small image blocks, and then apply the pre-trained network model for corresponding feature extraction, and then splice the semantic feature vectors, edge feature vectors and position vectors of the plurality of image blocks in the same dimension, that is, the corresponding feature vectors of the complete image can be obtained, so that the information amount of feature extraction of the above network model can be greatly reduced to ensure high running efficiency, so that the instruction cycle is maintained faster, and the subtle features on the image can be fully retained to ensure the accuracy of defect recognition. Moreover, by applying the multi-head self-attention module, more feature details of the image can be retained in multiple dimensions, so as to improve the feature extraction precision of the above network model under the training of a small amount of normal sample data. Moreover, by performing multivariate Gaussian distribution calculation on the discriminant feature vectors of a plurality of normal sample images, a reasonable normal sample low-rank feature matrix can be obtained, so as to judge the degree of deviation of each vector element of the product image feature vector from the normal sample low-rank feature matrix by Mahalanobis distance, so as to realize efficient and accurate recognition of the defect area in the product image.

[0034] In addition, by applying the edge feature extraction network model, the edge feature vectors of the objects in the image can be extracted, so that the technical solution can focus on the structural defects of the product surface, effectively solve the reflection problem of the metal parts when shooting the image, and reduce the misjudgment caused by the special color or style of the non-defect on the product surface. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 The step flow chart of the defect detection method based on a small amount of normal sample data sets of the present application;

[0036] Figure 2 The module schematic diagram of the defect detection device based on a small amount of normal sample data sets of the present application.

[0037] Explanation of symbols in the drawings: 1 defect detection device; 10 image acquisition module; 20 image processing module; 30 model pre-training module; 31 image feature extraction network model; 32 edge feature extraction network model; 40 image feature extraction module; 41 pre-trained image feature extraction network model; 42 pre-trained edge feature extraction network model; 43 cosine position coding mode; 44 parallel connection mode; 45 semantic feature vector; 46 edge feature vector; 47 position vector; 48 multi-head self-attention module; 49 discriminative feature vector; 50 normal sample feature distribution creation module; 51 normal sample low-rank feature matrix; 60 defect detection module; 61 discriminative feature vector; 70 display module; 71 hot area. DETAILED DESCRIPTION

[0038] The detailed description and technical content of the present application are described below in conjunction with the drawings, however, the accompanying drawings are provided for reference and illustration only, and are not intended to limit the present application.

[0039] At present, in the production field of 3C products and high-end toys, etc., the production characteristics of high mix low volume have begun to appear, and the existing defect detection model based on hand crafted features gradually shows the problem of insufficient universality, and the design of the defect detection method based on hand crafted features is time-consuming, and cannot be quickly iterated to adapt to the products of high mix low volume on the production line. Therefore, the present application proposes a defect detection method and device based on a small amount of normal sample data set training, which can train a stable and accurate AI model in a short time by using a small amount of normal sample data, so as to quickly and accurately detect defects of products on the production line with complex structure surface in a small batch and multiple styles, and can overcome the problems of product surface reflection, special color or style of non-defect on product surface, etc.

[0040] Specifically, in combination with the step flow chart of the defect detection method based on a small amount of normal sample data set training of the present application shown in Figure 1 The steps of the defect detection method of the present application mainly include:

[0041] Making a normal sample data set: obtaining a plurality of high-resolution normal sample images containing object structure surfaces to form a normal sample data set;

[0042] Constructing a feature extraction model: applying a convolutional neural network technology to construct an image feature extraction network model and an edge feature extraction network model respectively;

[0043] Feature extraction model pre-training: a plurality of normal sample images in the normal sample data set are subjected to image size standardization processing, the normal sample images subjected to image size standardization processing are segmented into a plurality of small image blocks of equal image size, and the plurality of small image blocks of the normal sample images are combined into a tensor having a size, the tensor having a size of the plurality of normal sample images is input into an image feature extraction network model and an edge feature extraction network model respectively to perform feature extraction, image reconstruction, comparison before and after reconstruction, and model parameter correction, so as to train the feature extraction capability of the image feature extraction network model and the edge feature extraction network model;

[0044] Normal sample feature extraction and splicing: the tensor having a size of the plurality of normal sample images is input into the pre-trained image feature extraction network model and edge feature extraction network model respectively to extract semantic feature vectors and edge feature vectors of a plurality of small image blocks of the plurality of normal sample images, while using a cosine position coding method to assign a corresponding position vector to each small image block according to the position of each small image block in the normal sample image, and using a parallel connection method to splice the semantic feature vectors, edge feature vectors and position vectors of the plurality of small image blocks of each normal sample image in the same dimension to obtain a tensor having another size;

[0045] Creating normal sample feature distribution: applying a multi-head self-attention module to perform feature fusion on the tensor having another size of the plurality of normal sample images to obtain discriminative feature vectors of the plurality of normal sample images, and then performing multivariate Gaussian distribution calculation on the discriminative feature vectors of the plurality of normal sample images to obtain a normal sample low-rank feature matrix;

[0046] Product image feature acquisition: a high-resolution product image containing an object structure surface is obtained, the product image is subjected to image size standardization processing, and then segmented into a plurality of small image blocks of equal image size, and the plurality of small image blocks of the product image are combined into a tensor having a size, the tensor having a size of the product image is input into the pre-trained image feature extraction network model and edge feature extraction network model respectively to extract semantic feature vectors and edge feature vectors of the plurality of small image blocks of the product image, while using a cosine position coding method to assign a corresponding position vector to each small image block of the product image according to the position of each small image block in the product image, and using a parallel connection method to splice the semantic feature vectors, edge feature vectors and position vectors of the plurality of small image blocks of the product image in the same dimension to obtain a tensor having another size, and then applying a multi-head self-attention module to perform feature fusion on the tensor having another size of the product image to obtain a feature vector of the product image;

[0047] product image defect detection: calculating Mahalanobis distance between each vector element of normal sample low-rank feature matrix and feature vector of product image to obtain abnormal distribution heat map of product image;

[0048] defect display: extracting heat area of abnormal distribution heat map to overlay corresponding area of product image for display.

[0049] The defect detection method is to obtain normal sample images and product images of products on the production line by industrial cameras installed in the production line from the same fixed position and angle, and the resolution of the images captured by the industrial cameras is greater than 20 megapixels. In the step of making the normal sample data set, only 100 to 200 normal sample images are needed to make the normal sample data set. In the feature extraction model pre-training step, only 100 to 200 normal sample images in the normal sample data set are needed to train the image feature extraction network model and the edge feature extraction network model for 1 to 2 hours, and the pre-training of the above network model is completed.

[0050] In the present application, the image feature extraction network model and the edge feature extraction network model are respectively constructed based on convolutional neural network, which can extract the features of the image, including color, shape, line, etc. By giving different weights to the convolutional neural network during training, different feature extraction models focusing on image color, shape or line features can be constructed. In the present application, the extraction of product edge features is very important for the defect detection and recognition of the entire product. The combination of product edge features and product semantic features can overcome the adverse effects of product surface reflection, special colors or patterns on the surface of the product, etc. on defect detection.

[0051] Further, the image size normalization process in the defect detection method is as follows: if the length and / or width of the image is less than the preset size, black pixels are filled in the edge of the image to make the length and / or width of the filled black pixel image equal to the preset size. In the feature extraction model pre-training step and the product image feature acquisition step, the normal sample images and product images are all divided into 16 small image blocks with equal size. Of course, the present application does not strictly limit the number of small image blocks, and in actual operation, the normal sample images and product images can be divided into 9, 25 or other number of small image blocks according to the size and resolution of the image.

[0052] Further, in the step of creating normal sample feature distribution in the defect detection method, the specific method of calculating the multivariate Gaussian distribution of the discriminant feature vector of multiple normal sample images to obtain the normal sample low-rank feature matrix is as follows:

[0053] calculating an average value μ of the discriminant feature vectors of the plurality of normal sample images, N is the number of normal sample images, V i is the discriminant feature vector of the i-th normal sample image;

[0054] calculating a multivariate Gaussian distribution of each vector element of the normal sample to obtain a normal sample low-rank feature matrix (σ, μ), σ is the covariance matrix.

[0055] Further, in the product image defect detection step of the defect detection method, the specific method of calculating the Mahalanobis distance D between the normal sample low-rank feature matrix and each vector element of the feature vector of the product image is: x is the feature vector of the product image.

[0056] In addition, as shown in Figure 2 The present application also provides a defect detection device 1 trained based on a small amount of normal sample data set, which comprises:

[0057] An image acquisition module 10 is configured to acquire high-resolution product images containing object structure surfaces to make normal sample data sets or perform defect detection.

[0058] An image processing module 20 is configured to perform image size standardization processing on the product images, and divide the product images after the image size standardization processing into a plurality of small image blocks with equal image sizes, and combine the plurality of small image blocks of the product images into a tensor with a size.

[0059] A model pre-training module 30 is configured to input the tensor with a size of the normal sample images processed by the image processing module 20 in the normal sample data set into an image feature extraction network model 31 and an edge feature extraction network model 32 respectively, so as to train the feature extraction capability of the image feature extraction network model 31 and the edge feature extraction network model 32.

[0060] The image feature extraction module 40 is configured to input the tensor with a size of the product image processed by the image processing module 20 into a pre-trained image feature extraction network model 41 and a pre-trained edge feature extraction network model 42 respectively, so as to extract a semantic feature vector 45 and an edge feature vector 46 of the product image respectively, and use a cosine position encoding method 43 to assign a position vector 47 to the product image, and use a parallel connection method 44 to splice the semantic feature vector 45, the edge feature vector 46 and the position vector 47 of the product image in the same dimension to obtain a tensor with another size, and then apply a multi-head self-attention module 48 to perform feature fusion on the tensor with another size of the product image to obtain a discriminative feature vector 49 of the product image.

[0061] The normal sample feature distribution creation module 50 is configured to perform multivariate Gaussian distribution calculation on the discriminative feature vectors 49 of the plurality of normal sample images in the normal sample data set processed by the image processing module 20 and the image feature extraction module 40, so as to obtain a normal sample low-rank feature matrix 51.

[0062] The defect detection module 60 is configured to calculate the Mahalanobis distance between each vector element of the normal sample low-rank feature matrix 51 and a discriminative feature vector 61 of a product image, so as to obtain an abnormal distribution heat map of the product image.

[0063] The display module 70 is configured to extract a heat area 71 of the abnormal distribution heat map to be displayed on a corresponding area of the product image.

[0064] Further, the image acquisition module 10 in the defect detection device 1 is an industrial camera with an image resolution greater than 20 megapixels.

[0065] The normal sample feature distribution creation module 50 in the defect detection device 1 is configured to first calculate the average value μ of the discriminative feature vectors of the plurality of normal sample images, N is the number of normal sample images, V i is the discriminative feature vector of the i-th normal sample image, and then calculate the multivariate Gaussian distribution of each vector element of the normal sample to obtain a normal sample low-rank feature matrix (σ, μ), σ is a covariance matrix.

[0066] The defect detection module 60 in the defect detection device 1 is configured to calculate the Mahalanobis distance D between each vector element of the normal sample low-rank feature matrix (σ, μ) and the feature vector x of the product image,

[0067] In summary, the defect detection method and the defect detection device of the present application apply the features of the hidden layer output of the convolutional neural network, rather than the output results of the convolutional neural network, and only a small number (100 to 200) of normal sample images are needed to train the feature extraction capability of the image feature extraction network model and the edge feature extraction network model based on the convolutional neural network technology. Moreover, the present application uses high-resolution images (including normal sample images and product images), and does not perform downsampling to compress the size while maintaining the original resolution of the image, so that subtle defects (less than 3mm) on the image can be accurately detected. However, due to the use of high-resolution images, the difficulty of feature extraction of the above network model is increased and the efficiency of feature extraction is reduced, therefore, the present application first divides the high-resolution image into a plurality of small image blocks, and then applies the pre-trained network model for corresponding feature extraction, and then splices the semantic feature vectors, edge feature vectors and position vectors of the plurality of image blocks in the same dimension, to obtain the corresponding feature vector of the complete image. Accordingly, the information amount of feature extraction of the above network model can be greatly reduced to ensure high running efficiency, thereby maintaining a fast instruction cycle, and the subtle features on the image can be fully retained to ensure the accuracy of defect recognition. By applying the multi-head self-attention module, more feature details of the image can be retained in multiple dimensions, thereby improving the feature extraction precision of the above network model under the training of a small amount of normal sample data. By performing multivariate Gaussian distribution calculation on the discrimination feature vectors of a plurality of normal sample images, a reasonable normal sample low-rank feature matrix can be obtained, so as to judge the degree of deviation of each vector element of the product image feature vector from the normal sample low-rank feature matrix by Mahalanobis distance, thereby realizing efficient and accurate recognition of the defect area in the product image.

[0068] In addition, by applying the edge feature extraction network model, the edge feature vector of the object in the image can be extracted, so that the present application can focus on the structural defects of the product surface, effectively solve the reflection problem of the metal parts when shooting the image, and reduce the misjudgment caused by the special color or style of the non-defect on the product surface.

[0069] The present application can be applied in many different production fields, including small 3C products, toy manufacturing, metal part processing, and even fabric production.

[0070] The above is only a preferred embodiment of the present application, and is not intended to limit the patent scope of the present application. Other equivalent changes made by applying the patent concept of the present application should also fall within the patent protection scope of the present application.

Claims

1. A defect detection method trained on a small dataset of normal samples, characterized by the following steps: include: Creating a normal sample dataset: Acquire multiple high-resolution normal sample images containing the surface of object structures to form a normal sample dataset; Constructing Feature Extraction Models: Convolutional neural network technology is used to construct an image feature extraction network model and an edge feature extraction network model respectively; Feature extraction model pre-training: Multiple normal sample images in the normal sample dataset are subjected to image size standardization processing. The normal sample images after image size standardization are segmented into multiple small image blocks of equal size. The multiple small image blocks of the normal sample images are combined into a tensor with size. The tensor with size of the multiple normal sample images is respectively input into the image feature extraction network model and the edge feature extraction network model for feature extraction, image reconstruction, comparison before and after reconstruction, and model parameter correction, so as to train the feature extraction capabilities of the image feature extraction network model and the edge feature extraction network model. Normal sample feature extraction and concatenation: The tensors of multiple normal sample images with different dimensions are respectively input into the pre-trained image feature extraction network model and the edge feature extraction network model to extract the semantic feature vectors and edge feature vectors of multiple small image patches in the multiple normal sample images. At the same time, according to the position of each small image patch in the normal sample image, a corresponding position vector is assigned to each small image patch using the cosine position encoding method. The semantic feature vectors, edge feature vectors and position vectors of multiple small image patches in each normal sample image are concatenated in the same dimension using the parallel connection method to obtain a tensor with another size. Create a normal sample feature distribution: Apply a multi-head self-attention module to perform feature fusion on the tensors of the other size of the multiple normal sample images to obtain the discriminative feature vectors of the multiple normal sample images. Then, perform multivariate Gaussian distribution calculation on the discriminative feature vectors of the multiple normal sample images to obtain the low-rank feature matrix of the normal samples. Product Image Feature Acquisition: A high-resolution product image containing the surface structure of the object is acquired. After image size normalization, the product image is divided into multiple small image blocks of equal size. These small image blocks are then combined into a tensor of a specific size. This tensor is input into a pre-trained image feature extraction network model and an edge feature extraction network model to extract semantic feature vectors and edge feature vectors from the multiple small image blocks. Simultaneously, based on the position of each small image block within the product image, a cosine position encoding method is used to assign a corresponding position vector to each small image block. Furthermore, the semantic feature vectors, edge feature vectors, and position vectors of the multiple small image blocks are concatenated along the same dimension using a parallel concatenation method to obtain a tensor of another size. Finally, a multi-head self-attention module is applied to perform feature fusion on this tensor to obtain the feature vector of the product image. Product image defect detection: Calculate the Mahalanobis distance between each vector element of the low-rank feature matrix of the normal sample and the feature vector of the product image to obtain the abnormal distribution heat map of the product image; Defect display: Extract the heat regions from the abnormal distribution heat map and display them over the corresponding areas of the product image.

2. The defect detection method according to claim 1, characterized in that, The image size normalization process involves filling the edges of the image with black pixels if the length and / or width of the image is smaller than a preset size, so that the length and / or width of the image after filling with black pixels is equal to the preset size.

3. The defect detection method according to claim 1, characterized in that, In the feature extraction model pre-training step and the product image feature acquisition step, the normal sample image and the product image are both divided into 16 small image blocks of equal size.

4. The defect detection method according to claim 1, characterized in that, The normal sample images and product images are images with a resolution greater than 20 megapixels taken by an industrial camera.

5. The defect detection method according to claim 1, characterized in that, In the step of creating the normal sample feature distribution, the specific method for calculating the low-rank feature matrix of the normal samples by performing multivariate Gaussian distribution on the discriminant feature vectors of multiple normal sample images is as follows: Calculate the average value μ of the discriminative feature vectors of the multiple normal sample images. N is the number of normal sample images, V i Let be the discriminant feature vector of the i-th normal sample image; Calculate the multivariate Gaussian distribution of each vector element of the normal sample to obtain the low-rank feature matrix (σ, μ) of the normal sample. σ is the covariance matrix.

6. The defect detection method according to claim 5, characterized in that, In the product image defect detection step, the specific method for calculating the Mahalanobis distance D between each vector element of the low-rank feature matrix of the normal sample and the feature vector of the product image is as follows: x is the feature vector of the product image.

7. A defect detection device trained on a small dataset of normal samples, characterized in that, include: The image acquisition module is used to acquire high-resolution product images containing the surface structure of objects in order to create normal sample datasets or to perform defect detection. The image processing module is used to perform image size normalization processing on the product image, divide the product image after image size normalization into multiple small image blocks of equal size, and combine the multiple small image blocks of the product image into a tensor with size; The model pre-training module is used to input the tensors with dimensions of the normal sample images processed by the image processing module in the normal sample dataset into an image feature extraction network model and an edge feature extraction network model, respectively, so as to train the feature extraction capabilities of the image feature extraction network model and the edge feature extraction network model. The image feature extraction module is used to input the tensor of the product image after processing by the image processing module into the pre-trained image feature extraction network model and the edge feature extraction network model, respectively, to extract the semantic feature vector and edge feature vector of the product image. At the same time, the product image is assigned a position vector using cosine position encoding, and the semantic feature vector, edge feature vector and position vector of the product image are concatenated in the same dimension using a parallel connection method to obtain a tensor with another size. Then, a multi-head self-attention module is applied to perform feature fusion on the tensor of the product image with another size to obtain the discriminative feature vector of the product image. The normal sample feature distribution creation module is used to perform multivariate Gaussian distribution calculation on the discriminant feature vectors of multiple normal sample images in the normal sample dataset after being processed by the image processing module and the image feature extraction module, so as to obtain the normal sample low-rank feature matrix. The defect detection module is used to calculate the Mahalanobis distance between each vector element of the low-rank feature matrix of the normal sample and the discriminant feature vector of a product image, so as to obtain the abnormal distribution heat map of the product image. The display module is used to extract the heat regions of the abnormal distribution heat map and display them over the corresponding areas of the product image.

8. The defect detection device according to claim 7, characterized in that, The image acquisition module is an industrial camera with an image resolution greater than 20 megapixels.

9. The defect detection device according to claim 7, characterized in that, The normal sample feature distribution creation module first calculates the average value μ of the discriminant feature vectors of multiple normal sample images. N is the number of normal sample images, V i Let be the discriminant feature vector of the i-th normal sample image; then calculate the multivariate Gaussian distribution of each vector element of the normal sample to obtain the low-rank feature matrix (σ, μ) of the normal sample. σ is the covariance matrix.

10. The defect detection device according to claim 9, characterized in that, The defect detection module calculates the Mahalanobis distance D between each vector element using the low-rank feature matrix (σ, μ) of the normal samples and the feature vector x of the product image.

Images