Model training method and system for domain adaptive small sample defect classification in non-standard shooting scene and application

By optimizing the image embedding extractor and the automatic cropping recommendation module, the problems of low training efficiency and insufficient accuracy of defect detection models in non-standard shooting scenarios of large-scale equipment manufacturing industry are solved, and efficient and accurate defect classification is achieved under small sample data.

CN118918360APending Publication Date: 2024-11-08吴泽蔚
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410929847.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-11
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In large-scale equipment manufacturing, defect detection models under non-standard shooting scenarios require a large number of images for training to achieve good results, and it is difficult to accurately predict the defect category.

Method used

A domain-adaptive few-sample defect classification model for non-standard shooting scenarios is adopted. By optimizing the image embedding extractor, the automatic cropping recommendation module, and the category embedding matrix readjustment module, the accuracy of image embedding and training efficiency are improved, thereby enhancing the accuracy of defect judgment.

Benefits of technology

Achieve more efficient model training and operation on consumer-grade CPUs and GPUs, intelligently analyze key regions of images, and improve the accuracy of defect detection and training efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918360A_ABST
    Figure CN118918360A_ABST
Patent Text Reader

Abstract

The invention discloses a model training method and system for domain adaptive small sample defect classification in a non-standard shooting scene and application, and belongs to the field of convolutional neural networks, the method comprises the following steps: establishing a surface defect small sample data set in the large equipment manufacturing industry, and dividing the data set into a support set and a query set; obtaining a basic model and a defect class metadata set; obtaining an image embedding extractor FQ through iterative learning; obtaining category embedding, and adjusting category embedding through machine learning; the image is input into the model, the automatic cutting module obtains a cut image through training, and the image is cut into a plurality of image blocks; image block embedding of the image blocks is achieved through FQ, and the recommendation module selects the optimal image block as image embedding; comparing representative image embedding with category embedding to determine classification; according to the method, a mode of combining model field self-adaption with image embedding is adopted, related feature information can be better learned for surface defects of small samples in the large equipment manufacturing industry, and the defect detection accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and specifically relates to the fields of computer vision, convolutional neural networks and machine learning, and in particular to a model training method, system and application for domain adaptive small sample defect classification in non-standard shooting scenarios. Background Art

[0002] With the rapid development of large-scale equipment manufacturing industry, the requirements for detecting product defects are increasing day by day. Defect classification is one of the important tasks in the field of computer vision. With the rise of deep learning and convolutional neural networks, the rapid advancement of technology, defect classification tasks have made significant breakthroughs.

[0003] For general defect classification methods, better results can be obtained by relying on large-scale data, and data collection is generally very convenient. However, for large-scale equipment manufacturing industries with strict intellectual property protection, it is not easy to obtain defect data. Therefore, it is very important to explore methods to obtain better prediction results using limited defect data.

[0004] Small sample cross-domain learning aims to effectively transfer knowledge and generalize models between source and target domains through limited labeled samples. Small sample learning can usually adopt data enhancement and augmentation methods. Data enhancement techniques (such as rotation, cropping, flipping, color change, etc.) are used to increase the diversity of training data, which helps the model to better learn the characteristic distribution of data. At the same time, data from other related fields can also be used for augmentation to provide additional information. Generative models such as generative adversarial networks can also be used to generate more samples. By generating samples similar to the target domain, the amount of training data can be increased and the generalization ability of the model can be improved. However, due to intellectual property rights and other reasons, many component defects of large equipment manufacturers are not disclosed, resulting in very few mature defect data sets. Even with the above methods, there is still too little available data.

[0005] In addition, sample generation, feature selection and extraction can be used in domain adaptation. In cross-domain problems, feature selection and extraction are very important. Unsupervised learning or supervised learning methods can be used to select features related to the target domain and ignore features that are not related to the target domain. This helps reduce the model's dependence on source domain data and improves performance on the target domain. Many defect data in large-scale equipment manufacturing industries are taken in non-standardized scenes. In non-standardized scenes, the detection objects and backgrounds are often complex and changeable. For example, factors such as light, angle, occlusion, and deformation affect the core features of the image data. The knowledge learned in the source domain makes it difficult to distinguish the complex features of the target domain. Summary of the invention

[0006] The technical problem to be solved by the present invention is to overcome the problem that the defect detection method based on deep learning model in non-standard shooting scenes in large-scale equipment manufacturing industry has the problem that the training of the model requires a large number of images to achieve better results, which makes it difficult to accurately predict the defect category. In view of the shortcomings of the prior art, the purpose of the present invention is to provide a model training method, system and application for domain adaptive small sample defect classification in non-standard shooting scenes in the field of defective parts identification in large-scale equipment manufacturing industry. This model obtains more accurate image embedding and improves training efficiency through an optimized image embedding extractor; through an automatic cropping recommendation module and a category embedding matrix re-adjustment module, the background is highlighted and the accuracy of defect judgment is improved.

[0007] In order to achieve the above-mentioned invention object, the present invention solves the above-mentioned technical problem through the following technical solution: The technical solution adopted by the present invention includes:

[0008] In a first aspect, the present invention provides a model training method for domain adaptive small sample defect classification in a non-standard shooting scene, and the training steps include:

[0009] 1) Establish a small sample dataset for large-scale equipment manufacturing industry, divide the dataset into a support set and a query set, mark the defective areas of such images, and annotate the label data corresponding to the defective images. Obtain the model and defect class metadata dataset, which is a medium-to-large surface defect dataset, including the label data corresponding to the metadata dataset.

[0010] 2) First, we train the metadata dataset to obtain an image embedding extractor F. M , and then train the data of the support set and query set to obtain an optimized and updated image embedding extractor F Q .

[0011] 3) Input the images of the support set into the optimized image embedding extractor mentioned above to obtain the image embedding of each image, and average the above image embeddings by class to obtain the category embedding. Compare the true classification with the model prediction again, calculate the loss through the cross entropy loss function, and update the category embedding through the back-propagation algorithm.

[0012] 4) Input the image of the support set into the model, and the model will cut the image into multiple image blocks. Each image block contains multiple possible defect block images. These image blocks are compared with the real marked defects, and the L1 regression loss is calculated. The image block segmentation weight table is updated through the back propagation algorithm.

[0013] In a second aspect, the present invention further provides a system for domain adaptive small sample defect classification in a non-standard shooting scene, wherein the system modules include:

[0014] A data enhancement module is used to obtain a model and a training set, wherein the training set includes labeled data and corresponding defect annotations and performs data enhancement on these data. The data enhancement strategies include random flipping, rotation, scaling and translation, etc.;

[0015] The image embedding extractor module is used to initially obtain an image embedding extractor by training the metadata dataset, based on the image embedding extractor F M , and then train the support set to obtain an optimized and updated image embedding extractor F Q ;

[0016] The category embedding matrix re-adjustment module is used to input the images of the support set and obtain the image embedding of each image, and average the image embeddings by class to obtain the category embedding. This module compares the true classification with the model prediction, calculates the loss through the cross entropy loss function, and updates the category embedding through the back-propagation algorithm;

[0017] The automatic cropping module is used to cut the image of the support set into multiple image blocks. Each image block contains multiple possible defect blocks. These image blocks are compared with the real annotated defects, the loss is calculated and reversely updated, and the input query set image is automatically cropped into multiple image blocks;

[0018] The automatic recommendation module inputs the image patch into the above-mentioned image embedding extractor F Q In the process, multiple image block embeddings are output, and an average image block embedding is determined by finding a prototype function. The most irrelevant image block embedding is removed by comparison, and then the average image block embedding is determined in the remaining image blocks, and finally the most representative image block embedding is obtained.

[0019] The similarity comparison classification module is used to compare the cosine similarity between the image block embedding and the above-mentioned category embedding. The classification with the highest similarity is the result of defect classification. It can also be used to compare the similarity between the image block embedding and the category embedding when finding the prototype function.

[0020] In a third aspect, the present invention also provides a defect classification model obtained by training using the above training method.

[0021] In a fourth aspect, the present invention relates to an electronic device that is specifically designed to perform training and operation of an image classification model. The electronic device includes a memory and a processor, wherein a specific computer program is stored in the memory. When the computer program is executed by the processor, it can automatically perform the steps in the aforementioned model training method, or start and run the image classification model. This design enables the electronic device to efficiently perform image classification tasks and provide accurate results to users.

[0022] In a fifth aspect, the present invention further provides a readable storage medium having a computer program stored thereon. When these computer programs are loaded and executed by any suitable computing device, they can automatically execute the specific steps in the aforementioned model training method, or realize the operation of the image classification model. This readable storage medium is not only convenient to carry and transmit, but also ensures the consistency and reliability of the computer program in various environments, providing strong support for the application of the image classification model.

[0023] Based on the above technical solution, compared with the prior art, the beneficial effects of the present invention include:

[0024] The optimized image embedding extractor provided by the present invention can obtain more accurate image embedding and improve the training efficiency of cross-domain small sample models, and can be trained and run on consumer-grade CPUs and GPUs.

[0025] The automatic cropping recommendation module provided by the present invention can intelligently analyze the key areas in the image and automatically recommend the best cropping area, so that the model can better focus on the main part of the image, remove unnecessary interference, highlight defect information, and increase the probability of correct classification.

[0026] The category embedding matrix readjustment module provided by the present invention can intelligently adjust the weight of the category embedding module, so that the category classification of the model is more accurate and the accuracy of defect judgment is improved.

[0027] The above description is only an overview of the technical solution of the present invention. In order to enable those skilled in the art to more clearly understand the technical means of the present application and implement them according to the contents of the specification, the following is a description of the preferred embodiments of the present invention in conjunction with detailed drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification.

[0029] Figure 1 It is a flowchart of a training method provided by a typical implementation case of the present invention;

[0030] Figure 2 It is a schematic diagram of the system structure and process of a defect classification model provided by a typical implementation case of the present invention.

[0031] Figure 3 It is the abstract figure of the present invention. DETAILED DESCRIPTION

[0032] In view of the deficiencies in the prior art, the inventor of this case, after long-term research and practice, was able to propose the technical solution of the present invention.

[0033] The technical solution, its implementation process and principle are further explained as follows. In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention can also be implemented in other ways different from those described herein, and therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0034] Moreover, relational terms such as “first” and “second” are merely used to distinguish one component or method step from another with the same name, but do not necessarily require or imply any such actual relationship or order between these components or method steps.

[0035] See also Figure 1-Figure 2 The embodiment of the present invention provides a specific implementation method of a domain adaptive small sample defect classification model in a non-standard shooting scene. Figure 2 A left rotation is performed, including the following steps:

[0036] S1. Establish a small sample dataset of surface defects in large-scale equipment manufacturing industry, and divide the dataset into a support set and a query set. The dataset includes three categories of defect images, marks the defect areas of such images, and annotates the label data corresponding to the defect images. Establish a metadata dataset covering the strip surface defect dataset and the aluminum surface defect dataset, including the label data corresponding to the metadata dataset. Randomly read 80% of the data in the strip surface defect dataset and the aluminum surface defect dataset as training data, and the remaining 20% ​​of the data as test data for verification.

[0037] S2. Read the model and the data in the defect metadata set that has been enhanced into memory in batches, and perform data enhancement processing such as random flipping, rotation, scaling, and translation on these data. Before training the model on the metadata dataset, initialize the model with the weight table for the ImageNet dataset instead of using a random weight table.

[0038] S3, first obtain an image embedding extractor F by training the metadata dataset M Based on the image embedding extractor, the data of the support set and the query set are input into the model for training to obtain an optimized and updated image embedding extractor F Q .

[0039] S4. Input the support set images into the optimized image embedding extractor to obtain image embeddings for each image, and average the image embeddings by class to obtain category embeddings. The category embeddings are then used to predict the classification through the category embedding adjustment module, and the true classification is compared with the prediction, and the loss is calculated through the cross entropy loss function, and the value of the category embedding is adjusted through the back propagation algorithm.

[0040] S5. Input the query set image into the model. The automatic cropping module will cut the image into multiple image blocks. Each image block contains multiple possible defect blocks. The image blocks are input into the above-mentioned image embedding extractor to output multiple image block embeddings.

[0041] S6. The image block embedding is passed through the automatic recommendation module, and the most irrelevant image block embedding is removed each time, and finally the most representative image block embedding is obtained as the image embedding of the picture.

[0042] S7. Compare the cosine similarity of the most representative image block embedding and the category embedding, and the classification with the highest similarity is the result of defect classification.

[0043] Based on the above technical solution, as some typical application examples, the training steps S3-S6 in the above operation can be implemented by the following steps, and for the convenience of demonstration:

[0044] 1) Input the data in the metadata set into the VGG model without the activation layer and the pooling layer, and then train the support set based on the above weights to obtain an optimized and updated image embedding extractor F M , and then based on the above weights, train the support set to obtain an optimized and updated image embedding extractor F Q .

[0045] 2) Input the image of the support set into the above image embedding extractor F Q , get the image embedding of each picture, and average the above image embeddings by category to get the category embedding. Then input the query set into the image embedding extractor F Q , the obtained image embedding is compared with the true value, the loss is calculated through the cross entropy loss function, and the category embedding is updated through the back-propagation algorithm.

[0046] 3) Input the image of the support set into the model, and the model will cut the image into multiple image blocks. Each image block contains multiple possible defect block images. These image blocks are compared with the real marked defects, the L1 regression loss is calculated, and the image block segmentation weight table is updated through the back propagation algorithm.

[0047] The training process can be summarized as follows: the image embedding extractor is obtained by training with labeled data; the category embedding is obtained through self-learning iteration of the support set data; and the image block embedding is obtained through the automatic cropping recommendation module.

[0048] In some embodiments, step S3 may specifically include:

[0049] First, 80% of the data in the strip surface defect dataset and the aluminum surface defect dataset are randomly read as the training data to start training, and the remaining 20% ​​of the data is used as the test data for verification. Then, a weight table is preliminarily obtained through training on the meta-dataset, and an image embedding extractor is obtained by removing the activation and pooling layers of the feature extraction model VGG, and any picture is converted into a 4096-dimensional image embedding. Based on the image embedding extractor, the support set in the small sample dataset of steel surface defects in the large equipment manufacturing industry is trained in the same way to obtain an optimized and updated image embedding extractor.

[0050] In some embodiments, step S4 may specifically include:

[0051] The images of the support set are input into the above-mentioned optimized image embedding extractor to obtain the image embedding of each image, and the above-mentioned image embeddings are averaged by category to obtain the category embedding.

[0052] The calculation formula for obtaining the image embedding of the support set image is:

[0053] ES i =F (Q) (S i )

[0054] Among them, F (Q) is the image embedding extractor, S i is the i-th data of the support set, ES i Image embedding for this data.

[0055] The calculation formula for the category embedding is:

[0056]

[0057] Among them, μ c (c∈{1,2,3}) is the row vector of the category embedding matrix representing category c. i c represents the image embedding of the i-th sample in category c (c∈{1,2,3}), N c Represents the number of data belonging to category c in the support set S.

[0058] The query set is then fed into the image embedding extractor to obtain the image embeddings. The classification predictions are then compared with the true values.

[0059] The calculation formula for obtaining the image embedding of the query set image is:

[0060] EQ i =F (Q) (Q i )

[0061] The calculation formula for obtaining the classification prediction value of the query set image is:

[0062]

[0063] The loss is calculated by the cross entropy loss function and the category embeddings are updated by the back-propagation algorithm.

[0064] The formula for calculating the loss and updating the class embedding is:

[0065]

[0066] In some embodiments, steps S5 and S6 may specifically include:

[0067] Input the query set image into the automatic cropping recommendation module of the model, which will cut the image into multiple image blocks, each of which contains one or more possible defects. The image blocks are input into the above-mentioned image embedding extractor, which outputs multiple image block embeddings and obtains a representative image block embedding by finding the prototype function.

[0068] The calculation formula for obtaining the image block embedding is:

[0069] E i,j =F (Q) (Q i,j )

[0070] The calculation formula for obtaining the average embedding is:

[0071]

[0072] The calculation formula for calculating the minimum loss value is:

[0073]

[0074] In some embodiments, step S7 may specifically include:

[0075] The cosine similarity of the most representative image block embedding and the category embedding is compared, and the classification with the highest similarity is the result of defect classification.

[0076] The similarity calculation formula between the above category embedding and image block embedding is:

[0077]

[0078] The above calculation formula for image defect classification is:

[0079] c = argmax c cossim(u c * ,Ei )

[0080] Similarly, an embodiment of the present invention also provides an image classification model trained by the above training method.

[0081] The above training method is applied to a small sample set, but in some embodiments, the iteration of the meta-training part in the training method can be performed in batches, that is, the training set is divided into multiple batches, and each batch corresponds to one iteration. The number of data in the batch can be a number greater than one, such as 2-256, etc., or only one, which is equivalent to not performing the training in batches.

[0082] Based on the above technical solution, in practical applications, the execution steps of the training method of the above meta-training set may be:

[0083] (1) Create and initialize the model. Secondly, the training set consists of a small amount of labeled data and a large amount of unlabeled data. The labeled data and unlabeled data are divided into batches of specific sizes and input into the model for training in sequence.

[0084] (2) For small batches of labeled metadata data, data enhancement is performed. The corresponding formula can be as follows:

[0085]

[0086] Where X i Represents the i-th image data in the batch, indicating the data augmentation operation. X i N Represents the image data after data augmentation processing.

[0087] (3) After the above operations, the input of the model will be obtained. Then the model's prediction and the corresponding true value label are used to calculate the loss value. The cross entropy function is used as the loss function, and the formula is as follows:

[0088]

[0089] where Y (1) represents the probability vector predicted by the base model, Y (t) Represents the one-hot encoding of the data after ground truth label encoding.

[0090] As some typical application examples of the above technical solutions, the embodiments of the present invention also apply the above training methods and systems, actually train multiple models, and compare them with various training methods and systems in the prior art, especially their selection of the number of labeled data and recognition accuracy.

[0091] The present invention is further described in detail with reference to specific embodiments below.

[0092] This embodiment illustrates the comparison between the training method provided by the present invention and the existing small sample classification model training method:

[0093] Table 1 Pre-training accuracy at different iterations

[0094]

[0095] In the pre-training stage, the VGG16 network is used as the image embedding extraction network, and the stochastic gradient descent (SGD) algorithm is used to train the model. When training the metadata dataset, the batch size used is 320 and the initial learning rate is 0.001. This example uses data augmentation methods in all experiments, including random flipping, rotation, scaling, and translation.

[0096] Table 2 Accuracy comparison experiment

[0097]

[0098] In order to verify the effectiveness of this model, experiments were conducted on the 3way 3shot setting. With limited samples, each model showed different accuracy in the domain adaptation task. On a large-scale equipment manufacturing industry small sample dataset, Table 2 lists the comparison results of the classification model described in this example with the small sample classification models ProtoNet, MatchingNet and the general classification model VGGNet.

[0099] The expressiveness of this model is not limited to judging the known defect categories in the pre-training set, but can also predict defect categories that do not exist in the meta-training set. This experiment was conducted in a 1way 3shot scenario, that is, only one category of defect samples was provided, and there were only 3 pictures of this category as reference.

[0100] Table 3 Accuracy of identifying stain defects that do not exist in pre-training

[0101] 1way 3shot This model ProtoNet MatchingNet VGG16 Accuracy 83.33% 66.67% 66.67% 16.67%

[0102] In the 1way 3shot scenario, the recognition rates of the small sample classification models ProtoNet, MatchingNet, and the common classification model VGGNet for defect classes that have never appeared in this example.

[0103] It can be seen from Table 2 that the recognition method described in this embodiment effectively improves the training efficiency of the classification model and the classification accuracy of the image classification model.

[0104] Specific embodiments of the present invention may be embodied as systems, methods and / or computer program products, wherein the computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to execute various aspects of the present invention.

[0105] A computer readable storage medium is a tangible device for holding and storing instructions used by an instruction execution device. The medium includes, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (a non-exhaustive list) are: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, and any suitable combination of the above.

[0106] It should be emphasized that, although the above describes each step in a specific order, it does not mean that these steps must be performed in the above specific order. In fact, some of these steps can be performed concurrently or even in a different order, as long as the required functions can be achieved.

[0107] It should be understood that the embodiments described above are only some embodiments of the present invention, rather than all embodiments. The detailed description of the embodiments of the present invention is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should be deemed to fall within the scope of protection of the present invention.

Claims

1. A model training method for domain adaptive small sample defect classification in non-standard shooting scenes, characterized in that: include: 1) Establish a small sample dataset of steel surface defects in large-scale equipment manufacturing industry, and divide the dataset into a support set and a query set. The dataset includes three categories of defect images, marks the defect areas of such images, and annotates the label data corresponding to the defect images. 2) Obtain the basic model and defect class metadata data set, read the basic model completely into the cache, and the metadata data set covers the strip surface defect data set and the aluminum surface defect data set, including the label data corresponding to the metadata data set. 3) First, we train the metadata dataset to obtain an image embedding extractor F. M The extractor transforms the image into a 4096-dimensional image embedding by mapping, and then obtains an image embedding extractor F that is more suitable for small sample sets through training on the support set and the query set. Q . 4) Input the support set data into the model through F Q After obtaining the image embedding, the category embedding and re-adjustment module first performs classification prediction, then compares the predicted classification with the true classification and calculates the loss, and updates the category embedding through the back-propagation algorithm. 5) Input the support set images into the model, train the automatic cropping module, input the query set images, and crop them into multiple defect block images. 6) Input the defect block image into the image embedding extractor F Q In the process, multiple image block embeddings are output, and the most representative image block embedding is obtained as the image embedding through the image block recommendation module. 7) Compare the above image embedding with the above category embedding by cosine similarity, and the classification with the highest similarity is the result of defect classification.

2. The training method according to claim 1, characterized in that: Step 3) specifically includes: First, a weight table is initially obtained by training the metadata dataset, and then an image embedding extractor is obtained by removing the model activation and pooling layers, which converts any image into a 4096-dimensional image embedding. Based on the image embedding extractor F M , and then train the support set in the small sample data set of defects in large equipment manufacturing industry, and obtain an image embedding extractor F that is more suitable for small sample data sets through optimization iteration. Q .

3. The training method according to claim 1, characterized in that: In step 4), updating the category embedding specifically includes: The image embedding extractor extracts the image embedding of the query set and performs matrix multiplication with the matrix composed of the above-mentioned category embedding. The normalized exponential function is then used to calculate the probability of the multiplication result belonging to each class. The true classification is compared with the classification predicted by the normalized exponential function, the loss is calculated through the cross entropy loss function, and the value of the category embedding matrix is ​​updated through the back propagation algorithm.

4. The training method according to claim 1, characterized in that: In step 6), the method for generating a representative picture block specifically includes: The image block embeddings of each image are averaged to obtain the image block average embedding. Each image block embedding is compared with the image block average embedding by cosine similarity. The most dissimilar image block embedding is removed. The remaining image block embeddings are averaged to obtain a new image block average embedding. The remaining image block embeddings are then compared with the new image block average embedding. The most dissimilar image block embedding is removed. This process is repeated until the most representative image block embedding is left. This image block embedding is used as the recommended image embedding.

5. The training method according to claim 3, characterized in that: Compute category embeddings: The calculation formula for category embedding is: Among them, μ c (c∈{1,2,3}) is the row vector of the category embedding matrix representing category c. i c represents the image embedding of the i-th sample in category c (c∈{1,2,3}), N c Represents the number of data belonging to category c in the support set S.

6. The training method according to claim 3, characterized in that: The probability of belonging to each class is calculated by multiplying the result with a normalized exponential function; The probability formula for predicting that an image belongs to each class is: Where μ and B refer to the category embedding matrix and bias term of dimension K*4096, respectively, where μ is a matrix composed of μ1, μ2, and μ3. Q , the sample is projected from the three-dimensional visual space to the 4096-dimensional feature space. Q (S i ) indicates S i Image embedding, represents the predicted probability of belonging to each category.

7. The training method according to claim 3, characterized in that: Update the category embedding matrix through the back-propagation algorithm; The formula is: in, is the predicted value of the model, y i (S) is the true value of the label, N S represents the number of data in the support set, μ1, μ2, and μ3 represent the category embeddings of different defect categories, and μ is the weight matrix composed of (μ1, μ2, and μ3). Through the back propagation algorithm, μ is updated to μ * .

8. The training method according to claim 4, characterized in that: Get image embeddings of representative image patches; The formula for embedding image patches for each image is: E i,j =F (Q) (Q i,j ) F (Q) is the image embedding extractor, E i,j It's Q i,j The embedding of the corresponding image patch. The formula for averaging the image patch embeddings for each image is: Among them, E i,1 , E i,2 . …, E i,n is the image patch embedding of N image patches, AE i is the average embedding of the image block. N Ei Indicates the total number of image blocks of the i-th data in the dataset Q; E i,1 , E i,2 . …, E i,n With AE i Perform similarity comparison and remove the most dissimilar image embedding; recalculate the average embedding of the remaining image blocks, and then remove the most dissimilar image embedding until only one image block is left embedded in E i,j* That is the most representative image block embedding mentioned above.

9. The training method according to claim 1, in step 7), characterized in that: Image embeddings are compared to category embeddings, including: Its related formula: c=argmax c cossim(μ c * ,AND i ) Represents the dot product of two vectors. E i represents the i-th image embedding in the dataset Q, μ c * is μ * Any row vector of , c is the final classification result.

10. An image classification model obtained by training using the training method described in any one or more of claims 1-8.