A small sample-based defect incremental learning method and device

Abstract

The application discloses a small sample-based defect incremental learning method and device. The method comprises the following steps: training a base-class defect anomaly classification model, extracting first-class features and first-class parameters by using the trained base-class defect anomaly classification model, and putting the first-class features into a first-class feature set and the first-class parameters into a first-class parameter set; constructing an incremental defect anomaly classification model with the same number of small sample incremental defect picture data sets, training each incremental defect anomaly classification model, extracting second-class features and second-class parameters by using the trained incremental defect anomaly classification model, then updating the class parameters of an incremental classifier by using the first-class parameter set, and training the incremental classifier by using the first-class feature set; and obtaining the trained incremental classifier after the training of all the incremental defect anomaly classification models is completed. The application avoids the problems of class imbalance and trade-off between new and old classes caused by small samples.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of defect detection, and more particularly to a defect incremental learning method and device based on a small sample. BACKGROUND

[0002] Improving the quality of the outer packaging of cigarette products is of great significance to the production enterprise. A good outer packaging not only represents the production level and brand image of the enterprise, but also affects the satisfaction of consumers to the enterprise. At the same time, reducing the generation of outer packaging defects can also improve the yield and reduce the production cost. However, in the process of cigarette manufacturing, due to the influence of production raw materials, machine failure, external environment of the factory and other factors, various defects are easy to appear in the outer packaging of cigarettes. In order to ensure product quality, defective products must be detected and removed in time. Especially in some specific scenarios, different defect types need to be classified, because identifying different defect types helps to find the cause of the defect, so as to take relative measures to fundamentally reduce the number of defective products. However, the model trained based on the existing defect types cannot identify new outer packaging defect types, especially in the actual cigarette production process, new types of small defect samples will continue to appear, which brings great challenges to anomaly detection and classification. The current anomaly detection method mainly identifies by artificial naked eye, which is slow and low in accuracy.

[0003] With the rise of artificial intelligence and deep learning in recent years, the cigarette outer packaging defect anomaly detection and classification method based on computer vision technology gradually replaces the manual detection method. Among them, the reconstruction-based cigarette defect anomaly detection method adopts the network structure of autoencoder, and discriminates the anomaly through the large reconstruction error generated after the cigarette defect picture is input into the model. The method based on representation discriminates the anomaly by calculating the distance between the input picture and the normal cigarette picture in the feature space. However, these anomaly detection methods cannot classify the cigarette defects. Under the problem setting of incremental learning, in addition to the initially existing defect categories, new defect categories will also come in batches in the subsequent stage, and the model needs to continuously learn the knowledge of new categories, which is very suitable for the dynamic change scene in the actual factory.

[0004] The method commonly used in the current incremental learning problem is to save the knowledge of the old class based on knowledge distillation, and learn the new defect class without forgetting the old knowledge. However, when the number of incremental defect type pictures is very small, the method based on knowledge distillation is prone to class imbalance, and the knowledge distilled will be more inclined to the class with a large number of defect pictures. At the same time, due to the need for a larger learning rate and loss function to learn from a small number of cigarette defect pictures, the ordinary knowledge distillation method will have the problem of trade-off between new and old classes. Therefore, the existing knowledge distillation method is difficult to adapt to the small sample cigarette defect class incremental learning problem, which faces great challenges. SUMMARY

[0005] The present application provides a small sample based defect incremental learning method and device, which trains a base class defect anomaly classification model and a plurality of incremental defect anomaly classification models respectively, extracts class features based on defect pictures and class parameters of the classification model using the trained classification model, and realizes incremental learning by interacting and knowledge transfer using all class features and class parameters, thereby utilizing existing defect classification knowledge and extracting defect classification knowledge in the small sample dataset, avoiding the problem of class imbalance caused by small samples, and avoiding the trade-off between new and old classes.

[0006] The present application provides a small sample based defect incremental learning method, comprising:

[0007] An incremental learning dataset is constructed, which includes a base class defect picture dataset and at least one small sample incremental defect picture dataset;

[0008] The base class defect anomaly classification model is trained using the base class defect picture dataset, and the first class features corresponding to all defect classes in the base class defect picture dataset are extracted using the trained base class defect anomaly classification model, and the first class parameters of the trained base class defect anomaly classification model are obtained, and the first class features are put into the first class feature set, and the first class parameters are put into the first class parameter set;

[0009] An incremental defect anomaly classification model is constructed, which is the same as the number of small sample incremental defect picture datasets, each incremental defect anomaly classification model is trained, and the second class features corresponding to all defect classes in the corresponding small sample incremental defect picture dataset are extracted using the trained incremental defect anomaly classification model, and the second class parameters of the trained incremental defect anomaly classification model are obtained, and all second class features are put into the first class feature set, and all second class parameters are put into the first class parameter set, then the class parameters of the incremental classifier are updated using the first class parameter set, and the incremental classifier is trained using the first class feature set;

[0010] After training of all the incremental defect anomaly classification models is completed, the trained incremental classifiers are obtained.

[0011] Preferably, the base defect anomaly classification model comprises an encoder, a first classifier and a mapping network connected in sequence.

[0012] The encoder is configured to extract a first feature vector of a base defect image in the base defect image dataset.

[0013] The first classifier is configured to predict a probability of each defect category according to each first feature vector.

[0014] The mapping network is configured to extract an embedding feature of the corresponding base defect image according to the first feature vector.

[0015] Preferably, when training the base defect anomaly classification model, the loss function comprises a first cross-entropy loss function and a contrastive loss function, and the contrastive loss function is obtained based on positive sample embedding feature vectors and negative sample embedding feature vectors obtained by the encoder and the mapping network respectively for positive samples and negative samples in the base defect image dataset.

[0016] Preferably, each incremental defect anomaly classification model comprises a trained encoder and a second classifier obtained after training of the base defect anomaly classification model.

[0017] When training each incremental defect anomaly classification model, the trained encoder is used to extract features of incremental defect images in the corresponding small sample incremental defect image dataset to obtain second feature vectors, and the second classifier obtains a predicted classification result of the incremental defect images based on the second feature vectors.

[0018] Preferably, the class parameters of the incremental classifier are updated using the first set of class parameters, specifically including:

[0019] Mapping all class features in the first set of class features and all class parameters in the first set of class parameters to nodes in the graph network model;

[0020] Calculating third class parameters of the incremental classifier according to node information of the graph network model to form a second set of class parameters, and updating the first set of class parameters using the second set of class parameters;

[0021] Updating parameters of the incremental classifier using a parameter matrix formed by splicing all third class parameters in the second set of class parameters.

[0022] The application also provides a small sample-based defect incremental learning device, comprising a first construction module, a first training module, a second training module and an incremental classifier obtaining module.

[0023] The first construction module is configured to construct an incremental learning dataset, the incremental learning dataset comprising a base-class defect picture dataset and at least one small-sample incremental defect picture dataset;

[0024] The first training module is configured to train the base-class defect anomaly classification model using the base-class defect picture dataset, extract first-class features corresponding to all defect categories in the base-class defect picture dataset using the trained base-class defect anomaly classification model, obtain first-class parameters of the trained base-class defect anomaly classification model, and put the first-class features into a first-class feature set and the first-class parameters into a first-class parameter set;

[0025] The second training module is configured to construct as many incremental defect anomaly classification models as the number of the small-sample incremental defect picture datasets, train each incremental defect anomaly classification model, extract second-class features corresponding to all defect categories in the corresponding small-sample incremental defect picture dataset using the trained incremental defect anomaly classification model, obtain second-class parameters of the trained incremental defect anomaly classification model, put all the second-class features into the first-class feature set and all the second-class parameters into the first-class parameter set, and then update the class parameters of the incremental classifier using the first-class parameter set and train the incremental classifier using the first-class feature set;

[0026] The incremental classifier obtaining module is configured to obtain the trained incremental classifier after training of all the incremental defect anomaly classification models is completed.

[0027] Preferably, the base-class defect anomaly classification model comprises an encoder, a first classifier and a mapping network connected in sequence.

[0028] The encoder is configured to extract first feature vectors of base-class defect pictures in the base-class defect picture dataset.

[0029] The first classifier is configured to predict probabilities of each defect category according to each first feature vector.

[0030] The mapping network is configured to extract embedding features of the corresponding base-class defect picture according to the first feature vector.

[0031] Preferably, when training the base-class defect anomaly classification model, the loss function comprises a first cross-entropy loss function and a contrastive loss function, and the contrastive loss function is obtained based on positive sample embedding feature vectors and negative sample embedding feature vectors obtained by the encoder and the mapping network respectively for positive samples and negative samples in the base-class defect picture dataset.

[0032] Preferably, each incremental defect anomaly classification model comprises a trained encoder and a second classifier obtained after training of the base-class defect anomaly classification model.

[0033] The second training module is configured to, when training each incremental defect anomaly classification model, extract features of the incremental defect pictures in the small sample incremental defect picture dataset by using the trained encoder to obtain a second feature vector, and obtain a predicted classification result of the incremental defect picture based on the second feature vector by using the second classifier.

[0034] Preferably, the second training module comprises a second construction module, a mapping module, a second parameter set obtaining module and an updating module.

[0035] The second construction module is configured to construct a graph network model and an incremental classifier.

[0036] The mapping module is configured to map all class features in the first class feature set and all class parameters in the first class parameter set to nodes in the graph network model.

[0037] The second parameter set obtaining module is configured to calculate third class parameters of the incremental classifier according to node information of the graph network model, form a second class parameter set, and update the first class parameter set by using the second class parameter set.

[0038] The updating module is configured to update parameters of the incremental classifier by using a parameter matrix formed by splicing all third class parameters in the second class parameter set.

[0039] Other features and advantages of the present application will become apparent from the following detailed description of illustrative embodiments thereof, which proceeds with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings incorporated in and forming a part of the specification, illustrate embodiments of the present application and, together with the description, serve to explain the principles of the present application.

[0041] Figure 1 A flowchart of a small sample based defect incremental learning method provided by the present application;

[0042] Figure 2 A schematic diagram of an embodiment of pre-processing of an original picture;

[0043] Figure 3 A flowchart of updating class parameters of an incremental classifier by using a first class parameter set provided by the present application;

[0044] Figure 4 A structural diagram of a small sample based defect incremental learning device provided by the present application. DETAILED DESCRIPTION

[0045] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and numerical values of components and steps set forth in these embodiments are not limiting to the scope of the present application unless specifically stated otherwise.

[0046] The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the application or its application or uses.

[0047] Techniques, methods, and apparatus known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered part of the specification.

[0048] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary, and not as a limitation. Thus, other examples of the exemplary embodiments can have different values.

[0049] The present application provides a small sample-based defect incremental learning method and device, trains a base class defect anomaly classification model and a plurality of incremental defect anomaly classification models respectively, extracts class features based on defect pictures and class parameters of the classification model using the trained classification model respectively, and realizes incremental learning by interacting and knowledge transfer using all class features and class parameters, thereby utilizing existing defect classification knowledge and extracting defect classification knowledge in the small sample dataset, avoiding the problem of class imbalance caused by small samples, and avoiding the trade-off problem between new and old classes. Moreover, the present application uses the trained encoder obtained after training the base class defect anomaly classification model as the encoder of the incremental defect anomaly classification model, thereby utilizing the feature extraction advantage of the base class defect anomaly classification model, but the two use different classifiers, so the base class defect classification knowledge will not affect the classification of incremental defects, ensuring the prediction accuracy of small sample incremental data. Moreover, the present application fuses the class features and class parameters of the base class and the increment together through a graph attention network to train the incremental classifier, so that the incremental classifier learns both the classification knowledge of the base class defect and the classification knowledge of the incremental defect, improving the learning ability of the incremental classifier for small sample defect types.

[0050] As shown in Figure 1 The small sample-based defect incremental learning method provided by the present application includes:

[0051] S110: Construct an incremental learning dataset, which includes a base class defect picture dataset and at least one small sample incremental defect picture dataset.

[0052] The present application will be described below taking the detected object as a cigarette outer package as an example. In step S110, first, the collected cigarette outer package original pictures are preprocessed,Figure 2 A schematic diagram of one embodiment of preprocessing the original picture is shown. Specifically, the redundant information of the cigarette outer package original picture taken on the pipeline is cut off, and only the information of the cigarette package in the center of the picture is retained. As one embodiment, all pictures are preprocessed to a size of 3*256*256.

[0053] An incremental learning data set is constructed using the preprocessed cigarette outer package defect picture, and the incremental learning data set includes a base class defect picture data set D0 and at least one small sample incremental defect picture data set D i ,D i ∈{D1,D2,…,D n}。

[0054] Among them, the base class defect picture data set D0 contains a large amount of labeled data of multiple defect categories, and the small sample incremental defect picture data set D i contains a small amount of labeled data of a small number of defect categories.

[0055] It should be noted that the defect categories in the small sample incremental defect picture data set D i may be known defect categories or newly added defect categories. Table 1 shows one embodiment of the incremental learning data set, wherein the base class defect picture data set D0 contains 6 defect categories, each of which has sufficient defect pictures (30), and is divided into training set and test set Each small sample incremental defect picture data set D i contains 2 defect categories, and each defect category contains 5 training pictures in the training set ; in the test set , each defect category contains a large number of cigarette defect pictures (50). Among them, small sample incremental defect picture data sets D4, D5, D6 have the same defect types as D1, D2, D3 respectively, in order to prevent class imbalance problem, they share a test set.

[0056] Table 1: Overview of small sample incremental learning data set D of cigarette outer package defect picture

[0057]

[0058] S120: training the base class defect anomaly classification model using the base class defect picture data set D0, and extracting the first class features corresponding to all defect categories in the base class defect picture data set D0 using the trained base class defect anomaly classification model M0 and obtaining the first class parameters h c1 of the trained base class defect anomaly classification model M0, and putting the first class features into the first class feature set Z Iputting the first type of parameters into a first type of parameter set H I . Wherein, c represents different defect categories.

[0059] As an embodiment, the base-class defect anomaly classification model comprises an encoder Enc, a first classifier F0 and a mapping network Proj which are sequentially concatenated. The base-class defect anomaly classification model can be composed by sequentially adding the first classifier F0 and the mapping network Proj behind the encoder Enc.

[0060] The encoder Enc is a multi-layer 2D convolutional neural network, which comprises a pre-trained ResNet18 network model and an adaptive average pooling layer. The encoder Enc is used to map the base-class defect pictures x in the base-class defect picture dataset to the latent space and convert them into first feature vectors z, K represents the number of features of each first feature vector.

[0061] The first classifier F0 is a linear fully connected layer. As an embodiment, the first classifier F0 is a one-layer linear mapping layer. The first classifier F0 is used to predict the probability of each defect category according to each first feature vector.

[0062] The mapping network Proj comprises a connection layer, a Relu activation function, a fully connected layer and a normalization layer which are sequentially concatenated. As an embodiment, the mapping network Proj comprises a linear mapping layer, a Relu activation function, a linear mapping layer and a Normalize layer. The mapping network Proj is used to extract the embedding features of the corresponding base-class defect picture according to the first feature vector.

[0063] Before training the base-class defect anomaly classification model using the base-class defect picture dataset D0 (actually , it is necessary to set the training round number, learning rate, loss function weight and other hyperparameters. When training the base-class defect anomaly classification model, all base-class defect pictures x k in a training batch K in are input into the base-class defect anomaly classification model, and the predicted classification result set and the embedding feature set z K are obtained through the first classifier F0 and the mapping network Proj respectively. Wherein, k ∈ K ≡ {1, …, n} represents the label of all base-class defect pictures in a training batch, and n represents the number of pictures in a training batch.

[0064] When training the base-class defect anomaly classification model, the loss function L total comprises a first cross-entropy loss function L ce1 and a contrastive loss function L ct , that is

[0065] L total = λL ce1 + (1 - λ)L ct (1)

[0066] wherein λ is a weight coefficient of the first cross-entropy loss function.

[0067] Specifically, the first cross-entropy loss function L is calculated according to a set of predicted classification results K and a set of true value labels y ce1 :

[0068]

[0069] wherein y k is a true label of the kth base class defect picture x k , c k is a defect class in the true label of the kth base class defect picture x k , and is a probability of the defect class c k in the predicted classification result of the kth base class defect picture x k .

[0070] The contrast loss function L ct is obtained based on positive sample embedding feature vectors and negative sample embedding feature vectors obtained by the encoder and the mapping network respectively from positive samples and negative samples in the base class defect picture dataset. Specifically, the contrast loss function L K is calculated according to a set of embedding features z ct :

[0071]

[0072] wherein P(k) represents all positive samples of the kth base class defect picture x k in the training batch; |P(k)| represents the number of positive samples; A(k) represents all negative samples of the kth base class defect picture x k in the training batch; p represents a positive sample of the kth base class defect picture x k in the training batch; a represents a negative sample of the kth base class defect picture x k = Proj(Enc(x k )), represents an embedding feature vector obtained after the kth base class defect picture x k passes through the encoder and the mapping network; and p = Proj(Enc(x p )), represents an embedding feature vector obtained after the pth positive sample x pThe positive sample embedding feature vector obtained after the encoder and mapping network; z a =Proj(Enc(x a )), represents the a-th negative sample x a The negative sample embedding feature vector obtained after the encoder and mapping network; τ is a temperature hyperparameter.

[0073] During the training process, the loss function L is minimized total The base class defect anomaly classification model is trained for the target. After reaching the set number of iterations, the trained base class defect anomaly classification model M0 is obtained and the parameters of the encoder Enc are frozen.

[0074] Specifically, in each iterative training, the loss function L is obtained total Finally, the SGD optimizer is used to update the network weights of the base class defect anomaly classification model.

[0075] When extracting the first type of features When the base class defect image dataset D0 (actually ) for each defect category in all base class defect images Input the trained base class defect anomaly classification model and use the encoder Enc with frozen parameters to calculate the first class feature of the cth defect category

[0076]

[0077] Among them, N c represents the set of all base class defect images with defect category c, |N c | represents the number of base class defect images of the c-th defect category, represents the i-th base class defect image of the c-th defect category, Represents the feature vectors of all base class defect images of the c-th defect category extracted by the encoder with frozen parameters.

[0078] Will The first-class features of all defect categories in Add to the first type feature set Z I middle, Among them, N0 is the number of defect categories in the base class defect image dataset D0.

[0079] After the base class defect anomaly classification model training is completed, the parameter matrix U0 of the first classifier F0 is extracted. C represents the number of channels.

[0080] According to the parameter matrix U0 of the first classifier, the first class parameter h of each defect category is extracted. c1 , And the first type parameter h c1 Add to the first parameter set H I middle,

[0081] S130: Construct incremental defect anomaly classification models with the same number as the small-sample incremental defect image dataset, train each incremental defect anomaly classification model, and use the trained incremental defect anomaly classification model to extract the second-category features corresponding to all defect categories in the corresponding small-sample incremental defect image dataset, and obtain the second-category parameters of the trained incremental defect anomaly classification model, and put all the second-category features into the first-category feature set, and put all the second-category parameters into the first-category parameter set, and then use the first-category parameter set to update the class parameters of the incremental classifier, and use the first-category feature set to train the incremental classifier.

[0082] Each incremental defect anomaly classification model M i (i>0) includes the trained encoder Enc and the second classifier F obtained after the base class defect anomaly classification model training is completed i ( where N i Denotes the i-th small sample incremental defect image dataset D i The second classifier F i is a linear fully connected layer. The second classifier F i Adding it to the encoder Enc can form the incremental defect anomaly classification model M i .

[0083] Before training each incremental defect anomaly classification model, it is necessary to set hyperparameters such as the number of training rounds and the learning rate. When training each incremental defect anomaly classification model, the incremental defect anomaly classification model M is used. i The trained encoder Enc corresponds to the small sample incremental defect image dataset D i (Actually ) to extract features from the incremental defect images and obtain the second feature vector. The second classifier F i Obtain the predicted classification results of the incremental defect images based on the second eigenvector, and obtain the predicted classification result set

[0084] Train the incremental defect anomaly classification model M i When , the loss function is the second cross entropy loss function L ce2 , please refer to formula (2) for calculation. In each iteration of training, the SGD optimizer is used to update the classifier F i To minimize the second cross entropy loss function Lce2 targeted to the incremental defect anomaly classification model M i in the second classifier F i is trained to obtain the trained second classifier F i , and finally the trained incremental defect anomaly classification model M i is obtained.

[0085] The second-class features corresponding to all defect categories in the small sample incremental defect picture data set are extracted from the trained incremental defect anomaly classification model M i and the second-class parameters h of the trained incremental defect anomaly classification model are obtained c2 . Please refer to the obtaining process of the first-class features and the first-class parameters. The second-class features are added to the first-class feature set Z I , where N k is the number of class parameters of the kth incremental defect anomaly classification model. The second-class parameters h c2 are added to the first-class parameter set H I , H I = {h 1 , h 2 , …, h m}.

[0086] It should be noted that after the training of each incremental defect anomaly classification model M i , the class parameters of the incremental classifier F I are updated using the first-class parameter set H t , and the incremental classifier F I is trained using the first-class feature set Z t .

[0087] As an embodiment, the incremental classifier is a linear fully connected layer.

[0088] When updating the class parameters of the incremental classifier F t , a graph network model G is constructed.

[0089] As an embodiment, the graph network model G includes a learnable linear mapping function and a linearly transformed weight matrix W,

[0090] As Figure 3 shown, the class parameters of the incremental classifier are updated using the first-class parameter set, specifically including:

[0091] S310: The first-class feature set Z​I All class features in and the first type parameter set H I All class parameters h c Mapped to the nodes in the graph network model G:

[0092]

[0093]

[0094]

[0095]

[0096] in, Class features The class feature vector representation of the corresponding node in the graph network model G, is the class parameter h c The class parameter vector representation of the corresponding node in the graph network model G is: is the first type feature set Z I The class feature node set of all class features in the graph network model G, is the first type parameter set H I The class parameter node set of all class parameters in the graph network model G.

[0097] S320: Calculate the third type parameters of the incremental classifier based on the node information of the graph network model to form the second type parameter set, and use the second type parameter set H′ I Update the first type parameter set H I .

[0098] Specifically, first, according to the class feature node set and class parameter node collections Calculate the inter-class correlation coefficient a of the image features respectively ij and the inter-class correlation coefficient b of the classifier ij Then, according to the correlation coefficient image feature inter-class correlation coefficient a ij , classifier inter-class correlation coefficient b ij and the first parameter set H I Calculate the third class parameter h of the incremental classifier i′ , all third-category parameters form the second-category parameter set H′ I .

[0099] Specifically, the inter-class correlation coefficient a of the image features is calculated by the following formula: ij and the inter-class correlation coefficient b of the classifier ij :

[0100]

[0101]

[0102]

[0103]

[0104] in, is a class feature node pair, is a class parameter node pair, <,> represents the inner product of two vectors.

[0105] The third parameter h is calculated using the following formula: i′ and the second parameter set H′ I :

[0106]

[0107] H′ I ={h 0′ ,h 1′ ,…,h m′} (14)

[0108] Among them, μ represents the inter-class correlation coefficient a of the image feature ij and the inter-class correlation coefficient b of the classifier ij The weight coefficient between i is the first type parameter set H I The i-th class parameter in .

[0109] S330: Using the second parameter set H′ I All third-category parameters h in i′ The parameter matrix formed by splicing Update the incremental classifier F t Parameters.

[0110] It should be noted that in the small sample incremental defect image dataset D i When the defect categories in are all known defect categories (such as the small sample incremental defect image dataset D4, D5, and D6 in Table 1), the following method is used:

[0111] Taking D4 as an example, since the defect category in D4 has already appeared in D1, in the incremental defect anomaly classification model training phase, it is only necessary to fine-tune the model parameters based on the classifier F1. After the training is completed, the first type parameter set H of the classifier is updated. I Parameters of the corresponding category in Instead of going to the first parameter set H I Add a new vector to represents the classifier parameter set obtained by training on D1; meanwhile, the first-class feature set obtained by D4 is used updating the first-class feature set Z I the first-class feature extracted in D1 represents the first-class feature extracted on the ith data set, and the remaining steps are consistent with the case of a new class, so that a smaller learning rate is used when training the graph network model G.

[0112] As an embodiment, the incremental classifier F is trained by training the graph network model G t . Specifically, all third-class features in the first-class feature set Z I are input into the incremental classifier F t , the corresponding predicted defect class is obtained, and the second cross-entropy loss function L l is calculated according to the true label y ce2 corresponding to the third-class feature and the predicted defect class . Please refer to formula (2).

[0113] The graph network model is iteratively trained to minimize the second cross-entropy loss function, and the trained graph network model is obtained, thereby obtaining the trained incremental classifier.

[0114] Specifically, after obtaining the second cross-entropy loss function L ce2 in each iteration, the network weights of the graph network model G are updated using the SGD optimizer.

[0115] S140: After training all incremental defect anomaly classification models, the trained incremental classifier is obtained.

[0116] Table 2 lists the classification accuracy of different methods on the incremental learning data set of cigarette outer packaging. Among them, the method of freezing the encoder only uses the ordinary cross-entropy loss function to train the encoder, and after freezing the encoder parameters, the classifier parameters are fine-tuned using the incremental data set. The contrast learning method uses a supervised contrast loss to train the encoder, and then freezes the encoder parameters and fine-tunes the classifier using the incremental data set. From the results in Table 2, it can be seen that the incremental learning algorithm based on the class feature and the class parameter interaction of the classification model of the defect picture, the contrast learning and the freezing of the encoder proposed in the present application has higher classification accuracy than the other two methods, which shows that the present application has good learning ability for new classes while preserving the original class knowledge.

[0117] Table 2 comparison results of different methods

[0118]

[0119] Based on the above small sample based defect incremental learning method, the application further provides a small sample based defect incremental learning device. As shown in the figure, the small sample based defect incremental learning device comprises a first construction module 410, a first training module 420, a second training module 430 and an incremental classifier obtaining module 440. Figure 4

[0120] The first construction module 410 is used for constructing an incremental learning data set, and the incremental learning data set comprises a base class defect picture data set and at least one small sample incremental defect picture data set.

[0121] The first training module 420 is used for training a base class defect anomaly classification model by using the base class defect picture data set, extracting first class features corresponding to all defect categories in the base class defect picture data set by using the trained base class defect anomaly classification model, obtaining first class parameters of the trained base class defect anomaly classification model, and putting the first class features into a first class feature set and the first class parameters into a first class parameter set.

[0122] The second training module 430 is used for constructing an incremental defect anomaly classification model with the same number as the small sample incremental defect picture data set, training each incremental defect anomaly classification model, extracting second class features corresponding to all defect categories in the corresponding small sample incremental defect picture data set by using the trained incremental defect anomaly classification model, obtaining second class parameters of the trained incremental defect anomaly classification model, putting all second class features into the first class feature set and all second class parameters into the first class parameter set, then updating the class parameters of the incremental classifier by using the first class parameter set, and training the incremental classifier by using the first class feature set.

[0123] The incremental classifier obtaining module 440 is used for training the incremental classifier by using the finally obtained first class feature set, and obtaining a trained incremental classifier.

[0124] Preferably, the base class defect anomaly classification model comprises an encoder, a first classifier and a mapping network which are sequentially spliced.

[0125] The encoder is used for extracting first feature vectors of base class defect pictures in the base class defect picture data set.

[0126] The first classifier is used for predicting the probability of each defect category according to each first feature vector.

[0127] The mapping network is used for extracting embedding features of the corresponding base class defect picture according to the first feature vector.

[0128] ​Preferably, when training the base defect anomaly classification model, the loss function comprises a first cross-entropy loss function and a contrastive loss function, and the contrastive loss function is obtained based on the positive sample embedding feature vectors and the negative sample embedding feature vectors obtained by the positive samples and the negative samples in the base defect picture dataset through the encoder and the mapping network.

[0129] Preferably, each incremental defect anomaly classification model comprises a trained encoder and a second classifier obtained after the training of the base defect anomaly classification model is completed.

[0130] The second training module 430 is configured to, when training each incremental defect anomaly classification model, extract features of the incremental defect pictures in the small sample incremental defect picture dataset through the trained encoder to obtain second feature vectors, and obtain the predicted classification results of the incremental defect pictures based on the second feature vectors.

[0131] Preferably, the second training module 430 comprises a second construction module 4301, a mapping module 4302, a second class parameter set obtaining module 4303, and an updating module 4304.

[0132] The second construction module 4301 is configured to construct a graph network model and an incremental classifier.

[0133] The mapping module 4302 is configured to map all class features in the first class feature set and all class parameters in the first class parameter set to nodes in the graph network model.

[0134] The second class parameter set obtaining module 4303 is configured to calculate third class parameters of the incremental classifier according to the node information of the graph network model to form a second class parameter set.

[0135] The updating module 4304 is configured to update the parameters of the incremental classifier by using a parameter matrix formed by splicing all the third class parameters in the second class parameter set.

[0136] Compared with the prior art, the present application has the following advantages:

[0137] 1. The present application adopts the idea of contrastive learning to train and optimize the encoder parameters, so that the encoder has stronger feature extraction capability.

[0138] 2. The present application uses a graph attention mechanism to construct a defect class relationship network, so that the model can use existing defect class knowledge to learn incremental defect classes with only a small number of pictures.

[0139] 3. The present application extracts defect class features from two aspects of class features of defect pictures and class parameters of classifiers, and performs information fusion, so as to obtain more accurate inter-class information, and perform parameter updating and incremental learning of the incremental classifier, thereby improving the classification accuracy.

[0140] While certain embodiments of the application have been described herein in detail, those skilled in the art will appreciate that modifications can be made of the above embodiments without departing from the scope and spirit of the application. The scope of the application is defined by the appended claims.

Claims

1. A defect incremental learning method based on small samples, characterized in that: include: Constructing an incremental learning dataset, wherein the incremental learning dataset includes a base defect image dataset and at least one small sample incremental defect image dataset; Using the base class defect image dataset to train a base class defect anomaly classification model, and using the trained base class defect anomaly classification model to extract first class features corresponding to all defect categories in the base class defect image dataset, and obtain first class parameters of the trained base class defect anomaly classification model, and put the first class features into a first class feature set, and put the first class parameters into a first class parameter set; Constructing incremental defect anomaly classification models with the same number as the small sample incremental defect image dataset, and training each incremental defect anomaly classification model, including: using the trained encoder obtained after training the base class defect anomaly classification model as the encoder of the incremental defect anomaly classification model, and using different classifiers for the base class defect anomaly classification model and the incremental defect anomaly classification model; Using the trained incremental defect anomaly classification model to extract the second-category features corresponding to all defect categories in the corresponding small sample incremental defect image dataset, and obtaining the second-category parameters of the trained incremental defect anomaly classification model, and putting all the second-category features into the first-category feature set, and putting all the second-category parameters into the first-category parameter set; Updating class parameters of an incremental classifier using the first set of parameters, and training the incremental classifier using the first set of features; After the training of all incremental defect anomaly classification models is completed, the trained incremental classifier is obtained.

2. The defect incremental learning method based on small samples according to claim 1 is characterized in that: The base class defect anomaly classification model includes an encoder, a first classifier and a mapping network that are sequentially spliced; The encoder is used to extract a first feature vector of a base class defect picture in the base class defect picture dataset; The first classifier is used to predict the probability of each defect category based on each first feature vector; The mapping network is used to extract the embedding features of the corresponding base class defect image based on the first feature vector.

3. The defect incremental learning method based on small samples according to claim 2 is characterized in that: When training the base class defect anomaly classification model, the loss function includes a first cross entropy loss function and a contrast loss function, and the contrast loss function is obtained based on the positive sample embedding feature vector and the negative sample embedding feature vector obtained by the encoder and the mapping network respectively based on the positive samples and negative samples in the base class defect image dataset.

4. The defect incremental learning method based on small samples according to claim 2, characterized in that: Each incremental defect anomaly classification model includes a trained encoder and a second classifier obtained after the base class defect anomaly classification model is trained; When training each incremental defect anomaly classification model, the trained encoder is used to extract features of the incremental defect images in the corresponding small sample incremental defect image data set to obtain a second feature vector, and the second classifier obtains the predicted classification result of the incremental defect image based on the second feature vector.

5. The defect incremental learning method based on small samples according to claim 1 is characterized in that: Updating the class parameters of the incremental classifier using the first set of parameters specifically includes: Mapping all class features in the first class feature set and all class parameters in the first class parameter set to nodes in a graph network model; Calculating the third type of parameters of the incremental classifier according to the node information of the graph network model to form a second type of parameter set, and using the second type of parameter set to update the first type of parameter set; The parameters of the incremental classifier are updated using a parameter matrix formed by concatenating all third-category parameters in the second-category parameter set.

6. A defect incremental learning device based on small samples, characterized in that: It includes a first construction module, a first training module, a second training module and an incremental classifier acquisition module; The first building module is used to build an incremental learning dataset, wherein the incremental learning dataset includes a base class defect image dataset and at least one small sample incremental defect image dataset; The first training module is used to train a base class defect anomaly classification model using the base class defect image dataset, and use the trained base class defect anomaly classification model to extract first class features corresponding to all defect categories in the base class defect image dataset, and obtain first class parameters of the trained base class defect anomaly classification model, and put the first class features into a first class feature set, and put the first class parameters into a first class parameter set; The second training module is used to construct incremental defect anomaly classification models with the same number as the small-sample incremental defect image dataset, and train each incremental defect anomaly classification model, including: using the trained encoder obtained after the base-class defect anomaly classification model is trained as the encoder of the incremental defect anomaly classification model, and the base-class defect anomaly classification model and the incremental defect anomaly classification model use different classifiers; and using the trained incremental defect anomaly classification model to extract the second-class features corresponding to all defect categories in the corresponding small-sample incremental defect image dataset, and obtain the second-class parameters of the trained incremental defect anomaly classification model, and put all the second-class features into the first-class feature set, and put all the second-class parameters into the first-class parameter set, and then use the first-class parameter set to update the class parameters of the incremental classifier, and use the first-class feature set to train the incremental classifier; The incremental classifier acquisition module is used to obtain the trained incremental classifier after the training of all incremental defect anomaly classification models is completed.

7. The device for incremental defect learning based on small samples according to claim 6, characterized in that: The base class defect anomaly classification model includes an encoder, a first classifier and a mapping network that are sequentially spliced; The encoder is used to extract a first feature vector of a base class defect picture in the base class defect picture dataset; The first classifier is used to predict the probability of each defect category based on each first feature vector; The mapping network is used to extract the embedding features of the corresponding base class defect image based on the first feature vector.

8. The device for incremental defect learning based on small samples according to claim 7, characterized in that: When training the base class defect anomaly classification model, the loss function includes a first cross entropy loss function and a contrast loss function, and the contrast loss function is obtained based on the positive sample embedding feature vector and the negative sample embedding feature vector obtained by the encoder and the mapping network respectively based on the positive samples and negative samples in the base class defect image dataset.

9. The device for incremental defect learning based on small samples according to claim 7, characterized in that: Each incremental defect anomaly classification model includes a trained encoder and a second classifier obtained after the base class defect anomaly classification model is trained; The second training module is used to train each incremental defect anomaly classification model, and uses the trained encoder to extract features of the incremental defect images in the corresponding small sample incremental defect image data set to obtain a second feature vector. The second classifier obtains the predicted classification result of the incremental defect image based on the second feature vector.

10. The device for incremental defect learning based on small samples according to claim 6, characterized in that: The second training module includes a second construction module, a mapping module, a second type parameter set acquisition module and an update module: The second building module is used to build a graph network model and an incremental classifier; The mapping module is used to map all class features in the first class feature set and all class parameters in the first class parameter set to nodes in the graph network model; The second parameter set obtaining module is used to calculate the third parameters of the incremental classifier according to the node information of the graph network model to form a second parameter set, and use the second parameter set to update the first parameter set; The updating module is used to update the parameters of the incremental classifier using a parameter matrix formed by splicing all third-category parameters in the second-category parameter set.

Images