Incremental training method and device based on multi-classifier, equipment and medium
By using a multi-classifier incremental training method, which utilizes pooling knowledge distillation and multiple sub-classifiers, the problem of low model plasticity in single-classifier image classification is solved, achieving better adaptation to new tasks and retention of old tasks.
Patent Information
- Application Number
- CN202210599548.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-30
- Publication Date
- 2026-02-24
- Estimated Expiration
- 2042-05-30
AI Technical Summary
In traditional image classification tasks, when a single classifier is used, the model has very low plasticity and cannot effectively learn new categories, leading to catastrophic forgetting problems.
An incremental training method based on multi-classifiers is adopted. The first loss function is constructed by pooling knowledge distillation to train the feature extractor and main classifier of the convolutional neural network, and the second loss function is constructed to train the sub-classifier. Multiple sub-classifiers are introduced to enhance the robustness of the model.
This improves the model's adaptability to new tasks, reduces forgetting of old tasks, and enhances the model's plasticity and robustness.
Smart Images
Figure CN114925828B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image classification technology, and for example to incremental training methods, apparatus, devices and media based on multi-classifiers. Background Technology
[0002] With the rise of deep neural networks, incremental learning has been widely applied in fields such as image classification, object detection, and semantic segmentation. In image classification, incremental learning is mainly used to address the catastrophic forgetting problem in deep neural networks. This means that while a deep learning model learns new knowledge for a new task and is trained, its performance on older tasks significantly declines as old knowledge is forgotten or overwritten. Currently, the main methods of incremental learning include playback-based incremental learning, constraint-based incremental learning, and parameter isolation-based incremental learning. These three methods can alleviate the catastrophic forgetting problem to some extent. However, traditional image classification tasks use a single classifier. When constraint-based knowledge distillation methods distill the original features, the loss function is too strict, resulting in very low model flexibility and an inability to learn new classes. Summary of the Invention
[0003] This application provides an incremental training method, apparatus, device, and medium based on a multi-classifier, aiming to solve the problem that the model has very low plasticity and cannot learn new classes when using a single classifier for image classification.
[0004] To solve the above problems, this application adopts the following technical solution:
[0005] This paper presents an incremental training method based on multi-classifiers, including:
[0006] A first loss function is constructed based on pooling knowledge distillation. This first loss function is then used to train the feature extractor and main classifier of the convolutional neural network, updating the parameters of the feature extractor and the main classifier. The formula for the first loss function is as follows:
[0007] L stage1 =L CE (F,M t )+λL POOL ;
[0008] Among them, L CE The main classifier's cross-entropy loss function, F is the feature extractor, and M is the cross-entropy loss function. t Let λ be the master classifier for the t-th task, and L be the balancing parameter. POOL The pooling knowledge distillation loss function;
[0009] A second loss function is constructed based on the cross-entropy loss function of the sub-classifier and the labeled data, as well as the classifier difference loss function between the sub-classifiers; the sub-classifier is then trained using the second loss function to update the sub-classifier parameters.
[0010] The feature extractor, the main classifier, and the sub-classifier are used to train the convolutional neural network.
[0011] The construction of the first loss function based on pooling knowledge distillation includes:
[0012] The pooling distillation loss of the main classifier is calculated based on spatial dimension and planar constraints.
[0013] The pooling distillation loss of the sub-classifier is calculated based on the spatial dimension and planar constraints.
[0014] The pooling knowledge distillation loss function is calculated based on the pooling distillation loss of the main classifier and the pooling distillation loss of the sub-classifier;
[0015] The formula for calculating the pooling knowledge distillation loss function is as follows:
[0016] L POOL =L final-M +L final-S ;
[0017] Among them, L final-M Pooling knowledge distillation loss for the primary classifier, L final-S Pooling knowledge distillation loss for the sub-classifier;
[0018] The first loss function is constructed based on the cross-entropy loss function and the pooling knowledge distillation loss function.
[0019] The calculation of the pooling distillation loss of the main classifier based on spatial dimension and planar constraints includes:
[0020] Calculate the loss of the main classifier in the width direction and the loss of the main classifier in the height direction;
[0021] Calculate the spatial dimension main classifier loss based on the width direction main classifier loss and the height direction main classifier loss;
[0022] Calculate the loss of the plane-constrained master classifier;
[0023] The pooling distillation loss of the master classifier is calculated based on the spatial dimension master classifier loss and the planar constraint master classifier loss.
[0024] The calculation of the pooling distillation loss of the sub-classifier based on spatial dimension and planar constraints includes:
[0025] Calculate the subclassifier loss in the width direction and the subclassifier loss in the height direction;
[0026] Calculate the spatial dimension subclassifier loss based on the width direction subclassifier loss and the height direction subclassifier loss;
[0027] Calculate the loss of the plane-constrained subclassifier;
[0028] The pooling distillation loss of the subclassifier is calculated based on the spatial dimension subclassifier loss and the planar constraint subclassifier loss.
[0029] The construction of the second loss function based on the cross-entropy between the sub-classifiers and the labeled data, and the classifier difference loss function between the sub-classifiers, includes:
[0030] Calculate the cross-entropy loss function of the sub-classifier, which is obtained by the cross-entropy between the softmax classification results of all the sub-classifiers and the labeled data;
[0031] Construct the classifier difference loss function based on the differences in classification results of all the subclassifiers;
[0032] The second loss function is constructed based on the cross-entropy loss function of the sub-classifier and the difference loss function of the classifier.
[0033] Further, training the feature extractor and main classifier of the convolutional neural network using the first loss function includes:
[0034] Obtain N training tasks in the training process, wherein the training process includes N / T training groups, and one training group includes T training tasks;
[0035] The t-th training task includes C t There are training samples, and the input image set corresponding to the t-th training task is X. t The tag category is Y t .
[0036] Furthermore, before training the sub-classifier using the second loss function, the method further includes:
[0037] An additional sample dataset is introduced as a supplementary training set for the sub-classifier, and the supplementary training set and the original training set are combined to form an expanded training set; wherein, the expanded training set is used to train the sub-classifier.
[0038] This application also provides an incremental training device based on a multi-classifier, including:
[0039] The module includes a first loss function construction module, a feature extractor and main classifier training module, a second loss function construction module, and a sub-classifier training module.
[0040] The first loss function construction module is used to construct the first loss function based on pooling knowledge distillation.
[0041] The feature extractor and main classifier training module is used to train the feature extractor and main classifier of the convolutional neural network using the first loss function, and to update the parameters of the feature extractor and the main classifier; the formula of the first loss function is as follows:
[0042] L stage1 =L Ce (F,M t )+λL POOL ;
[0043] Among them, L CE To calculate the cross-entropy, F is the feature extractor, and M... t Let L be the main classifier for the t-th task, λ be the tuning parameter, and L be the modulator. POOL This is a pooling operation;
[0044] The second loss function construction module is used to construct a second loss function based on the cross-entropy between the sub-classifiers and the labeled data, as well as the classifier difference loss function between the sub-classifiers;
[0045] The sub-classifier training module is used to stop the feature extractor from operating, train the sub-classifier using the second loss function, and update the sub-classifier parameters.
[0046] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the incremental training method based on a multi-classifier as described above.
[0047] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the incremental training method based on a multi-classifier as described above.
[0048] This application presents a multi-classifier-based incremental training method. A first loss function is constructed based on pooling knowledge distillation. This first loss function is used to train the feature extractor and main classifier of a convolutional neural network, updating their parameters. When calculating the difference between the old and new models, pooling is added to traditional knowledge distillation, making the first loss function more flexible and reducing constraints on the parameters. The feature extractor and main classifier obtained using the first loss function are used to train the model, resulting in a model that better adapts to new tasks. A second loss function is constructed based on the cross-entropy between the sub-classifiers and labeled data, as well as the classifier difference loss function among the sub-classifiers. The sub-classifiers are then trained using this second loss function, updating their parameters. Compared to a single-classifier incremental training model, introducing multiple sub-classifiers to calculate the second loss function makes the incremental training model more robust. Attached Figure Description
[0049] Figure 1 This is a flowchart illustrating an incremental training method based on a multi-classifier, as shown in one embodiment.
[0050] Figure 2 This is a schematic diagram illustrating the process of constructing a first loss function based on pooling knowledge distillation in one embodiment.
[0051] Figure 3 This is a schematic diagram illustrating the process of constructing a second loss function according to one embodiment;
[0052] Figure 4 This is a schematic block diagram of an incremental training device based on a multi-classifier according to an embodiment.
[0053] Figure 5 This is a schematic block diagram of the structure of a computer device according to one embodiment.
[0054] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0055] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0056] Those skilled in the art will understand that, unless explicitly stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in the specification of this application means the presence of features, integers, steps, operations, elements, units, cells, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, units, cells, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless couplings. The term “and / or” as used herein includes all or any of the units and all combinations thereof of one or more associated listed items.
[0057] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.
[0058] Reference Figure 1 This is a flowchart illustrating the incremental training method based on a multi-classifier proposed in this application, including:
[0059] S1: Construct the first loss function based on pooling knowledge distillation.
[0060] The first loss function consists of the cross-entropy loss function and the pooling knowledge distillation loss function;
[0061] The cross-entropy loss function reflects the difference in distribution between two quantities; the smaller the cross-entropy loss function, the closer the two quantities are. A model trained using the pooling knowledge distillation function can learn new tasks while also possessing some ability to handle older tasks.
[0062] The formula for calculating the pooling knowledge distillation loss function is as follows:
[0063] L POOL =L final-M +L final-S ;
[0064] Among them, L final-M Pooling knowledge distillation loss for the primary classifier, L final-S Pooling knowledge distillation loss for the sub-classifier;
[0065] The pooling distillation loss of the main classifier is calculated based on spatial dimension and planar constraints.
[0066] Calculating the pooling distillation loss of the main classifier based on spatial dimensions requires pooling operations across different spatial dimensions. In image processing, pooling can aggregate different locations within an image, resulting in a lower-dimensional image. Incorporating the pooling parameters into the loss function calculation allows for greater flexibility in the model derived from the loss function.
[0067] Constraint-based incremental learning methods reduce the model's forgetting of old tasks by adding constraints. Calculating the pooling distillation loss of the main classifier based on planar constraints can reduce the main classifier's forgetting of old tasks.
[0068] The pooling distillation loss of the sub-classifier is calculated based on the spatial dimension and planar constraints.
[0069] Calculating the pooling distillation loss of the subclassifier based on planar constraints can reduce the subclassifier's forgetting of old tasks.
[0070] The pooling knowledge distillation loss function is calculated based on the pooling distillation loss of the main classifier and the pooling distillation loss of the subclassifier.
[0071] S2: Use the first loss function to train the feature extractor and main classifier of the convolutional neural network, and update the parameters of the feature extractor and the main classifier.
[0072] Obtain N training tasks in the training process, wherein the training process includes N / T training groups, and one training group includes T training tasks;
[0073] The t-th training task includes C t There are training samples, and the input image set corresponding to the t-th training task is X. t The tag category is Y t .
[0074] The training sample set includes N t N training samples, N t Each training sample corresponds to N training tasks. By constructing a training sample set, the feature extractor, main classifier, and sub-classifier can be trained using the training sample set.
[0075] The feature extractor and main classifier of the convolutional neural network are trained on the training sample set using the first loss function. The parameters of the feature extractor and the main classifier are updated by backpropagation based on the first loss function.
[0076] Feature extractors are used to extract features from input images, while main classifiers are used to classify the input images. Commonly used feature extractors include recurrent neural networks, convolutional neural networks, and long short-term memory neural networks. Commonly used classifiers include SVM classifiers and softmax classifiers.
[0077] S3: Construct a second loss function based on the cross-entropy loss function of the sub-classifier and the labeled data, as well as the classifier difference loss function between the sub-classifiers.
[0078] Calculate the cross-entropy loss function of the sub-classifier, which is obtained by the cross-entropy between the softmax classification results of all the sub-classifiers and the labeled data;
[0079] Construct the classifier difference loss function based on the differences in classification results of all the subclassifiers;
[0080] The second loss function is constructed based on the cross-entropy and the classifier difference loss function.
[0081] The subclassifier cross-entropy loss function is used to measure the difference between the subclassifier and the labeled data, while the classifier difference loss function is used to measure the difference between different subclassifiers.
[0082] S4: Train the sub-classifier using the second loss function and update the sub-classifier parameters.
[0083] This causes the feature extractor to stop operating.
[0084] The total number of subclassifiers is K. The K subclassifiers are trained simultaneously according to the second loss function. The parameters of the K subclassifiers are updated by backpropagation according to the second loss function.
[0085] Stopping the feature extractor before training the sub-classifier using the second loss function can prevent the features extracted by the feature extractor from affecting the training process of the sub-classifier.
[0086] Using the second loss function to train the subclassifiers can take into account both the overall classification performance of multiple subclassifiers and the differences between subclassifiers.
[0087] The feature extractor, the main classifier, and the sub-classifier are used to train the convolutional neural network.
[0088] The incremental training method based on multi-classifiers in this application constructs a first loss function based on pooling knowledge distillation. This first loss function is used to train the feature extractor and main classifier of the convolutional neural network, updating the parameters of both. When calculating the difference between the old and new models, pooling is added to traditional knowledge distillation, making the first loss function more flexible and reducing constraints on the parameters. The feature extractor and main classifier obtained using the first loss function are used to train the model, resulting in a model that better adapts to new tasks. A second loss function is constructed based on the cross-entropy between the sub-classifiers and labeled data, as well as the classifier difference loss function between sub-classifiers. The sub-classifiers are then trained using this second loss function, updating their parameters. Compared to the incremental training model with a single classifier, introducing multiple sub-classifiers to calculate the second loss function makes the incremental training model more robust.
[0089] In one embodiment, a first loss function is constructed based on pooling knowledge distillation, referring to... Figure 2 , Figure 2 This is a schematic diagram of a process for constructing a first loss function based on pooling knowledge distillation, as shown in one embodiment, including:
[0090] S11: Calculate the pooling distillation loss of the master classifier based on the spatial dimension and planar constraints.
[0091] Spatial dimensions include width and height. By performing pooling operations on the width and height dimensions and aggregating them, the loss function that does not use pooling operations for knowledge distillation is calculated instead, making the model more flexible.
[0092] The formula for pooling the width dimension is as follows:
[0093]
[0094] Where t represents the t-th task, l represents the l-th convolutional neural network layer, c represents the number of channels, C is the total number of channels, h represents the width, H is the total width, w represents the height, and W is the total height. L represents the output of the l-th layer of the convolutional neural network for the t-th task. width This indicates pooling along the width dimension.
[0095] The formula for pooling the height dimension is as follows:
[0096]
[0097] L height This indicates pooling based on the height dimension.
[0098] The formula for pooling spatial dimensions is as follows:
[0099]
[0100] Pooling the spatial dimension is the sum of pooling the width dimension and pooling the height dimension.
[0101] Based on the spatial dimension-based pooling knowledge distillation described above, a planar constraint is set on the output of the last convolutional neural network layer, and the calculation formula is as follows:
[0102]
[0103] in, This represents the output of the last layer of the convolutional neural network for the t-th task. This is the output of the last layer of the convolutional neural network for the (t-1)th task.
[0104] Finally, the pooling distillation loss of the main classifier consists of two parts: spatial dimension and planar constraint, as shown in the following formula:
[0105]
[0106] Among them, L final-M Pooling distillation loss of the main classifier, λ c λ is a parameter used to measure spatial dimension. f λ is a parameter used to measure planar constraints. c With λ f The value is between 0 and 1.
[0107] S12: Calculate the pooling distillation loss of the sub-classifier based on the spatial dimension and planar constraints.
[0108] For k sub-classifiers, the output of the l-th layer convolutional neural network for the t-th task is the average of the outputs of the k sub-classifiers.
[0109] For width-dimensional pooling, height-dimensional pooling, spatial-dimensional pooling, and planar constraints, the calculation methods for the sub-classifier and the main classifier are consistent, and will not be elaborated here.
[0110] The formula for the pooling distillation loss of the sub-classifier is as follows:
[0111]
[0112] Among them, L final-S denoted as pooling distillation loss of the subclassifier, k is the subclassifier number, and K is the total number of subclassifiers.
[0113] S13: Calculate the pooling knowledge distillation loss function based on the pooling distillation loss of the main classifier and the pooling distillation loss of the sub-classifier.
[0114] The pooling knowledge distillation loss function consists of the pooling distillation loss of the main classifier and the pooling distillation loss of the subclassifier. The formula for the pooling knowledge distillation loss function is as follows:
[0115] L POOL =L final-M +L final-S ;
[0116] Among them, L POOL For the pooling knowledge distillation loss function, L final-M Pooling knowledge distillation loss for the primary classifier, L final-S The pooling knowledge distillation loss is used for the sub-classifier.
[0117] S14: Construct the first loss function based on the cross-entropy loss function and the pooling knowledge distillation loss function.
[0118] The first loss function consists of the cross-entropy loss function and the pooling knowledge distillation loss function.
[0119] The cross-entropy loss function is the cross-entropy between the output class of the main classifier and the labeled class. The formula for the cross-entropy loss function is as follows:
[0120]
[0121] in, The output of the main classifier after processing the i-th training sample for the t-th task is classified into labeled categories. The probability, Let be the i-th training sample for the t-th task, and log represent the logarithmic function.
[0122] The formula for the first loss function is as follows:
[0123] L stage1 =L CE (F,M t )+λL POOL ;
[0124] Among them, L CE Let F be the cross-entropy loss function, F be the feature extractor, and M be the cross-entropy loss function. t Let λ be the master classifier for the t-th task, and L be the balancing parameter. POOL This is the pooling knowledge distillation loss function.
[0125] This embodiment constructs a first loss function based on pooling knowledge distillation. The pooling distillation loss of the main classifier is calculated based on spatial dimension and planar constraints, and the pooling distillation loss of the sub-classifier is also calculated based on spatial dimension and planar constraints. The pooling knowledge distillation loss function is then calculated based on the pooling distillation losses of the main classifier and the sub-classifier. The first loss function is constructed based on the cross-entropy loss function and the pooling knowledge distillation loss function. Compared to traditional knowledge distillation loss functions, the pooling knowledge distillation loss function incorporating pooling operations has better plasticity, allowing the trained model to adapt to new tasks.
[0126] In one embodiment, a second loss function is constructed based on the cross-entropy between the sub-classifiers and the labeled data, and the classifier difference loss function among the sub-classifiers. (Refer to...) Figure 3 , Figure 3 A flowchart illustrating the construction of a second loss function according to one embodiment includes:
[0127] S31: Calculate the cross-entropy loss function of the sub-classifier.
[0128] The cross-entropy loss function of the sub-classifier is calculated, which is obtained by the cross-entropy between the softmax classification results of all the sub-classifiers and the labeled data.
[0129] There are a total of k subclassifiers. The formula for the cross-entropy between the softmax classification result of the subclassifier and the labeled data is as follows:
[0130]
[0131] Among them, among them, The output of the subclassifier for the t-th task and the i-th training sample is classified into labeled categories. The probability, Let $\log \in \mathbb{T}$ be the $i$-th training sample for the $t$-th task, $\log \in \mathbb{T}$ denote the logarithmic function, and $N$ be the total number of training samples. t k is the index of the subclassifier, and K is the total number of subclassifiers.
[0132] The cross-entropy loss function of the subclassifiers represents the difference between the classification results obtained by the k subclassifiers and the categories of the labeled data. The smaller the difference, the closer the classification result obtained by the k subclassifiers as a whole is to the labeled data, and the better the classification effect of the subclassifiers.
[0133] S32: Construct the classifier difference loss function based on the differences in classification results of all the subclassifiers.
[0134] During the training of the K sub-classifiers, a playback-based incremental learning method was adopted, and an additional sample dataset SVHN was introduced as the dataset x for the sub-classifiers. out ∈X out The dataset size is N out The difference loss function between sub-classifiers is calculated based on L1 regularization of the outputs of each pair of sub-classifiers. The formula for the difference loss function between sub-classifiers is as follows:
[0135]
[0136] Where, p m (y|x out ) and p n (y|x out ) represent the softmax classification results of subclassifiers with serial numbers m and n, respectively, where d represents L1 regularization, m is the number of one subclassifier, and n is the number of the other subclassifier.
[0137] The formula for L1 regularization of the softmax classification results of subclassifiers with ordinal numbers m and n is as follows:
[0138]
[0139] Here, || represents the absolute value operation.
[0140] For k subclassifiers, there are a total of A pair of subclassifiers.
[0141] The classifier difference loss function represents the difference between different subclassifiers. The larger the difference, the smaller the coupling between the subclassifiers, and the higher the classification accuracy of multiple subclassifiers as a whole.
[0142] S33: Construct the second loss function based on the cross-entropy and the classifier difference loss function.
[0143] The second loss function consists of the subclassifier cross-entropy loss function and the classifier difference loss function. The formula for the second loss function is as follows:
[0144]
[0145] in, The cross-entropy loss function of the sub-classifier. Let be the classifier difference loss function.
[0146] The k sub-classifiers are trained based on the second loss function. The training objective is to minimize the cross-entropy loss function of the first part and maximize the difference loss function of the second part classifiers.
[0147] The second loss function provided in this application embodiment is constructed by calculating the cross-entropy loss function of the sub-classifiers, constructing the classifier difference loss function based on the differences in classification results of all the sub-classifiers, and constructing the second loss function based on the cross-entropy loss function of the sub-classifiers and the classifier difference loss function. The cross-entropy loss function of the sub-classifiers is used to measure the classification performance of multiple sub-classifiers as a whole, and the classifier difference loss function is used to measure the degree of coupling between different sub-classifiers. During the training of the sub-classifiers based on the second loss function, the cross-entropy loss function of the sub-classifiers becomes smaller and smaller, while the classifier difference loss function becomes larger and larger, thereby improving the classification performance of the sub-classifiers.
[0148] Reference Figure 4 This is a schematic block diagram of the incremental training device based on a multi-classifier proposed in this application. The device includes:
[0149] The module consists of a first loss function construction module 10, a feature extractor and main classifier training module 20, a second loss function construction module 30, and a sub-classifier training module 40.
[0150] The first loss function construction module 10 is used to construct the first loss function based on pooling knowledge distillation;
[0151] The feature extractor and main classifier training module 20 is used to train the feature extractor and main classifier of the convolutional neural network using the first loss function, and to update the feature extractor parameters and main classifier parameters; the formula of the first loss function is as follows:
[0152] L stage1 =L Ce (F,M t )+λL POOL ;
[0153] Among them, L CE To calculate the cross-entropy, F is the feature extractor, and M... t Let L be the main classifier for the t-th task, λ be the tuning parameter, and L be the modulator. POOL This is a pooling operation;
[0154] The second loss function construction module 30 is used to construct a second loss function based on the cross-entropy between the sub-classifier and the labeled data, as well as the classifier difference loss function between the sub-classifiers;
[0155] The sub-classifier training module 40 is used to stop the feature extractor from operating, train the sub-classifier using the second loss function, and update the sub-classifier parameters.
[0156] The incremental training device based on a multi-classifier according to embodiments of this application includes a first loss function construction module, a feature extractor and a main classifier training module, a second loss function construction module, and a sub-classifier training module. This device enables incremental training based on a multi-classifier.
[0157] Reference Figure 5 This application also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 5 As shown, this computer device includes a processor, memory, network interface, and database connected via a system bus. The processor is designed to provide computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores parameters of the main classifier and sub-classifiers, etc. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an incremental training method based on a multi-classifier.
[0158] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment on which the present application is applied.
[0159] One embodiment of this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an incremental training method based on a multi-classifier. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.
[0160] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media provided in this application and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0161] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.
[0162] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. An incremental training method based on a multi-classifier, characterized in that, include: A first loss function is constructed based on pooling knowledge distillation. This first loss function is then used to train the feature extractor and main classifier of the convolutional neural network, updating the parameters of the feature extractor and the main classifier. The formula for the first loss function is as follows: ； Among them, L CE The main classifier's cross-entropy loss function, F is the feature extractor, and M is the cross-entropy loss function. t Let λ be the master classifier for the t-th task, and L be the balancing parameter. POOL The pooling knowledge distillation loss function is used; the main classifier cross-entropy loss function is the cross-entropy between the output classification and the labeled category of the main classifier. A second loss function is constructed based on the cross-entropy loss function of the sub-classifiers and the labeled data, as well as the classifier difference loss function among the sub-classifiers; the sub-classifiers are trained using the second loss function to update their parameters; the cross-entropy loss function of the sub-classifiers is obtained by the cross-entropy between the softmax classification results of all the sub-classifiers and the labeled data. The feature extractor, the main classifier, and the sub-classifier are used to train the convolutional neural network; The construction of the first loss function based on pooling knowledge distillation includes: The pooling distillation loss of the main classifier is calculated based on spatial dimension and planar constraints. The pooling distillation loss of the sub-classifier is calculated based on the spatial dimension and planar constraints. The pooling knowledge distillation loss function is calculated based on the pooling distillation loss of the main classifier and the pooling distillation loss of the sub-classifier; The formula for calculating the pooling knowledge distillation loss function is as follows: Among them, L final-M Pooling knowledge distillation loss for the primary classifier, L final-S Pooling knowledge distillation loss for the sub-classifier; The first loss function is constructed based on the cross-entropy loss function and the pooling knowledge distillation loss function; The spatial dimension includes a width dimension and a height dimension, and the spatial dimension is pooled as the sum of the pooling of the width dimension and the pooling of the height dimension; The formula for pooling the width dimension is as follows: ； Where t represents the t-th task, l represents the l-th convolutional neural network layer, c represents the number of channels, C is the total number of channels, h represents the width, H is the total width, w represents the height, and W is the total height. This represents the output of the l-th layer of the convolutional neural network for the t-th task. This indicates pooling along the width dimension; The formula for pooling the height dimension is as follows: ； This indicates pooling based on the height dimension; The step of training the feature extractor and main classifier of the convolutional neural network using the first loss function includes: Obtain N training tasks in the training process, wherein the training process includes N / T training groups, and one training group includes T training tasks; The t-th training task includes C t There are training samples, and the input image set corresponding to the t-th training task is X. t The tag category is Y t .
2. The incremental training method based on a multi-classifier according to claim 1, characterized in that, The calculation of the pooling distillation loss of the main classifier based on spatial dimension and planar constraints includes: Calculate the loss of the main classifier in the width direction and the loss of the main classifier in the height direction; Calculate the spatial dimension main classifier loss based on the width direction main classifier loss and the height direction main classifier loss; Calculate the loss of the plane-constrained master classifier; The pooling distillation loss of the master classifier is calculated based on the spatial dimension master classifier loss and the planar constraint master classifier loss.
3. The incremental training method based on a multi-classifier according to claim 1, characterized in that, The calculation of the pooling distillation loss of the sub-classifier based on spatial dimension and planar constraints includes: Calculate the subclassifier loss in the width direction and the subclassifier loss in the height direction; Calculate the spatial dimension subclassifier loss based on the width direction subclassifier loss and the height direction subclassifier loss; Calculate the loss of the plane-constrained subclassifier; The pooling distillation loss of the subclassifier is calculated based on the spatial dimension subclassifier loss and the planar constraint subclassifier loss.
4. The incremental training method based on a multi-classifier according to claim 1, characterized in that, The construction of the second loss function based on the cross-entropy between the sub-classifiers and the labeled data, and the classifier difference loss function between the sub-classifiers, includes: Calculate the cross-entropy loss function of the sub-classifier, which is obtained by the cross-entropy between the softmax classification results of all the sub-classifiers and the labeled data; Construct the classifier difference loss function based on the differences in classification results of all the subclassifiers; The second loss function is constructed based on the cross-entropy loss function of the sub-classifier and the difference loss function of the classifier.
5. The incremental training method based on a multi-classifier according to claim 1, characterized in that, Before training the sub-classifier using the second loss function, the method further includes: An additional sample dataset is introduced as a supplementary training set for the sub-classifier, and the supplementary training set and the original training set are combined to form an expanded training set; wherein, the expanded training set is used to train the sub-classifier.
6. An incremental training device based on a multi-classifier, characterized in that, include: The module includes a first loss function construction module, a feature extractor and main classifier training module, a second loss function construction module, and a sub-classifier training module. The first loss function construction module is used to construct the first loss function based on pooling knowledge distillation. The feature extractor and main classifier training module is used to train the feature extractor and main classifier of the convolutional neural network using the first loss function, and to update the parameters of the feature extractor and the main classifier; the formula of the first loss function is as follows: ； Among them, L CE The main classifier's cross-entropy loss function, F is the feature extractor, and M is the cross-entropy loss function. t Let L be the main classifier for the t-th task, λ be the tuning parameter, and L be the modulator. POOL This is a pooling operation; the main classifier cross-entropy loss function is the cross-entropy between the output classification of the main classifier and the labeled category. The second loss function construction module is used to construct a second loss function based on the sub-classifier cross-entropy loss function of the sub-classifier and the labeled data, and the classifier difference loss function between the sub-classifiers; the sub-classifier cross-entropy loss function is obtained by the cross-entropy of the softmax classification results of all the sub-classifiers and the labeled data; The sub-classifier training module is used to stop the feature extractor from operating, train the sub-classifier using the second loss function, and update the sub-classifier parameters. The construction of the first loss function based on pooling knowledge distillation includes: The pooling distillation loss of the main classifier is calculated based on spatial dimension and planar constraints. The pooling distillation loss of the sub-classifier is calculated based on the spatial dimension and planar constraints. The pooling knowledge distillation loss function is calculated based on the pooling distillation loss of the main classifier and the pooling distillation loss of the sub-classifier; The formula for calculating the pooling knowledge distillation loss function is as follows: Among them, L final-M Pooling knowledge distillation loss for the primary classifier, L final-S Pooling knowledge distillation loss for the sub-classifier; The first loss function is constructed based on the cross-entropy loss function and the pooling knowledge distillation loss function; The spatial dimension includes a width dimension and a height dimension, and the spatial dimension is pooled as the sum of the pooling of the width dimension and the pooling of the height dimension; The formula for pooling the width dimension is as follows: ； Where t represents the t-th task, l represents the l-th convolutional neural network layer, c represents the number of channels, C is the total number of channels, h represents the width, H is the total width, w represents the height, and W is the total height. This represents the output of the l-th layer of the convolutional neural network for the t-th task. This indicates pooling along the width dimension; The formula for pooling the height dimension is as follows: ； This indicates pooling based on the height dimension; The step of training the feature extractor and main classifier of the convolutional neural network using the first loss function includes: Obtain N training tasks in the training process, wherein the training process includes N / T training groups, and one training group includes T training tasks; The t-th training task includes C t There are training samples, and the input image set corresponding to the t-th training task is X. t The tag category is Y t .
7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the incremental training method based on a multi-classifier as described in any one of claims 1 to 5.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the incremental training method based on a multi-classifier as described in any one of claims 1 to 5.