Model training method and device

Abstract

The invention discloses a model training method and device, and relates to the technical field of deep learning. A specific embodiment of the model training method comprises the following steps: dividing a preset basic model into a shared sub-network and a basic sub-network; copying the basic sub-network to obtain an incremental sub-network, training the incremental sub-network by using preset incremental data, and taking the shared sub-network and the trained incremental sub-network as an incremental model; and fusing the basic model and the incremental model to obtain an improved model corresponding to the basic model. According to the embodiment, the model training efficiency can be improved, and system resources and time cost consumed by model training can be reduced.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a method and device for model training. Background Art

[0002] The trained deep learning model can be used to extract deep learning features from text, images, videos and other data. For example, in order to identify the product images uploaded by users and determine whether the product images are consistent with the category information marked by the users, e-commerce platforms usually use image classification models to identify the category information to which the product images belong. However, the category information of products is frequently updated. Therefore, every once in a while, it is necessary to use the new category information and the old category information to retrain the image classification model so that the retrained image classification model can recognize both the new category information and the old category information.

[0003] In the process of implementing the present invention, the inventors found that the prior art has at least the following problems:

[0004] As the amount of category information that needs to be identified increases, each retraining consumes a lot of system resources and time costs, and the model training efficiency is low. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a method and device for model training, which can improve the efficiency of model training and reduce the system resources and time cost consumed by model training.

[0006] To achieve the above object, according to a first aspect of an embodiment of the present invention, a model training method is provided, comprising:

[0007] Divide the pre-set basic model into a shared sub-network and a basic sub-network;

[0008] The basic sub-network is copied to obtain an incremental sub-network, the incremental sub-network is trained using preset incremental data, and the shared sub-network and the trained incremental sub-network are used as incremental models;

[0009] The basic model and the incremental model are fused to obtain an improved model corresponding to the basic model.

[0010] Optionally, training the incremental sub-network using preset incremental data includes:

[0011] Mixing the incremental data with the basic data corresponding to the basic model to obtain mixed data;

[0012] The incremental sub-network is progressively trained using the mixed data.

[0013] Optionally, the incremental data includes incremental data features and incremental data annotations, and the basic data includes basic data features and basic data annotations; and mixing the incremental data with the basic data corresponding to the basic model to obtain mixed data includes:

[0014] Determine the incremental data annotations and basic data annotations having overlapping relationships and the corresponding incremental data features and basic data features;

[0015] According to a preset mixing ratio, the incremental data annotations and basic data annotations having overlapping relationships are mixed into mixed data annotations;

[0016] According to the mixing ratio, the corresponding incremental data features and basic data features are mixed into mixed data features;

[0017] The mixed data is obtained according to the mixed data features and the mixed data annotations.

[0018] Optionally, the incremental data has multiple groups; duplicating the basic subnetwork to obtain an incremental subnetwork, and training the incremental subnetwork using the preset incremental data, including: duplicating the basic subnetwork to obtain multiple incremental subnetworks, and training different incremental subnetworks using incremental data of different groups;

[0019] The basic model and the incremental model are fused to obtain an improved model corresponding to the basic model, including: combining the shared subnetwork with each trained incremental subnetwork respectively to obtain multiple incremental models, and fusing the basic model with the multiple incremental models to obtain an improved model corresponding to the basic model.

[0020] Optionally, when the incremental subnetwork does not include a linear classifier, the method further comprises: copying the linear classifier of the base model to obtain an incremental linear classifier, and adding the incremental linear classifier to the incremental subnetwork;

[0021] The incremental sub-network is trained using the preset incremental data, including: the incremental sub-network to which the incremental linear classifier is added is trained using the preset incremental data.

[0022] Optionally, the basic sub-network is the last several layers of the network structure corresponding to the basic model.

[0023] According to a second aspect of an embodiment of the present invention, there is provided a method for image classification, comprising:

[0024] In response to receiving an image to be predicted, inputting the image to be predicted into a preset image classification model so that the image classification model extracts deep learning features of the image to be predicted, wherein the image classification model is obtained by using any method described in the first aspect of the embodiments of the present invention;

[0025] Generate category information of the image to be predicted based on the deep learning features of the image to be predicted.

[0026] According to a third aspect of an embodiment of the present invention, a model training device is provided, including:

[0027] A partitioning module, used for partitioning a preset basic model into a shared sub-network and a basic sub-network;

[0028] A training module, used to copy the basic sub-network to obtain an incremental sub-network, train the incremental sub-network using preset incremental data, and use the shared sub-network and the trained incremental sub-network as incremental models;

[0029] A fusion module is used to fuse the basic model and the incremental model to obtain an improved model corresponding to the basic model.

[0030] Optionally, training the incremental sub-network using preset incremental data includes:

[0031] Mixing the incremental data with the basic data corresponding to the basic model to obtain mixed data;

[0032] The incremental sub-network is progressively trained using the mixed data.

[0033] Optionally, the incremental data includes incremental data features and incremental data annotations, and the basic data includes basic data features and basic data annotations; and mixing the incremental data with the basic data corresponding to the basic model to obtain mixed data includes:

[0034] Determine the incremental data annotations and basic data annotations having overlapping relationships and the corresponding incremental data features and basic data features;

[0035] According to a preset mixing ratio, the incremental data annotations and basic data annotations having overlapping relationships are mixed into mixed data annotations;

[0036] According to the mixing ratio, the corresponding incremental data features and basic data features are mixed into mixed data features;

[0037] The mixed data is obtained according to the mixed data features and the mixed data annotations.

[0038] Optionally, the incremental data has multiple groups; duplicating the basic subnetwork to obtain an incremental subnetwork, and training the incremental subnetwork using the preset incremental data, including: duplicating the basic subnetwork to obtain multiple incremental subnetworks, and training different incremental subnetworks using incremental data of different groups;

[0039] The basic model and the incremental model are fused to obtain an improved model corresponding to the basic model, including: combining the shared subnetwork with each trained incremental subnetwork respectively to obtain multiple incremental models, and fusing the basic model with the multiple incremental models to obtain an improved model corresponding to the basic model.

[0040] Optionally, the device further comprises: an adding module, configured to copy the linear classifier of the basic model to obtain an incremental linear classifier, and add the incremental linear classifier to the incremental sub-network;

[0041] The incremental sub-network is trained using the preset incremental data, including: the incremental sub-network to which the incremental linear classifier is added is trained using the preset incremental data.

[0042] Optionally, the basic sub-network is the last several layers of the network structure corresponding to the basic model.

[0043] According to a fourth aspect of an embodiment of the present invention, there is provided an apparatus for image classification, comprising:

[0044] an extraction module, configured to, in response to receiving an image to be predicted, input the image to be predicted into a preset image classification model, so that the image classification model extracts deep learning features of the image to be predicted, wherein the image classification model is obtained by using any method described in the first aspect of the embodiments of the present invention;

[0045] A generation module is used to generate category information of the image to be predicted based on the deep learning features of the image to be predicted.

[0046] According to a fifth aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0047] one or more processors;

[0048] a storage device for storing one or more programs,

[0049] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any one of the above embodiments.

[0050] According to a sixth aspect of an embodiment of the present invention, there is provided a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the method described in any one of the above embodiments is implemented.

[0051] An embodiment of the above invention has the following advantages or beneficial effects: performing quasi-incremental learning on the basic model to obtain an incremental model can improve the model training efficiency, save system resources, and enable the incremental model to better learn the deep learning features of the incremental data; fusing the basic model and the incremental model to obtain the corresponding improved model, which can enable the improved model to meet the feature extraction requirements of different data and improve the accuracy of feature extraction; mixing the incremental data with the basic data, and using the mixed data to train the incremental sub-network, which is conducive to the rapid convergence of the learning process of the incremental model and further improves the training efficiency of the incremental model; obtaining mixed data according to the overlapping relationship and the mixing ratio can flexibly adjust the intensity of the basic data and the incremental data participating in the model training, which is conducive to improving the accuracy of the incremental model feature extraction; using different groups of incremental data to train different incremental models, and then fusing the basic model and multiple incremental models to obtain an improved model, which can make the improved model applicable to the feature extraction process of various data, improve the universal applicability and training efficiency of the improved model, avoid using too many resources, and reduce time cost.

[0052] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0054] Figure 1 is a schematic diagram of the main process of the model training method according to an embodiment of the present invention;

[0055] Figure 2 is a schematic diagram of network division according to a method for model training that can be used as a reference embodiment of the present invention;

[0056] Figure 3 is a schematic diagram of incremental training of a model training method according to a reference embodiment of the present invention;

[0057] Figure 4 is a schematic diagram of incremental training of a model training method according to another reference embodiment of the present invention;

[0058] Figure 5 is a schematic diagram of the main process of a model training method according to a reference embodiment of the present invention;

[0059] Figure 6is a schematic diagram of the main process of the image classification method according to an embodiment of the present invention;

[0060] Figure 7 is a schematic diagram of the overall process of an image classification method according to a reference embodiment of the present invention;

[0061] Figure 8 is a schematic diagram of the main process of an image classification method according to a reference embodiment of the present invention;

[0062] Fig. 9 is a schematic diagram of main modules of a device for model training according to an embodiment of the present invention;

[0063] Fig.10 is a schematic diagram of main modules of an apparatus for image classification according to an embodiment of the present invention;

[0064] Fig.11 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0065] Fig.12 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0067] It should be noted that in the technical solution of the present invention, the collection, use, storage, sharing and transfer of user personal information involved are in compliance with the provisions of relevant laws and regulations, and it is necessary to inform the user and obtain the user's consent or authorization. When applicable, the user's personal information is de-identified and / or anonymized and / or encrypted.

[0068] The trained deep learning model can be used to extract deep learning features from text, images, videos and other data. For example, in order to identify the product images uploaded by users and determine whether the product images are consistent with the category information marked by the users, e-commerce platforms usually use image classification models to identify the category information to which the product images belong. However, the category information of the products is frequently updated, which makes the image classification model unable to recognize the new category information. Therefore, every once in a while, it is necessary to use the new category information and the old category information to retrain the image classification model so that the retrained image classification model can recognize both the new category information and the old category information.

[0069] As more and more category information needs to be identified, each retraining consumes a lot of system resources and time costs, the model training efficiency is low, and the category information identified by the trained model is inaccurate.

[0070] In view of this, according to a first aspect of an embodiment of the present invention, a method for model training is provided.

[0071] Figure 1 Schematic diagram of the main process of the model training method according to an embodiment of the present invention. Figure 1 As shown, the model training method according to an embodiment of the present invention mainly includes the following steps S101 to S103.

[0072] Step S101: divide the preset basic model into a shared sub-network and a basic sub-network.

[0073] The pre-set basic model is a model used to extract the deep learning features of the data and make predictions based on the deep learning features of the data. Specifically, for the image to be predicted input into the basic model, the feature extractor of the basic model is used to extract the deep learning features of the image to be predicted, and the linear classifier of the basic model is used to perform matrix operations on the deep learning features, and then the operation results are subjected to nonlinear transformation to obtain the probability distribution of the category to which the image to be predicted belongs, thereby determining the category information of the image to be predicted. Among them, the category information has a variety of classification and induction standards. For example, according to the type of goods, it is divided into: food, clothing, daily necessities and other category information; according to the age of the commodity audience, it is divided into: elderly products, middle-aged products, young people's products, children's and infant products and other category information; from the perspective of business management, it is divided into: self-operated products, domestic third-party operated products, international third-party operated products and other category information. However, in the face of the increasing number of new category information and various new styles and new layouts of images to be predicted, it is necessary to improve the basic model so that the improved model can recognize new category information.

[0074] When improving the basic model, the basic model is divided into a shared subnetwork and a basic subnetwork. The basic subnetwork is a part of the network structure corresponding to the basic model, and the shared subnetwork is the remaining part of the network structure corresponding to the basic model except the basic subnetwork; for example, the network structure including the pooling layer and the fully connected layer in the basic model is used as the basic subnetwork, or the network structure including the pooling layer, the fully connected layer and a part of the convolution layer in the basic model is used as the basic subnetwork, or the network structure including the pooling layer and a part of the convolution layer in the basic model is used as the basic subnetwork, wherein the convolution layer can be a 1×1 convolution layer or a 3×3 convolution layer, etc. The basic subnetwork is a continuous network structure in the basic model; for example, the basic model includes a 6-layer network structure, wherein the 4th layer is a 1×1 convolution layer and the 5th layer is a maximum pooling layer, the 4th and 5th layer network structures of the above basic model are used as the basic subnetwork, and the remaining network structures are used as shared subnetworks; for another example, the basic model includes an 8-layer network structure, wherein the 5th layer is a 3×3 convolution layer, the 6th layer is a 1×1 convolution layer, and the 7th layer is an average pooling layer, the 5th, 6th and 7th layers of the above basic model are used as the basic subnetwork, and the remaining network structures are used as shared subnetworks.

[0075] Dividing the basic model into a shared subnetwork and a basic subnetwork is beneficial for training only the basic subnetwork later, improving the training efficiency of the incremental model. Dividing the basic model according to the network structure can improve the flexibility of the basic model division.

[0076] According to a reference embodiment of the present invention, the basic sub-network is the last several layers of the network structure corresponding to the basic model. For example, the last 1 layer, the last 2 layers or the last 3 layers of the network structure corresponding to the basic model are used as the basic sub-network, and the network structure before the basic sub-network in the basic model is used as a shared sub-network. Figure 2 is a schematic diagram of network partitioning of a model training method according to a reference embodiment of the present invention. For example, Figure 2 As shown, in a reference embodiment of the present invention, the basic model is an EfficientNetV2-S classification model, wherein the EfficientNetV2-S classification model has an 8-layer network structure, specifically including Stage 0 to Stage 7. The last layer (i.e., Stage 7) of the EfficientNetV2-S classification model is used as the basic sub-network 202, and the first 7 layers (i.e., Stage 0 to Stage 6) of the EfficientNetV2-S classification model are used as the shared sub-network 201, wherein the basic sub-network includes 202: a 1×1 convolutional layer (Conv1×1), a pooling layer (Pooling) and a fully connected layer (FC), with a total of 1280 channels.

[0077] As another example, the EfficientNetV2-M classification model and the EfficientNetV2-L classification model have the same structure as the EfficientNetV2-S classification model, and only have more layers in some network structures (partial stages). Therefore, the division method of the EfficientNetV2-M classification model and the EfficientNetV2-L classification model is the same as that of the EfficientNetV2-S classification model; that is, when the basic model is the EfficientNetV2-M classification model, the last layer (i.e., Stage 7) of the EfficientNetV2-M classification model is used as the basic subnetwork, and the remaining network structure is used as the shared subnetwork; when the basic model is the EfficientNetV2-L classification model, the last layer (i.e., Stage 7) of the EfficientNetV2-L classification model is used as the basic subnetwork, and the remaining network structure is used as the shared subnetwork.

[0078] Using the last several layers of the network structure corresponding to the basic model as the basic sub-network can simplify the division process of the basic model and improve the division efficiency of the basic model. The last several layers of the basic model usually include pooling layers and fully connected layers, which is conducive to the basic sub-network to better learn new deep learning features.

[0079] Step S102, copying the basic sub-network to obtain an incremental sub-network, training the incremental sub-network using preset incremental data, and using the shared sub-network and the trained incremental sub-network as incremental models.

[0080] After dividing the basic subnetwork, the basic subnetwork is copied to obtain an incremental subnetwork, which has the same network structure as the basic subnetwork. The pre-set incremental data is obtained. The incremental data is data with deep learning features corresponding to the new category information, and the basic model cannot accurately identify the new category information based on the incremental data. Therefore, the incremental subnetwork is iteratively trained using the incremental data, and the network parameters of the incremental subnetwork are fine-tuned using the pre-set loss function. When iteratively training the incremental subnetwork, the shared subnetwork is frozen to prevent changes in the shared subnetwork. After the incremental subnetwork training is completed, the shared subnetwork and the trained incremental subnetwork are used as incremental models.

[0081] For example, the category information that the basic model can identify includes: category information A1, category information A2, and category information A3. Then, category information A4 is added. Since the basic model cannot accurately identify category information A4, Figure 2As shown, the last layer (i.e., Stage 7) of the EfficientNetV2-S classification model is used as the basic subnetwork 202, and the basic subnetwork 202 is copied to obtain the incremental subnetwork 203. The incremental data is obtained, and the incremental data all belong to the category information A4. The incremental subnetwork 203 is trained using the incremental data; the preset cross entropy loss function is obtained: Among them, N represents the number of incremental data, x i Represents the data characteristics of incremental data, y i Data annotations representing incremental data, The probability value of the category information corresponding to the i-th incremental data is specifically expressed in the form of a vector, the number of elements of the vector is equal to the number of category information, each element represents the probability value of the corresponding category information, and the value range of each element is between 0 and 1. When the output value of the loss function meets the preset training conditions, the training of the incremental sub-network 203 is stopped, and the shared sub-network 201 and the trained incremental sub-network 203 are used as incremental models.

[0082] As another example, the basic model includes a 6-layer network structure, wherein the 4th and 5th layers are used as the basic subnetwork, and the shared subnetwork includes: the 1st layer, the 2nd layer, the 3rd layer and the 6th layer; the basic subnetwork is copied to obtain an incremental subnetwork, and the incremental subnetwork is trained using incremental data; when the incremental model is obtained based on the shared subnetwork and the trained incremental subnetwork, only the shared subnetwork before the basic subnetwork and the trained incremental subnetwork are used as incremental models, that is, the 1st layer, the 2nd layer, the 3rd layer and the trained incremental subnetwork of the above-mentioned basic model are used as incremental models. The shared subnetwork after the basic subnetwork is ignored, so that the output of the incremental model is the output of the trained incremental subnetwork, thereby improving the recognition accuracy of the incremental model for new category information.

[0083] Training the incremental sub-network and using the shared sub-network and the trained incremental sub-network as the incremental model can improve the training efficiency of the incremental model, save system resources, and enable the incremental model to better learn the deep learning features of the incremental data.

[0084] According to a reference embodiment of the present invention, before training the incremental sub-network, obtain the preset basic data, which is the data used to train the basic model, and the basic data is the data with deep learning features corresponding to the old category information. Mix the incremental data and the basic data to obtain mixed data. For example, first determine the total number of mixed data, and then select a part of the incremental data as mixed data and a part of the basic data as mixed data according to the proportion of the preset incremental data and basic data in the mixed data; for another example, select a part of the incremental data as the first data to be spliced, select a part of the basic data as the second data to be spliced, and splice the first data to be spliced ​​with the second data to be spliced ​​to obtain mixed data, each mixed data is the splicing result of the basic data and the incremental data, so that each mixed data has both the deep learning features corresponding to the old category information and the deep learning features corresponding to the new category information.

[0085] Exemplarily, there are 10 basic data and 10 incremental data, the total number of pre-set mixed data is 10, and the ratio of basic data to incremental data is 1:9, therefore, 1 basic data is randomly selected from the above 10 basic data, and 9 incremental data are randomly selected from the above 10 incremental data; 1 basic data and 9 incremental data are used as mixed data to train the incremental subnetwork.

[0086] After obtaining the mixed data, the incremental sub-network is progressively trained using the mixed data. For example, when extracting deep learning features based on the mixed data, the deep learning features of a portion of the channel outputs are randomly discarded by adjusting the preset discard rate, thereby solving the overfitting problem in the incremental sub-network and reducing the mutual adaptability of the incremental sub-network. For another example, the RandAugment data enhancement strategy is executed on the mixed data including the basic data and the incremental data, and the mixed data after data enhancement is trained again.

[0087] Figure 3 is a schematic diagram of incremental training of a model training method according to a reference embodiment of the present invention. For example, Figure 3 As shown, the basic model is the EfficientNetV2 classification model. Specifically, the basic model includes a shared subnetwork Φ s (i.e., Stage 0 to Stage 6 of the EfficientNetV2 classification model) and the basic subnetwork Φ s and W b (i.e. Conv1×1&Pooling&FC), {D b , Y b} is the basic data used to train the basic model. The basic sub-network is copied (i.e. cloned) to obtain the incremental sub-network Φ n and Wn , the incremental sub-network has the same network structure as the basic sub-network. When training the incremental sub-network, freeze the shared sub-network Φ s , filter out some basic data and add incremental data {D n , Y n}, use the incremental data after adding some basic data to the incremental sub-network Φ n and W n Conduct training.

[0088] Mixing the incremental data with the basic data and using the mixed data to train the incremental sub-network is conducive to the rapid convergence of the learning process of the incremental model and further improves the training efficiency of the incremental model.

[0089] According to another reference embodiment of the present invention, the incremental data includes incremental data features and incremental data annotations, and the basic data includes basic data features and basic data annotations; when the incremental data and the basic data corresponding to the basic model are mixed, the incremental data annotations and basic data annotations with overlapping relationships and the corresponding incremental data features and basic data features are first determined, specifically, the basic data annotations that are the same as the incremental data annotations are searched in the basic data annotation set, and when the basic data annotations are the same as the incremental data annotations, it is determined that the basic data annotations overlap with the incremental data annotations. After determining the basic data annotations and incremental data annotations with overlapping relationships, the basic data features corresponding to the basic data annotations and the incremental data features corresponding to the incremental data annotations are obtained.

[0090] For example, assume that the basic data annotation set is Y b , the number of basic data annotations is |Y b |, the incremental data annotation set is Y n , the number of incremental data annotations is |Y n |, then the total data annotation set is Y all =Y b ∪Y n The total number of data annotations is |Y all |, if |Y all |≤|Y b |+|Y n |, it means that there is an overlapping relationship between the basic data identifier and the incremental data identifier. b , obtain each basic data annotation, in the incremental data annotation set Y n The incremental data annotations that are the same as the obtained basic data annotations are searched in the data, so as to determine the basic data annotations and incremental data annotations that have overlapping relationships.

[0091] According to the preset mixing ratio λ, the incremental data annotations and basic data annotations with overlapping relationships are mixed to obtain mixed data annotations. For example, the basic data annotation set Y b The basic data annotation y in j and incremental data annotation set Y n Incremental data annotation y in i With overlapping relationship, the mixed data is labeled as According to the mixing ratio λ, the corresponding incremental data features and basic data features are mixed into mixed data features. For example, the basic data annotation y is obtained. j The corresponding basic data feature x j , get the incremental data annotation y i The corresponding incremental data feature x i , then the mixed data feature is For the mixing ratio λ, it obeys the Beta distribution, for example, λ~Beta(α, α), α∈(0,∞). During the training process, the mixing ratio λ is adjusted to control the participation intensity of the basic data and incremental data. The mixed data features obtained by the mixing operation and the mixed data annotation are combined to obtain the mixed data. Among them, M is the number of mixed data, is a mixed data feature, It is mixed data annotation.

[0092] By controlling the intensity of basic data and incremental data participating in training through the mixing ratio, the incremental data can be better integrated with the basic data, which is conducive to the rapid convergence of the learning process and improves the training efficiency of the incremental sub-network.

[0093] According to another reference embodiment of the present invention, when the incremental sub-network does not include a linear classifier, the method further includes: copying the linear classifier of the base model to obtain an incremental linear classifier, and adding the incremental linear classifier to the incremental sub-network, specifically, adding the incremental linear classifier to the end of the incremental sub-network. For example, Figure 2 As shown, in the incremental subnetwork 203, the fully connected layer (i.e., the FC of the incremental subnetwork) is the linear classifier of the incremental subnetwork. If the incremental subnetwork 203 does not include a fully connected layer, the fully connected layer in the base subnetwork is copied, and then the copied fully connected layer is added to the end of the incremental subnetwork 203, so that the fully connected layer is used as the last layer of the incremental subnetwork.

[0094] During training, the incremental subnetwork after adding the incremental linear classifier is trained as a whole. Specifically, the incremental subnetwork after adding the incremental linear classifier is progressively trained using the pre-set incremental data. The specific method of progressive training has been described in detail in the previous embodiment and will not be repeated here.

[0095] Adding a linear classifier to the incremental subnetwork can ensure that the incremental subnetwork includes the linear classifier, so that the incremental subnetwork after training has the ability to classify and can quickly and accurately determine the category information corresponding to the incremental data.

[0096] Step S103, fusing the basic model and the incremental model to obtain an improved model corresponding to the basic model.

[0097] The basic model includes a shared subnetwork and a basic subnetwork, the incremental model includes a shared subnetwork and a trained incremental subnetwork, and both the basic model and the incremental model include corresponding linear classifiers. The basic model and the incremental model are fused, that is, the linear classifier of the basic model and the linear classifier of the incremental model are fused to obtain a unified linear classifier, so that the unified linear classifier obtains the final classification result based on the classification result of the linear classifier of the basic model and the classification result of the linear classifier of the incremental model. The basic model, the incremental model and the unified linear classifier formed by the fusion of the linear classifiers of the two are used as components of the improved model.

[0098] There are many ways to fuse the linear classifiers of the basic model and the linear classifiers of the incremental model. For example, the confidence score of the classification result of each linear classifier is calculated, and the classification result with the largest confidence score is used as the classification result of the unified linear classifier. For example, if the confidence score of the classification result of the basic model is 0.95 and the confidence score of the classification result of the incremental model is 0.9, the classification result corresponding to the basic model is used as the classification result of the improved model. For another example, a routing classifier is trained based on the incremental data and the basic data, and the routing classifier is defined as: Among them, h b is the feature extractor corresponding to the base model, h n is the feature extractor corresponding to the incremental model, is the parameter of the routing classifier, σ is the activation function, for example, Softmax function, Sigmoid function, etc. The routing classifier is used to determine the category information of the input data, that is, to improve the classification result of the model.

[0099] The basic model and the incremental model are integrated to obtain the corresponding improved model, which can enable the improved model to meet the feature extraction requirements of different data and improve the accuracy of feature extraction.

[0100] According to another reference embodiment of the present invention, the pre-set incremental data has multiple groups, and the incremental data of each group corresponds to different category information; according to the number of groups of the incremental data, the basic subnetwork is copied to obtain multiple incremental subnetworks, and the number of the multiple incremental subnetworks is the same as the number of groups of the incremental data; different incremental subnetworks are trained using incremental data of different groups, so that each incremental subnetwork learns the deep learning features of the incremental data of different groups; after training the multiple incremental subnetworks, the shared subnetwork is combined with each trained incremental subnetwork respectively to obtain multiple incremental models, and the number of incremental models is the same as the number of groups of the incremental data; the basic model and the multiple incremental models are fused to obtain an improved model corresponding to the basic model.

[0101] Figure 4 is a schematic diagram of incremental training of a model training method according to another reference embodiment of the present invention. For example, Figure 4 As shown, the basic model is the EfficientNetV2 classification model. Specifically, the basic model includes a shared subnetwork Φ s (i.e., Stage 0 to Stage 6 of the EfficientNetV2 classification model) and the basic subnetwork Φ s and W b (i.e. Conv1×1&Pooling&FC), {D b ,Y b} is the basic data used to train the basic model. n1 ,Y n2}, {D n2 ,Y n2}, ..., {D nt ,Y nt} represents the incremental data of t groups, each of which has different deep learning features and corresponds to different new category information. The basic subnetwork is copied (i.e. cloned) to obtain t incremental subnetworks: Incremental subnetwork Φ n1 &W n2 , incremental sub-network Φ n1 &W n2 ,..., incremental sub-network Φ nt &W nt Each incremental subnetwork has the same network structure as the basic subnetwork. When training each incremental subnetwork, the shared subnetwork Φ is frozen. s, filter out some basic data and add them to the incremental data of each group. You can add the same basic data to each group, or determine the basic data to be added to the incremental data of each group according to the overlapping relationship between the basic data and the incremental data of each group. Then mix the filtered basic data with the incremental data with overlapping relationship, and use the incremental data after adding the basic data or mixing the basic data to train the corresponding incremental sub-network. s Each trained incremental sub-network is used as an incremental model to obtain t incremental models; the basic model is further fused with the t incremental models to obtain the improved model corresponding to the basic model.

[0102] Different incremental models are trained using incremental data of different groups, and then the basic model and multiple incremental models are integrated to obtain an improved model. This allows the improved model to be applicable to the feature extraction process of various data, improves the universal applicability and training efficiency of the improved model, avoids the use of excessive resources, and reduces time costs.

[0103] Figure 5 FIG. 1 is a schematic diagram of the main process of a model training method according to a reference embodiment of the present invention. Figure 5 As shown, the model training method may include:

[0104] Step S501, dividing the preset basic model into a shared sub-network and a basic sub-network;

[0105] Step S502, copy the basic sub-network to obtain an incremental sub-network; obtain incremental data and basic data;

[0106] Step S503, acquiring incremental data and basic data, where the incremental data includes incremental data features and incremental data annotations, and the basic data includes basic data features and basic data annotations;

[0107] Step S504, determining the incremental data annotations and basic data annotations having overlapping relationships and the corresponding incremental data features and basic data features;

[0108] Step S505, mixing the incremental data annotations and the basic data annotations having overlapping relationships into mixed data annotations according to a preset mixing ratio;

[0109] Step S506, mixing the corresponding incremental data features and basic data features into mixed data features according to the mixing ratio;

[0110] Step S507, obtaining mixed data according to the mixed data features and the mixed data annotations, and using the mixed data to perform progressive training on the incremental sub-network;

[0111] Step S508, using the shared sub-network and the trained incremental sub-network as an incremental model;

[0112] Step S509: fuse the basic model and the incremental model to obtain an improved model corresponding to the basic model.

[0113] The specific implementation content of the model training method of the above-mentioned reference embodiment of the present invention has been described in detail in the above-mentioned model training method, so the repeated content will not be described here.

[0114] According to a second aspect of an embodiment of the present invention, a method for image classification is provided.

[0115] Figure 6 FIG. 1 is a schematic diagram of the main process of the method for image classification according to an embodiment of the present invention. Figure 6 As shown, the image classification method according to the embodiment of the present invention mainly includes the following steps S601 to S602.

[0116] Step S601, in response to receiving an image to be predicted, inputting the image to be predicted into a preset image classification model so that the image classification model extracts deep learning features of the image to be predicted, and the image classification model is obtained by using any method described in the first aspect of the embodiments of the present invention.

[0117] Step S602: Generate category information of the image to be predicted according to the deep learning features of the image to be predicted.

[0118] A trained basic model is pre-set, and the basic model can recognize the category information of most of the images to be predicted, but it is difficult for the basic model to accurately recognize the newly added category information. Therefore, the basic model is trained using any of the methods described in the first aspect of the embodiments of the present invention to obtain an image classification model. After receiving the image to be predicted, the image to be predicted is input into the image classification model, and the feature extractor of the image classification model is used to extract the deep learning features contained in the image to be predicted. After obtaining the deep learning features, the linear classifier of the image classification model is used to perform matrix operations on the deep learning features, and then after nonlinear transformation, the probability value of the image to be predicted belonging to each category of information is obtained, and the category information of the image to be predicted is determined according to the probability value of the image to be predicted belonging to each category of information.

[0119] Figure 7 FIG. 1 is a schematic diagram of the overall process of an image classification method according to a reference embodiment of the present invention. For example, Figure 7As shown, the basic model is an EfficientNetV2 classification model trained using a base class data set. After adding new category information, according to any method described in the first aspect of an embodiment of the present invention, a new class n1 data set, a new class n2 data set, and other data sets annotated with new category information are used to train the basic model (i.e., class incremental learning) to obtain an image classification model based on EfficientNetV2. After receiving the image to be predicted uploaded by the user, the image to be predicted is preprocessed, such as format conversion, size adjustment, data cleaning, etc., and the image to be predicted after preprocessing is input into the image classification model for feature extraction, matrix operation, and nonlinear transformation to obtain the probability value of the image to be predicted belonging to each category information. For example, the probability value calculation formula is p(x)=σ(W T h(x;Φ)), where σ is the activation function, such as Softmax, Sigmoid, etc., and W T is a linear classifier, h(x;Φ) is a feature extractor, specifically, x represents the image to be predicted, The network parameters of the feature extractor are shown.

[0120] When determining the category information of the image to be predicted, the category information corresponding to the element with the largest probability value is used as the category information of the image to be predicted, or the category information corresponding to multiple elements with higher probability values ​​are selected as the category information of the image to be predicted, that is, the category information of the image to be predicted can be one or more category information. For example, the probability values ​​of the image to be predicted belonging to each category information are obtained as "{'category1':'0.8'; 'category2':'0.3'; 'category3':'0.9'}", wherein "'category1':'0.8'" indicates that the probability of the image to be predicted belonging to category information 1 is 0.8, "'category2':'0.3'" indicates that the probability of the image to be predicted belonging to category information 2 is 0.3, and "'category3':'0.9'" indicates that the probability of the image to be predicted belonging to category information 3 is 0.9; when the probability threshold is set to 0.85, category information 1 with a probability value greater than the probability threshold is used as the category information of the image to be predicted. When each image to be predicted can have at most two category information and the probability value of each category information should be greater than 0.6, category information 1 and category information 2 are used as the category information of the image to be predicted.

[0121] Using an image classification model trained by any of the methods described in the first aspect of the embodiments of the present invention to identify the image to be predicted can improve the recognition efficiency and accuracy of the image to be predicted, avoid manual verification, and reduce labor costs; and the training method described in any of the first aspect of the embodiments of the present invention is generally applicable to judgment tasks and recognition tasks in multiple fields, and is suitable for many situations such as frequent updates and iterations of input data, the need to judge input data according to new judgment criteria, large amounts of training data, and many parameters in the network structure.

[0122] Figure 8 FIG. 1 is a schematic diagram of the main process of an image classification method according to a reference embodiment of the present invention. Figure 8 As shown, the image classification method may include:

[0123] Step S801, receiving an image to be predicted sent by a user, and performing image preprocessing operations such as format conversion, size adjustment, and data cleaning on the image to be predicted;

[0124] Step S802, inputting the image to be predicted after image preprocessing into a preset image classification model, where the image classification model is obtained according to any method of the first aspect of the embodiments of the present invention;

[0125] Step S803, using a feature extractor of the image classification model to extract deep learning features contained in the image to be predicted;

[0126] Step S804, using the linear classifier of the image classification model to perform matrix operations on the deep learning features, and then performing nonlinear transformation to obtain the probability value of the image to be predicted belonging to each category of information;

[0127] Step S805 , determining the category information of the image to be predicted according to the probability value that the image to be predicted belongs to each category information.

[0128] The specific implementation content of the image classification method of the above-mentioned reference embodiment of the present invention has been described in detail in the above-mentioned image classification method, so the repeated content will not be described again here.

[0129] According to a third aspect of an embodiment of the present invention, a device for model training is provided.

[0130] Fig. 9 is a schematic diagram of the main modules of the device for model training according to an embodiment of the present invention, such as Fig. 9 As shown, the model training device 900 mainly includes:

[0131] A division module 901 is used to divide a preset basic model into a shared sub-network and a basic sub-network;

[0132] A training module 902 is used to copy the basic sub-network to obtain an incremental sub-network, train the incremental sub-network using preset incremental data, and use the shared sub-network and the trained incremental sub-network as incremental models;

[0133] The fusion module 903 is used to fuse the basic model and the incremental model to obtain an improved model corresponding to the basic model.

[0134] According to a reference embodiment of the present invention, the incremental sub-network is trained using preset incremental data, including:

[0135] Mixing the incremental data with the basic data corresponding to the basic model to obtain mixed data;

[0136] The incremental sub-network is progressively trained using the mixed data.

[0137] According to another reference embodiment of the present invention, the incremental data includes incremental data features and incremental data annotations, and the basic data includes basic data features and basic data annotations; the incremental data and the basic data corresponding to the basic model are mixed to obtain mixed data, including:

[0138] Determine the incremental data annotations and basic data annotations having overlapping relationships and the corresponding incremental data features and basic data features;

[0139] According to a preset mixing ratio, the incremental data annotations and basic data annotations having overlapping relationships are mixed into mixed data annotations;

[0140] According to the mixing ratio, the corresponding incremental data features and basic data features are mixed into mixed data features;

[0141] The mixed data is obtained according to the mixed data features and the mixed data annotations.

[0142] According to another reference embodiment of the present invention, the incremental data has multiple groups; the basic sub-network is copied to obtain an incremental sub-network, and the incremental sub-network is trained using the preset incremental data, including: copying the basic sub-network to obtain multiple incremental sub-networks, and training different incremental sub-networks using incremental data of different groups;

[0143] The basic model and the incremental model are fused to obtain an improved model corresponding to the basic model, including: combining the shared subnetwork with each trained incremental subnetwork respectively to obtain multiple incremental models, and fusing the basic model with the multiple incremental models to obtain an improved model corresponding to the basic model.

[0144] According to another reference embodiment of the present invention, the model training device 900 further includes: an adding module, which is used to copy the linear classifier of the basic model to obtain an incremental linear classifier, and add the incremental linear classifier to the incremental sub-network;

[0145] The incremental sub-network is trained using the preset incremental data, including: the incremental sub-network to which the incremental linear classifier is added is trained using the preset incremental data.

[0146] According to another reference embodiment of the present invention, the basic sub-network is the last several layers of the network structure corresponding to the basic model.

[0147] It should be noted that the specific implementation content of the model training device described in the embodiment of the present invention has been described in detail in the model training method described above, so the repeated content will not be described here.

[0148] According to a fourth aspect of an embodiment of the present invention, a device for image classification is provided.

[0149] Fig.10 is a schematic diagram of main modules of an apparatus for image classification according to an embodiment of the present invention, such as Fig.10 As shown, the image classification device 1000 mainly includes:

[0150] An extraction module 1001 is used for, in response to receiving an image to be predicted, inputting the image to be predicted into a preset image classification model, so that the image classification model extracts deep learning features of the image to be predicted, wherein the image classification model is obtained by using any method described in the first aspect of the embodiments of the present invention;

[0151] The generating module 1002 is used to generate the category information of the image to be predicted according to the deep learning features of the image to be predicted.

[0152] It should be noted that the specific implementation content of the image classification device in the embodiment of the present invention has been described in detail in the above-mentioned image classification method, so the repeated content will not be described again here.

[0153] According to the technical solution of the embodiment of the present invention, the basic model is subjected to quasi-incremental learning to obtain an incremental model, which can improve the model training efficiency, save system resources, and enable the incremental model to better learn the deep learning features of the incremental data; the basic model and the incremental model are fused to obtain the corresponding improved model, which can enable the improved model to meet the feature extraction requirements of different data and improve the accuracy of feature extraction; the incremental data is mixed with the basic data, and the incremental sub-network is trained using the mixed data, which is conducive to the rapid convergence of the learning process of the incremental model and further improves the training efficiency of the incremental model; the mixed data is obtained according to the overlapping relationship and the mixing ratio, which can flexibly adjust the intensity of the basic data and the incremental data participating in the model training, which is conducive to improving the accuracy of the incremental model feature extraction; different incremental models are trained using incremental data of different groups, and then the basic model and multiple incremental models are fused to obtain an improved model, which can make the improved model applicable to the feature extraction process of various data, improve the universal applicability and training efficiency of the improved model, avoid using too many resources, and reduce time cost.

[0154] According to a fifth aspect of an embodiment of the present invention, there is provided an electronic device for model training, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided by the first aspect of the embodiment of the present invention and / or the second aspect of the embodiment of the present invention.

[0155] According to a sixth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method provided by the first aspect of the embodiment of the present invention and / or the second aspect of the embodiment of the present invention is implemented.

[0156] Fig.11 An exemplary system architecture 1100 is shown to which the method for model training or the apparatus for model training according to an embodiment of the present invention can be applied.

[0157] like Fig.11 As shown, system architecture 1100 may include terminal devices 1101, 1102, 1103, a network 1104, and a server 1105. Network 1104 is used to provide a medium for communication links between terminal devices 1101, 1102, 1103 and server 1105. Network 1104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0158] The user can use the terminal devices 1101, 1102, 1103 to interact with the server 1105 through the network 1104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 1101, 1102, 1103, such as model training applications, work order feedback applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0159] The terminal devices 1101 , 1102 , and 1103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0160] Server 1105 may be a server that provides various services, such as a background management server (for example only) that provides support for model training requests sent by upstream terminal devices 1101, 1102, and 1103. In response to receiving a model training request, the background management server may obtain the corresponding video data and operation data set, and generate an association relationship between the video data and the operation data set according to the playback duration of the video data and the operation time point corresponding to the operation data set; in response to receiving a video playback request, play the video data, and determine the operation data corresponding to the playback time point of the video data according to the association relationship, so that the target user generates feedback data according to the video data and the operation data; in response to receiving a work order feedback request, obtain the feedback data, and if the feedback data passes the test, use the feedback data to update the target system corresponding to the model training request; and feedback the model training situation (for example only) to the terminal device.

[0161] It should be noted that the model training method provided in the embodiment of the present invention is generally performed by the server 1105, and accordingly, the model training device is generally set in the server 1105. The model training method provided in the embodiment of the present invention can also be performed by the terminal devices 1101, 1102, and 1103, and accordingly, the model training device can be set in the terminal devices 1101, 1102, and 1103.

[0162] It should be understood that Fig.11 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0163] Reference below Fig.12 , which shows a schematic diagram of the structure of a computer system 1200 of a terminal device suitable for implementing an embodiment of the present invention. Fig.12 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0164] like Fig.12 As shown, the computer system 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage part 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the system 1200 are also stored. The CPU 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0165] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed, so that a computer program read therefrom is installed into the storage section 1208 as needed.

[0166] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1209, and / or installed from the removable medium 1211. When the computer program is executed by the central processing unit (CPU) 1201, the above-mentioned functions defined in the system of the embodiment of the present invention are executed.

[0167] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In an embodiment of the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In an embodiment of the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0168] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0169] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes an integration module, a playback module, and an update module, wherein the names of these modules do not constitute a limitation on the modules themselves under certain circumstances, for example, the integration module may also be described as a "module for integrating video data and operating data sets".

[0170] As another aspect, an embodiment of the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device implements the following method: in response to receiving a model training request, obtaining corresponding video data and operation data sets, generating an association relationship between the video data and the operation data set according to the playback duration of the video data and the operation time point corresponding to the operation data set; in response to receiving a video playback request, playing the video data, determining the operation data corresponding to the playback time point of the video data according to the association relationship, so that the target user generates feedback data according to the video data and the operation data; in response to receiving a work order feedback request, obtaining the feedback data, and when the feedback data passes the test, using the feedback data to update the target system corresponding to the model training request.

[0171] According to the technical solution of the embodiment of the present invention, the basic model is subjected to quasi-incremental learning to obtain an incremental model, which can improve the model training efficiency, save system resources, and enable the incremental model to better learn the deep learning features of the incremental data; the basic model and the incremental model are fused to obtain the corresponding improved model, which can enable the improved model to meet the feature extraction requirements of different data and improve the accuracy of feature extraction; the incremental data is mixed with the basic data, and the incremental sub-network is trained using the mixed data, which is conducive to the rapid convergence of the learning process of the incremental model and further improves the training efficiency of the incremental model; the mixed data is obtained according to the overlapping relationship and the mixing ratio, which can flexibly adjust the intensity of the basic data and the incremental data participating in the model training, which is conducive to improving the accuracy of the incremental model feature extraction; different incremental models are trained using incremental data of different groups, and then the basic model and multiple incremental models are fused to obtain an improved model, which can make the improved model applicable to the feature extraction process of various data, improve the universal applicability and training efficiency of the improved model, avoid using too many resources, and reduce time cost.

[0172] The above specific implementations do not constitute a limitation on the protection scope of the embodiments of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. A model training method, characterized in that: include: Divide the pre-set basic model into a shared sub-network and a basic sub-network; The basic sub-network is copied to obtain an incremental sub-network, the incremental sub-network is trained using preset incremental data, and the shared sub-network and the trained incremental sub-network are used as incremental models; The basic model and the incremental model are fused to obtain an improved model corresponding to the basic model.

2. The method according to claim 1, characterized in that The incremental sub-network is trained using the preset incremental data, including: Mixing the incremental data with the basic data corresponding to the basic model to obtain mixed data; The incremental sub-network is progressively trained using the mixed data.

3. The method according to claim 2, characterized in that The incremental data includes incremental data features and incremental data annotations, and the basic data includes basic data features and basic data annotations; the incremental data and the basic data corresponding to the basic model are mixed to obtain mixed data, including: Determine the incremental data annotations and basic data annotations having overlapping relationships and the corresponding incremental data features and basic data features; According to a preset mixing ratio, the incremental data annotations and basic data annotations having overlapping relationships are mixed into mixed data annotations; According to the mixing ratio, the corresponding incremental data features and basic data features are mixed into mixed data features; The mixed data is obtained according to the mixed data features and the mixed data annotations.

4. The method according to claim 1, characterized in that: The incremental data has multiple groups; the basic sub-network is copied to obtain an incremental sub-network, and the incremental sub-network is trained using the preset incremental data, including: the basic sub-network is copied to obtain multiple incremental sub-networks, and different incremental sub-networks are trained using incremental data of different groups; The basic model and the incremental model are fused to obtain an improved model corresponding to the basic model, including: combining the shared subnetwork with each trained incremental subnetwork respectively to obtain multiple incremental models, and fusing the basic model with the multiple incremental models to obtain an improved model corresponding to the basic model.

5. The method according to claim 1, characterized in that In the case where the incremental subnetwork does not include a linear classifier, the method further includes: copying the linear classifier of the base model to obtain an incremental linear classifier, and adding the incremental linear classifier to the incremental subnetwork; The incremental sub-network is trained using the preset incremental data, including: the incremental sub-network to which the incremental linear classifier is added is trained using the preset incremental data.

6. The method according to claim 1, characterized in that The basic sub-network is the last several layers of the network structure corresponding to the basic model.

7. A method for image classification, characterized in that: include: In response to receiving an image to be predicted, inputting the image to be predicted into a preset image classification model so that the image classification model extracts deep learning features of the image to be predicted, wherein the image classification model is obtained by using any one of the methods described in claims 1 to 6; Generate category information of the image to be predicted based on the deep learning features of the image to be predicted.

8. A device for model training, characterized in that: include: A partitioning module, used for partitioning a preset basic model into a shared sub-network and a basic sub-network; A training module, used to copy the basic sub-network to obtain an incremental sub-network, train the incremental sub-network using preset incremental data, and use the shared sub-network and the trained incremental sub-network as incremental models; A fusion module is used to fuse the basic model and the incremental model to obtain an improved model corresponding to the basic model.

9. An electronic device, characterized in that: include: One or more processors; A storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images