Data processing method, device, electronic device and medium

Abstract

The embodiments of the present application disclose a data processing method, device, electronic device and medium, which can be applied to the field of data processing technology. The method includes: obtaining a second classification model, and adding a first target category to the classification category corresponding to the second classification model; obtaining the central sample data corresponding to the first sample data under each initial category, each central sample data refers to the first sample data with the smallest feature distance between the corresponding first global feature and the average global feature of the initial category to which it belongs; obtaining a set of second sample data; iteratively training the second classification model with added classification categories based on multiple second sample data and central sample data corresponding to N initial categories to obtain a third classification model, and using the third classification model to classify and identify the input data. The use of the embodiments of the present application helps to improve the model performance of the model obtained based on incremental learning training without relying on a large amount of old task data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to data processing methods, devices, electronic devices and media. Background Art

[0002] The ability of incremental learning is the ability to continuously process the continuous flow of information in the real world, and to retain, even integrate and optimize old knowledge while absorbing new knowledge. At present, there are many ways to do incremental learning, such as replay-based incremental learning, which converts incremental learning into small data set training tasks and trains them together with new task data by retaining some old task data to ensure that the old task indicators do not decrease. However, the inventors found in practice that the training method of replay-based incremental learning, as the classification categories of the model increase, the retained old task data will also increase, and then the training data will continue to increase, which is contrary to the essence of incremental learning. Summary of the invention

[0003] The embodiments of the present application provide a data processing method, device, electronic device and medium, which help to improve the model performance of a model obtained through incremental learning training without relying on a large amount of old task data.

[0004] On the one hand, an embodiment of the present application discloses a data processing method, the method comprising:

[0005] Obtain a second classification model, and add a first target category to the classification category corresponding to the second classification model; the second classification model is obtained by training the first classification model based on the first sample data set, the classification category corresponding to the second classification model includes N initial categories, and the N initial categories are different from the first target category; the first sample data set includes a plurality of first sample data under the N initial categories; N is a positive integer;

[0006] Obtaining central sample data corresponding to the first sample data under each initial category, each central sample data refers to the first sample data having the smallest feature distance between the corresponding first global feature and the average global feature of the initial category to which it belongs, the first global feature of each first sample data being obtained by encoding the first sample data by an encoder based on a classification model, and the average global feature of each initial category being an average feature of the first global features of the corresponding plurality of first sample data;

[0007] Acquire a second sample data set, where the second sample data set includes a plurality of second sample data, and each second sample data belongs to the first target category;

[0008] Based on the multiple second sample data and the central sample data corresponding to the N initial categories, the second classification model with added classification categories is iteratively trained to obtain a third classification model, and the third classification model is used to classify and identify the input data; the classification categories corresponding to the third classification model include the N initial categories and the first target category.

[0009] In one aspect, an embodiment of the present application discloses a data processing device, the device comprising:

[0010] an acquisition unit, configured to acquire a second classification model, and add a first target category to the classification category corresponding to the second classification model; the second classification model is obtained by training the first classification model based on the first sample data set, the classification category corresponding to the second classification model includes N initial categories, and the N initial categories are different from the first target category; the first sample data set includes a plurality of first sample data under the N initial categories; N is a positive integer;

[0011] The acquisition unit is further used to acquire the central sample data corresponding to the first sample data under each initial category, each central sample data refers to the first sample data having the smallest feature distance between the corresponding first global feature and the average global feature of the initial category to which it belongs, the first global feature of each first sample data is obtained by encoding the first sample data by an encoder based on a classification model, and the average global feature of each initial category refers to the average feature of the first global features of the corresponding plurality of first sample data;

[0012] The acquisition unit is further used to acquire a second sample data set, where the second sample data set includes a plurality of second sample data, and each second sample data belongs to the first target category;

[0013] A processing unit is used to iteratively train a second classification model with added classification categories based on the multiple second sample data and the central sample data corresponding to the N initial categories to obtain a third classification model, and use the third classification model to classify and identify input data; the classification categories corresponding to the third classification model include the N initial categories and the first target category.

[0014] On the one hand, an embodiment of the present application provides an electronic device, the electronic device includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to perform the following steps:

[0015] Obtain a second classification model, and add a first target category to the classification category corresponding to the second classification model; the second classification model is obtained by training the first classification model based on the first sample data set, the classification category corresponding to the second classification model includes N initial categories, and the N initial categories are different from the first target category; the first sample data set includes a plurality of first sample data under the N initial categories; N is a positive integer;

[0016] Obtaining central sample data corresponding to the first sample data under each initial category, each central sample data refers to the first sample data having the smallest feature distance between the corresponding first global feature and the average global feature of the initial category to which it belongs, the first global feature of each first sample data being obtained by encoding the first sample data by an encoder based on a classification model, and the average global feature of each initial category being an average feature of the first global features of the corresponding plurality of first sample data;

[0017] Acquire a second sample data set, where the second sample data set includes a plurality of second sample data, and each second sample data belongs to the first target category;

[0018] Based on the multiple second sample data and the central sample data corresponding to the N initial categories, the second classification model with added classification categories is iteratively trained to obtain a third classification model, and the third classification model is used to classify and identify the input data; the classification categories corresponding to the third classification model include the N initial categories and the first target category.

[0019] On the one hand, an embodiment of the present application provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed by a processor, the computer program instructions are used to perform the following steps:

[0020] Obtain a second classification model, and add a first target category to the classification category corresponding to the second classification model; the second classification model is obtained by training the first classification model based on the first sample data set, the classification category corresponding to the second classification model includes N initial categories, and the N initial categories are different from the first target category; the first sample data set includes a plurality of first sample data under the N initial categories; N is a positive integer;

[0021] Obtaining central sample data corresponding to the first sample data under each initial category, each central sample data refers to the first sample data having the smallest feature distance between the corresponding first global feature and the average global feature of the initial category to which it belongs, the first global feature of each first sample data being obtained by encoding the first sample data by an encoder based on a classification model, and the average global feature of each initial category being an average feature of the first global features of the corresponding plurality of first sample data;

[0022] Acquire a second sample data set, where the second sample data set includes a plurality of second sample data, and each second sample data belongs to the first target category;

[0023] Based on the multiple second sample data and the central sample data corresponding to the N initial categories, the second classification model with added classification categories is iteratively trained to obtain a third classification model, and the third classification model is used to classify and identify the input data; the classification categories corresponding to the third classification model include the N initial categories and the first target category.

[0024] In one aspect, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions. When the computer instructions are executed by a processor, the method provided in the above aspect can be implemented.

[0025] By using the embodiment of the present application, when training the second classification model of the classification category with the first target category added, the central sample data corresponding to each classification category of the second classification model (i.e., N initial categories) can be determined, and each central sample data refers to the first sample data with the smallest feature distance between the corresponding first global feature and the average global feature of the initial category to which it belongs. Then, the second classification model of the added classification category is iteratively trained in combination with the central sample data corresponding to each initial category and the second sample data in the second sample data set belonging to the added first target category to obtain a third classification model. That is, for each initial category, a central sample data that best represents the initial category can be stored, and the central sample data corresponding to each initial category is used to train the classification model with the added first target category without relying on a large amount of old task data (i.e., based on the training task before obtaining the second classification model), so that the new task (i.e., the task of training the second classification model with the first target category added to obtain the third classification model) can be effectively trained, which helps to improve the model performance of the model obtained by incremental learning training without relying on a large amount of old task data. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0028] Figure 2 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0029] Figure 3 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0030] Figure 4 It is a schematic diagram of the effect of a model structure of a classification model provided in an embodiment of the present application;

[0031] Figure 5 It is a structural diagram of a classification model provided in an embodiment of the present application;

[0032] Figure 6 It is a flowchart of an incremental learning method provided in an embodiment of the present application;

[0033] Figure 7 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0034] Figure 8 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0036] An embodiment of the present application proposes a data processing scheme, which can determine the central sample data corresponding to each classification category (i.e., N initial categories) of the second classification model when training a second classification model with a first target category added to the classification category, and each central sample data refers to the first sample data with the smallest feature distance between the corresponding first global feature and the average global feature of the initial category to which it belongs, and then iteratively train the second classification model with the added classification category in combination with the central sample data corresponding to each initial category and the second sample data in the second sample data set belonging to the added first target category to obtain a third classification model. That is, for each initial category, a central sample data that best represents the initial category can be stored, and the central sample data corresponding to each initial category is used to train the classification model with the added first target category without relying on a large amount of old task data (i.e., based on the training task before obtaining the second classification model), so that the new task (i.e., the task of training the second classification model with the added first target category to obtain the third classification model) can be effectively trained, which helps to improve the model performance of the model obtained based on incremental learning training without relying on a large amount of old task data.

[0037] It should be noted that all user data collected by this application is collected with the user's consent and authorization, and the collection, use and processing of relevant user data must comply with relevant laws, regulations and standards of relevant countries and regions. For example, before collecting relevant user data and during the process of collecting relevant user data, this application can display a prompt message to remind the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining the user's relevant data after obtaining the user's confirmation operation issued by the prompt message, otherwise (that is, the user's confirmation operation issued by the prompt message is not obtained), the relevant steps of obtaining the user's relevant data are terminated, that is, the user's relevant data is not obtained.

[0038] The technical solution of the present application can be used in electronic devices. The electronic device can be a terminal, a server, or other devices for data processing, which is not limited in the present application. Optionally, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, and cloud communications. Terminals include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, and vehicle-mounted terminals.

[0039] It is understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. For example, it is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0040] Based on the above description, the present application embodiment proposes a data processing method. Figure 1 , Figure 1 1 is a flow chart of a data processing method provided in an embodiment of the present application. The method can be executed by the above-mentioned electronic device. The data processing method can include the following steps.

[0041] S101. Obtain a second classification model, and add a first target category to the classification category corresponding to the second classification model; the second classification model is obtained by training the first classification model based on the first sample data set, and the classification categories corresponding to the second classification model include N initial categories, and the N initial categories are different from the first target category; the first sample data set includes multiple first sample data under the N initial categories; N is a positive integer.

[0042] Among them, the second classification model can be a classification model trained on the basis of the first classification model. It is understandable that the second classification model can be obtained by fully training the basic classification model, or it can be obtained by one or more incremental learning based on the trained basic classification model. In other words, the first classification model can be a basic classification model, that is, it can be an untrained classification model. The first classification model can also be a classification model that has performed one or more training tasks on the basic classification model, and there is no limitation here. It is understandable that the basic classification model can be an untrained classification model, and the classification category corresponding to the basic classification model can be called the basic category. That is to say, if the first classification model is the basic classification model, the N initial categories are the basic categories corresponding to the basic classification model.

[0043] It is understandable that the training task before obtaining the second classification model can be called the old task for the second classification model. For example, for the second classification model, the old task can be the task of obtaining the second classification model based on the first classification model training, and the model training based on the second classification model through incremental learning can be called the new task for the second classification model. It is understandable that for the sake of ease of description, the task of training the basic classification model based on the full amount of sample data can be called the 0th task, and the training task of performing the first incremental learning on the trained basic classification model can be called the 1st task. By analogy, the training task of performing incremental learning again on the classification model after one incremental learning can be called the 2nd task.

[0044] It is understandable that the first sample data set may be sample data required for training the second classification model. The first sample data may also be referred to as a set of sample data for the old task of the second classification model. The first sample data set may include multiple first sample data, and the number of first sample data under each initial category may be one or more.

[0045] It can be further understood that the second classification model can be obtained by fully training the basic classification model, and the first sample data set can be the full amount of sample data for the basic classification model. For example, the classification categories corresponding to the basic classification model are set to N initial categories: {category 0, category 1, category 2, category 3, category 4}, and then the first classification model of the basic classification model can be iteratively trained based on each first sample data pair under the N initial categories to obtain the second classification model.

[0046] If the second classification model is obtained by performing one or more incremental learning on the trained basic classification model, the first sample data set may include the sample data set for training the basic classification model and the sample data set obtained by each incremental learning. For example, the classification categories corresponding to the basic classification model include {category 0, category 1, category 2, category 3, category 4}, and then a sample data set 1 consisting of sample data belonging to category 0, category 1, category 2, category 3, and category 4 can be obtained, and the basic classification model is trained using the sample data set 1 to obtain classification model 1; then, based on classification model 1, a second classification model is obtained by incremental learning, that is, category 5 can be added to the classification category corresponding to classification model 1, and a sample data set 2 consisting of sample data belonging to category 5 can be obtained, and the classification model 1 is trained using the sample data set 2 to obtain the second classification model. It can be understood that the above-mentioned first sample data set may include sample data set 1 and sample data set 2. Category 0, category 1...category 5 can all be called classification categories corresponding to the second classification model, that is, the N initial categories mentioned above. In the first sample data set, each initial category has one or more sample data, that is, sample data set 1 can have sample data under each category of categories 0-4, and sample data set 2 can have sample data under category 5.

[0047] It can be understood that, in the first sample data set, each first sample data may be associated with a corresponding sample label, and the sample label associated with each first sample data may be used to indicate the initial category to which the first sample data belongs.

[0048] Among them, it can be understood that the first target category can be the target category required to be added for incremental learning of the second classification model, and the target category can be used to indicate the category required to be added for incremental learning of the classification model, also known as the incremental category. The number of first target categories added to the classification category of the second classification model can be one or more, and there is no restriction here. Adding the first target category to the classification category corresponding to the second classification model can be to adjust the parameters used to indicate the classification category in the second classification model, so that after training the second classification model with the added classification category, the N initial categories and the added first target category can be classified and identified. For example, the N initial categories include: category 0, category 1, category 2, category 3, category 4, category 5, and the added first target category is category 6, and the added first target category is different from each initial category.

[0049] It can be understood that the first classification model and the second classification model here can both be called classification models, and the model structures of the first classification model and the second classification model can be the same, but due to incremental training, the model parameters therein can be different.

[0050] In one embodiment, the classification model may include an encoder and a classification layer, wherein the encoder may be used to perform feature encoding based on input data to obtain corresponding global features. The classification layer may be used to perform classification and recognition processing on the global features encoded by the encoder to obtain classification and recognition results, also known as a fully connected layer. Optionally, the classification model may also include a decoder, which may be used to decode the global features encoded by the encoder to obtain corresponding decoded data.

[0051] S102. Obtain the central sample data corresponding to the first sample data under each initial category. Each central sample data refers to the first sample data having the smallest feature distance between the corresponding first global feature and the average global feature of the initial category to which it belongs. The first global feature of each first sample data is obtained by encoding each first sample data by an encoder based on a classification model. The average global feature of each initial category refers to the average feature of the first global features of the corresponding first sample data.

[0052] The central sample data may be the first sample data whose corresponding first global feature under any initial category is closest to the average global feature of the initial category.

[0053] In one embodiment, if the second classification model is obtained by fully training the basic classification model, that is, the first classification model is the basic classification model, then the first global feature of each first sample data is a global feature obtained by encoding the first sample data by the encoder of the first classification model during the last iterative training, and then the average global feature of each initial category can be determined based on the first global feature corresponding to the first sample data under each initial category.

[0054] If the second classification model is obtained through one or more incremental learning based on the trained basic classification model, the first global feature of each first sample data can be a global feature obtained by the encoder encoding the first sample data of each basic category during the last iteration of the basic classification model training, or can also be a global feature obtained by the encoder encoding the first sample data of each added target category during the incremental learning of the trained basic classification model. For example, the first sample data set includes a sample data set 1 for training the basic classification model, and a sample data set 2 required for the first classification model is obtained by training the trained basic classification model (such as classification model 1) through incremental learning, and a sample data set 3 required for the second classification model is obtained by training the first classification model through incremental learning. Among them, sample data set 1 includes sample data under categories 0-4, sample data set 2 includes sample data under category 5, and sample data set 3 includes sample data under category 6. Then, the first global feature of the first sample data under categories 0-4 in the first sample data set is obtained by feature encoding each first sample data under categories 0-4 based on the encoder of the basic classification model at the last iteration training, the first global feature of the first sample data under category 5 in the first sample data set is obtained by feature encoding each first sample data under category 5 based on the encoder of the classification model 1 at the last iteration training, and the first global feature of the first sample data under category 6 in the first sample data set is obtained by feature encoding each first sample data under category 6 based on the encoder of the first classification model at the last iteration training. Based on this, the first global feature of the first sample data under each initial category in the first sample data set can be determined, and then the average global feature of each initial category can be determined based on the average feature of the first global feature of the first sample data under each initial category.

[0055] Specifically, obtaining the central sample data corresponding to the first sample data under each initial category may include the following steps: obtaining the first global feature of the first sample data under each initial category, and determining the average global feature of each initial category based on the average feature of the first global feature of the first sample data under each initial category; calculating the feature distance between the first global feature of each first sample data and the average global feature of the initial category to which it belongs; and determining the first sample data with the smallest corresponding feature distance among the first sample data under each initial category as the central sample data corresponding to each initial category.

[0056] Among them, the average global feature of each initial category can be the average feature of the first global feature of the first sample data under each initial category. Calculating the feature distance can be calculating the Euclidean distance, Manhattan distance, etc. between the first global feature and the average global feature, which is not limited here. For example, there are multiple first sample data under category 1, such as samples 1, 2, 3...9, so that the first global features corresponding to samples 1, 2, 3...9 can be obtained, and the average global feature corresponding to category 1 can be determined based on the average features of the first global features corresponding to samples 1, 2, 3...9, and then the feature distances between the first global features corresponding to samples 1, 2, 3...9 and the average global features corresponding to category 1 are calculated respectively. If the feature distance between the first global feature of sample 3 and the average global feature corresponding to category 1 is the smallest, sample 3 can be determined as the central sample data corresponding to category 1.

[0057] S103: Acquire a second sample data set, where the second sample data set includes a plurality of second sample data, and each second sample data belongs to the first target category.

[0058] The second sample data set may be a sample data set for training the second classification model by incremental learning. It is understandable that the plurality of second sample data in the second sample data set all belong to the first target category. It is understandable that each second sample data may be associated with a corresponding sample label, and the sample label associated with each second sample data may be used to indicate the first target category to which the second sample data belongs.

[0059] S104. Iteratively train the second classification model with added classification categories based on multiple second sample data and central sample data corresponding to N initial categories to obtain a third classification model, and use the third classification model to classify and identify the input data; the classification categories corresponding to the third classification model include N initial categories and the first target category.

[0060] The third classification model may be a classification model obtained by training the second classification model in an incremental learning manner. The classification categories corresponding to the third root class model may include N initial categories and a first target category.

[0061] In some embodiments, iteratively training a second classification model with added classification categories based on multiple second sample data and central sample data corresponding to N initial categories to obtain a third classification model may include: performing feature encoding processing on the central sample data corresponding to each initial category and each second sample data through an encoder in the second classification model with added classification categories to obtain target training features for the second classification model with added classification categories; performing classification and recognition processing on each target training feature respectively through the second classification model with added classification categories to obtain a classification recognition result corresponding to each target training feature; iteratively training the second classification model with added classification categories based on the classification recognition result corresponding to each target training feature to obtain a third classification model.

[0062] The target training feature may be a training feature for training a second classification model for adding classification categories. It is understandable that the target training feature may include a second global feature obtained by performing feature encoding processing based on the second sample data, and a central sampling feature obtained by performing random sampling processing based on a global feature obtained by performing feature encoding processing on the central sample data.

[0063] It is understandable that the classification and recognition processing for each target training feature can be performed based on the classification layer in the second classification model, and the classification layer can also be called a fully connected layer, which can include a softmax layer (a logical function). It is understandable that the classification and recognition result corresponding to each target training feature can include the recognition probability that the sample data (i.e., the central sample data or the second sample data) corresponding to the target training feature belongs to each classification category (i.e., the N initial categories and the first target category).

[0064] It can be understood that the sample data corresponding to each target training feature is associated with a corresponding sample label. When the sample data corresponding to a target training feature is the central sample data, the corresponding sample label can be used to indicate the initial category to which the central sample data belongs; when the sample data corresponding to a target training feature is the second sample data, the corresponding sample label can be used to indicate the first target category to which it belongs.

[0065] It can be understood that the label probability for each classification category (i.e., N initial categories and the first target category) can be determined based on the sample label associated with the sample data corresponding to each target training feature. If the corresponding sample data belongs to a certain classification category, then in the label probability for each classification category (i.e., N initial categories and the first target category), the label probability of the certain classification category is 1, and the label probability of other classification categories is 0.

[0066] It can be understood that the second classification model with added classification categories is iteratively trained based on the classification recognition results corresponding to each target training feature to obtain a third classification model. The first reference information can be determined based on the difference between the recognition probability for each classification category indicated by the classification recognition results corresponding to each target training feature and the label probability for each classification category indicated by the sample label associated with the sample data corresponding to each target training feature, and then the second classification model with added classification categories is iteratively trained based on the first reference information to obtain the third classification model. The first reference information can be calculated based on a loss function, such as the loss function can be a cross entropy loss function. It can be understood that as the training proceeds, the classification recognition results corresponding to each training feature gradually approach the sample label associated with the sample data corresponding to each target training feature, that is, the first reference information gradually decreases. When the first reference information converges, the second classification model with added classification categories after iterative training can be determined as the third classification model.

[0067] It can be understood that the above-mentioned method of training the first classification model by incremental learning to obtain the second classification model can refer to the process of training the second classification model by incremental learning to obtain the third classification model here. It can be understood that the third classification model can be incrementally learned by this method on the basis of the third classification model, so that the trained classification model can recognize more classification categories and can improve the model performance of the classification model trained by incremental learning without relying on a large amount of old task data.

[0068] It is understandable that the third classification model can perform feature encoding on the input data, and then use the global features obtained by encoding for classification and recognition. For example, the third classification model can be used to identify key parts of the input image data, such as identifying key parts such as face, elbows, knees, etc., and then the encoder in the third classification model can be used to perform feature encoding on the input image data to obtain the corresponding global features of the image, and then the classification layer in the third classification model can be used to determine the classification and recognition results, which can include the probability that the image data belongs to each key part, so that the key part corresponding to the maximum probability is the key part identified based on the third classification model.

[0069] In one embodiment, the first sample data and the second sample data are both image data of key parts of the object; the third classification model includes an encoder and a classification layer, and the classification category corresponding to the third classification model includes the part category of the key part to which the input image data belongs; then, using the third classification model to classify and identify the input data may include the following steps: acquiring the image data to be processed, and obtaining the image global features of the image data to be processed based on the encoder encoding in the third classification model; performing classification and identification processing on the image global features through the classification layer in the third classification model to obtain a part recognition result corresponding to the image data to be processed, and the part recognition result is used to indicate the part category of the key part to which the image data to be processed belongs.

[0070] The image data to be processed may be image data of a key part to be identified. The global image feature may be a feature obtained by encoding the image data to be processed by an encoder in a third classification model. It is understandable that the part identification result may be used to indicate the part category of the key part to which the image data to be processed belongs. Optionally, the part identification result may be expressed as a probability of a part category for each key part, so that the key part corresponding to the part category with the highest probability may be determined as the key part to which the identified image data to be processed belongs.

[0071] It can be understood that, due to the training, the model parameters of each classification model (such as the third classification model, the second classification model and the first classification model) may be different, but the model structure may be the same, such as including encoders and classification layers, the encoders in each classification model may be different, and the parameters of the classification layer in each classification model may also be different.

[0072] It is understandable that the third classification model can also be used to feature encode the input data through an encoder, and then compare the global features obtained by encoding with other global features obtained in the same way. For example, in some identity authentication scenarios, the encoder in the third classification model can feature encode the user's reference part image (such as face, eyes, lips, etc.) collected in advance to obtain the reference image global features, and store the reference image global features. When the user needs to authenticate the identity, the user's to-be-authenticated part image (such as face, eyes, lips, etc.) can be obtained, and the to-be-authenticated part image can be feature encoded by the encoder in the third classification model to obtain the to-be-authenticated image global features, and then the to-be-authenticated image global features are compared with the reference image global features to obtain the feature comparison results. If the feature comparison result indicates that the similarity between the to-be-authenticated image global features and the reference image global features reaches a certain threshold, it can be determined that the user's identity authentication is passed. On the contrary, if the feature comparison result indicates that the similarity between the to-be-authenticated image global features and the reference image global features does not reach a certain threshold, it can be determined that the user's identity authentication is not passed.

[0073] It can be understood that the number of times incremental learning is performed on the basis of the classification model and the additional classification categories are determined based on actual needs. Each incremental learning process can refer to the above steps, wherein the second classification model is the classification model to be incrementally learned, and the third classification model is the classification model that has completed incremental learning. When incremental learning is required for a classification model, the classification model can be used as the second classification model, and the classification model obtained by incremental training is determined through the above steps. For example, the classification categories corresponding to classification model 1 are categories 0-category 4. If incremental learning is required for classification model 1, classification model 1 can be used as the above second classification model, and through the training of the above steps, classification model 2 can be obtained. The classification categories corresponding to classification model 2 are categories 0-category 5 (that is, category 5 is added on the basis of the original corresponding classification categories). Furthermore, if it is necessary to perform incremental learning on classification model 2, classification model 2 can be used as the second classification model, and through the training of the above steps, classification model 3 can be obtained. The classification categories corresponding to classification model 3 are category 0-category 6 (that is, category 6 is added on the basis of the original corresponding classification categories), and so on. If necessary, classification model 3 can continue to be incrementally learned, which will not be elaborated here.

[0074] See also Figure 2 , Figure 2 1 is a flow chart of a data processing method provided in an embodiment of the present application. The method can be executed by the above-mentioned electronic device. The data processing method can include the following steps.

[0075] S201. Obtain a second classification model, and add a first target category to the classification category corresponding to the second classification model; the second classification model is obtained by training the first classification model based on the first sample data set, and the classification categories corresponding to the second classification model include N initial categories, and the N initial categories are different from the first target category; the first sample data set includes multiple first sample data under the N initial categories; N is a positive integer.

[0076] S202. Obtain the central sample data corresponding to the first sample data under each initial category. Each central sample data refers to the first sample data having the smallest feature distance between the corresponding first global feature and the average global feature of the initial category to which it belongs. The first global feature of each first sample data is obtained by encoding each first sample data by an encoder based on a classification model. The average global feature of each initial category refers to the average feature of the first global features of the corresponding first sample data.

[0077] S203: Acquire a second sample data set, where the second sample data set includes a plurality of second sample data, and each second sample data belongs to the first target category.

[0078] Steps S201-S203 may refer to the relevant description of the above-mentioned steps S101-S103, which will not be repeated here.

[0079] S204, obtaining the central global feature of the central sample data corresponding to each initial category by adding the encoder encoding of the second classification model of the classification category, and performing random sampling processing based on each central global feature to obtain the central sampling feature corresponding to each central global feature.

[0080] Among them, it can be understood that the central global feature can be a feature obtained by feature encoding the central sample data by the encoder in the second classification model that increases the classification category. The central sampling feature corresponding to the central global feature can be a feature obtained by random sampling within a certain fluctuation range based on the central global feature. Optionally, a central sampling feature can be obtained by randomly sampling the central global feature based on a certain feature variance, and the feature variance can be calculated based on the feature variance corresponding to the old task (that is, the task of obtaining the second classification model based on the first classification model training). Optionally, the certain feature variance here can be a preset value, or it can be determined based on the first global feature of each first sample data, and there is no limitation here.

[0081] For example, the central sampling feature may be determined by referring to the following formula (Formula 1).

[0082]

[0083] Among them, oldfm is the central sampling feature of the old task (such as the central sampling feature corresponding to the central global feature of each initial category here), is the central global feature of the old task (such as the central global feature of the central sample data corresponding to each initial category here), e~N(0,1) is the derived Gaussian noise with the same dimension as the prototype, and r is the average feature variance of the old task (such as the target average feature variance here).

[0084] Specifically, performing random sampling processing based on each central global feature to obtain the central sampling feature corresponding to each central global feature can include the following steps: obtaining the target average feature variance corresponding to N initial categories; the target average feature variance is the average feature variance corresponding to the first global feature of the first sample data under the N initial categories; performing random sampling processing on each central global feature based on the target average feature variance to obtain the central sampling feature of each central global feature; the number of central sampling features of each central global feature is one or more.

[0085] It can be understood that the method of performing random sampling processing on each central global feature based on the target average feature variance to obtain the central sampling feature of each central global feature can refer to the above description and will not be repeated here. That is to say, the above-mentioned certain feature variance can also be the target average feature variance.

[0086] Among them, the first average feature variance can also be called the average feature variance of the first global feature under multiple classification categories (i.e., N initial categories) corresponding to the above-mentioned old task (such as the task of obtaining the second classification model based on the first classification model training).

[0087] It can be understood that if the second classification model can be obtained by fully training the basic classification model, then the N initial categories can be the basic categories corresponding to the basic classification model, and the method for determining the target average feature variance corresponding to the N initial categories can include: determining the average feature variance corresponding to the N initial categories based on the first global feature of the first sample data under each basic category, and determining the average feature variance corresponding to the N initial categories as the target average feature variance.

[0088] It can be understood that if the second classification model is obtained through one or more incremental learning based on the trained basic classification model, the target average feature variance can be determined based on the average feature variance of the first global feature of the first sample data under the basic category corresponding to the basic classification model, and the average feature variance of the first global feature of the first sample data under the target category added when incremental learning is performed on the basic classification model after full training.

[0089] In one embodiment, the first classification model is obtained by training the basic classification model through the basic sample data set, and the second classification model is obtained by training the first classification model with the second target category added to the classification category; the classification categories corresponding to the basic classification model include M basic categories, and the N initial categories include M basic categories and the second target category, and M is a positive integer less than N; the sample data in the basic sample data set and the sample data used to train the first classification model with the added classification category are all first sample data; then, obtaining the target average feature variance corresponding to the N initial categories may include the following steps: determining the average feature variance corresponding to the M basic categories based on the first global feature of the first sample data under each basic category, and determining the average feature variance corresponding to the N initial categories based on the first global feature of the first sample data under each initial category; determining the target average feature variance based on the average feature variance corresponding to the M basic categories and the average feature variance corresponding to the N initial categories.

[0090] It is understandable that when calculating the target average feature variance, the average feature variance for each classification category during each training can be combined, thereby retaining more knowledge of old tasks while learning knowledge of newer tasks, thereby improving the model performance of the trained classification model. It is understandable that the first classification model is obtained by fully training the basic classification model, or it can be obtained by one or more incremental learning based on the trained basic classification model, which is not limited here.

[0091] It is understandable that the second target category may be a target category that is required to be added for incremental learning of the first classification model. The first classification model is obtained by training the basic classification model through the basic sample data set, which may mean that the first classification model may be obtained by training the basic classification model in full through the basic sample data set, or may be obtained by incremental learning of the basic classification model after full training, which is not limited here.

[0092] It can be understood that the average feature variance corresponding to the M basic categories can be understood as the average of the feature variances corresponding to each basic category; the average feature variance corresponding to the N initial categories can be understood as the average of the feature variances of each initial category.

[0093] The target average feature variance is determined based on the average feature variances corresponding to the M basic categories and the average feature variances corresponding to the N initial categories. The target average feature variance can be obtained by averaging the average feature variances corresponding to the M basic categories and the average feature variances corresponding to the N initial categories.

[0094] For example, a method for determining the central sampling feature may be represented by referring to the following formula (Formula 2).

[0095]

[0096] in, To obtain the average feature variance of each classification category when training the t-th task, it can be understood that when performing incremental learning on the t+1-th task, the target average feature variance to be obtained can be the average feature variance of each classification category at the t-th task. For example, when performing incremental training on the second classification model, the average feature variance to be obtained is the average feature variance of the global features of the sample data under each classification category when the first classification model is trained. It can be understood that the task refers to the training based on the basic classification model. Training the basic classification model can be the 0th task, adding a classification category to the trained basic classification model for incremental training can be the 1st task, and so on. Adding another classification category to the classification model after an incremental training for incremental training can be the 2nd task. cls refers to the number of classification categories corresponding to the classification model. It can be understood that cls old It can be the number of classification categories of the old task (i.e., the t-1th task) when training the tth task, cls new It can be the number of classification categories of the new task when training the tth task. For example, incremental learning of the second classification model is the third task, and training the first classification model to obtain the second classification model is the second task. Then, when performing incremental training on the second classification model (the third task), the target average feature variance to be obtained is the average feature variance corresponding to each classification category when the first classification model is trained to obtain the second classification model (i.e., the second task). The target average feature variance to be obtained when performing incremental learning on the third task can be the average feature variance of each classification category in the second task. Specifically, it can be determined based on the average feature variance of each classification category in the first task (i.e., the old task of the second task) and the average feature variance of each classification category in the second task. Then, cls old is the number of classification categories for the first task, cls new is the number of classification categories for the second task. Where D is the size of the feature dimension, (∑t,cls) is the covariance matrix of the classification categories corresponding to the new task for the tth task, and Tr is used to calculate the trace of a matrix (the sum of the diagonal elements of the matrix).

[0097] S205 , obtaining a second global feature of each second sample data by encoding the encoder of the second classification model adding the classification category.

[0098] The second global feature may be a feature encoded by the first sample data.

[0099] S206: Determine the central sampling feature corresponding to each central global feature and the second global feature of each second sample data as target training features for the second classification model with added classification categories.

[0100] It can be understood that the target training feature may be a feature used to train a second classification model with an added classification category.

[0101] S207, performing classification and recognition processing on each target training feature respectively by adding a second classification model of the classification category, and obtaining a classification and recognition result corresponding to each target training feature.

[0102] It can be understood that the classification recognition result can be used to indicate the classification category corresponding to the target training feature. The classification recognition result can include recognition probability information for each classification category (i.e., N initial categories and the first target category) corresponding to the second classification model with added classification categories.

[0103] S208. Iteratively train the second classification model with added classification categories based on the classification recognition results corresponding to each target training feature to obtain a third classification model, and use the third classification model to classify and recognize the input data; the classification categories corresponding to the third classification model include N initial categories and the first target category.

[0104] Among them, based on the classification recognition results corresponding to each target training feature, the second classification model with added classification categories is iteratively trained to obtain the third classification model. The second classification model with added classification categories can be iteratively trained through the classification recognition results corresponding to each target training feature and the sample labels associated with the sample data corresponding to each target training feature to obtain the third classification model.

[0105] Specifically, the classification recognition result corresponding to any target training feature includes recognition probability information for N initial categories and the first target category; each central sample data is associated with a sample label for indicating the initial category to which it belongs, and each second sample data is associated with a sample label for indicating that it belongs to the first target category; then, based on the classification recognition result corresponding to each target training feature, the second classification model with added classification categories is iteratively trained to obtain a third classification model, which may include the following steps: based on the recognition probability information for N initial categories and the first target category in the classification recognition result corresponding to each target training feature, the first reference information is determined based on the difference between the sample labels associated with the sample data corresponding to each target training feature; based on the first reference information, the second classification model with added classification categories is iteratively trained to obtain a third classification model.

[0106] See also Figure 3 , Figure 3 1 is a flow chart of a data processing method provided in an embodiment of the present application. The method can be executed by the above-mentioned electronic device. The data processing method can include the following steps.

[0107] S301. Obtain a second classification model, and add a first target category to the classification category corresponding to the second classification model; the second classification model is obtained by training the first classification model based on the first sample data set, and the classification categories corresponding to the second classification model include N initial categories, and the N initial categories are different from the first target category; the first sample data set includes multiple first sample data under the N initial categories; N is a positive integer.

[0108] Among them, it can be understood that each classification model (such as the first classification model, the second classification model, etc.) can include the above-mentioned encoder and classification layer, and can also include a decoder, so that in the process of training the classification model, the decoder can be used as an auxiliary training, and the decoder and the classification layer share the same encoder. While training the classification recognition training task, the encoding and decoding training task is performed, so that the global features encoded by the encoder of the trained classification model have not only classification capabilities, but also global modeling capabilities, so that the encoder's ability to encode features will not disappear with the reduction of old task data in the incremental learning process, thereby improving the encoding ability of the encoder in the classification model. It can be understood that when training the classification model based on decoded data and sample data, it does not rely on the sample labels associated with the sample data, and the training of the classification model can be called a self-supervised training method.

[0109] For example, see Figure 4 , Figure 4 Schematic diagram of the effect of the model structure of a classification model provided in the embodiment of the present application. Figure 4As shown, the classification model may include an encoder 401, a decoder 402, and a classification layer 403. The decoder 402 and the classification layer 403 share the same encoder 401, that is, based on the encoder 401, feature encoding can be performed on the input sample data to obtain global features corresponding to the input sample data, and then the encoder 401 decodes the global features to obtain decoded data, and while the classification model is trained based on the decoded data and the sample data, the global features can be classified and identified through the classification layer to obtain corresponding classification identification results, and the classification model is trained based on the classification identification results and the corresponding sample labels.

[0110] S302, obtaining the central sample data corresponding to the first sample data under each initial category, each central sample data refers to the first sample data having the smallest feature distance between the corresponding first global feature and the average global feature of the initial category to which it belongs, the first global feature of each first sample data is obtained by encoding each first sample data by an encoder based on a classification model, and the average global feature of each initial category refers to the average feature of the first global features of the corresponding first sample data.

[0111] S303: Acquire a second sample data set, where the second sample data set includes a plurality of second sample data, and each second sample data belongs to the first target category.

[0112] It can be understood that steps S302 to S303 can refer to the above steps S102 to S103, which will not be described in detail here.

[0113] S304, performing feature encoding processing on the central sample data and each second sample data corresponding to each initial category by using the encoder in the second classification model with the added classification category, so as to obtain target training features for the second classification model with the added classification category.

[0114] It can be understood that step S304 may include the following steps: obtaining the central global feature of the central sample data corresponding to each initial category by encoding the encoder of the second classification model adding the classification category, and performing random sampling processing based on each central global feature to obtain the central sampling feature corresponding to each central global feature; obtaining the second global feature of each second sample data by encoding the encoder of the second classification model adding the classification category; determining the central sampling feature corresponding to each central global feature and the second global feature of each second sample data as the target training feature for the second classification model adding the classification category.

[0115] It can be understood that the relevant description of this step can refer to the above-mentioned relevant description, and will not be repeated here.

[0116] S305. Perform classification and recognition processing on each target training feature by adding a classification layer in the second classification model of the classification category to obtain a classification and recognition result corresponding to each target training feature, and determine the first reference information based on the difference between the classification and recognition result corresponding to each target training feature and the sample label associated with the sample data corresponding to each target training feature.

[0117] The method for determining the first reference information may refer to the above related description.

[0118] Specifically, the classification recognition result corresponding to any target training feature includes recognition probability information for N initial categories and the first target category; each central sample data is associated with a sample label for indicating the initial category to which it belongs, and each second sample data is associated with a sample label for indicating that it belongs to the first target category; then, determining the first reference information based on the difference between the classification recognition result corresponding to each target training feature and the sample label associated with the sample data corresponding to each target training feature may include the following steps: determining the first reference information based on the recognition probability information for N initial categories and the first target category in the classification recognition result corresponding to each target training feature, and the difference between the sample label associated with the sample data corresponding to each target training feature.

[0119] Referring to the above description, the first reference information can be calculated based on a loss function, such as a cross entropy loss function. It can be understood that as the training progresses, the classification recognition result corresponding to each training feature gradually approaches the sample label associated with the sample data corresponding to each target training feature, that is, the first reference information gradually decreases. When the first reference information converges, the second classification model with increased classification categories in iterative training can be determined as the third classification model.

[0120] S306. Decode the central global feature of each central sample data and the second global feature of each second sample data by adding a decoder in the second classification model of the classification category to obtain decoded data of each central sample data and each second sample data, and determine the second reference information based on the similarity between each decoded data obtained by decoding and the corresponding sample data.

[0121] It can be understood that the decoded data can be data obtained by decoding the global features obtained based on the encoder (i.e., the central global features and the second global features here) through the decoder, which is equivalent to restoring the global features obtained based on the encoder to the data type corresponding to the sample data through the decoder, and the data type can be an image data type, a text data type, etc. For example, if the sample data is image data, then decoding the global features obtained based on the encoder can be used to restore the target training features to image data.

[0122] It is understandable that the second reference information can be information for training a second classification model for adding classification categories, and can also be called a loss value. The second reference information can be determined based on a loss function, such as MSEloss (a loss function). It is understandable that the greater the similarity between the decoded data and the corresponding sample data, the smaller the value of the second reference information, and the smaller the similarity between the decoded data and the corresponding sample data, the larger the value of the second reference information.

[0123] S307, iteratively train the second classification model with added classification categories based on the first reference information and the second reference information to obtain a third classification model, and use the third classification model to classify and identify the input data; the classification categories corresponding to the third classification model include N initial categories and the first target category.

[0124] It can be understood that the second classification model with the added classification category is iteratively trained based on the first reference information and the second reference information to obtain the third classification model, which can be iteratively trained based on the first reference information and the second reference information at the same time. As the model training proceeds, the first reference information and the second reference information obtained are gradually reduced, that is, the classification recognition result corresponding to each target training feature gradually approaches the sample label associated with the sample data corresponding to each target training feature, and the closer the decoded data corresponding to each target training feature is to the sample data corresponding to each target training feature (that is, the greater the similarity), until the first reference information and the second reference information converge.

[0125] For example, see Figure 5 , Figure 5 Schematic diagram of a classification model provided in an embodiment of the present application. Figure 5 As shown, based on the above description, Figure 5The classification model shown may include an encoder 501, a decoder 502, and a classification layer 503. In a process of incremental learning based on a classification model, a second sample data set 504 consisting of second sample data belonging to an added target category and a central sample data 505 corresponding to each initial category may be obtained. Then, feature encoding processing may be performed on each central sample data based on the encoder 501 to obtain a central global feature 506 corresponding to each central sample data, and random sampling processing may be performed on the central global feature based on the corresponding feature variance to obtain a central sampling feature 507 corresponding to each central global feature, and feature encoding processing may be performed on each second sample data based on the encoder 501 to obtain a second global feature 508 corresponding to each second sample data. Then, the classification layer 503 may be used to perform classification and recognition processing on the central sampling feature 507 and the second global feature 508 corresponding to each second sample data to obtain a corresponding classification and recognition result, and the corresponding first reference information may be determined based on the classification and recognition result of each feature (i.e., the central sampling feature 507 and the second global feature 508) and the difference between the corresponding sample data associated sample labels. At the same time, based on the decoder 502, the central global feature 506 corresponding to each central sample data and the second global feature 508 corresponding to each second sample data can be decoded to obtain the corresponding decoded data, and the second reference information can be determined based on the similarity between the decoded data and the corresponding sample data. Then, iterative training can be performed based on the first reference information and the second reference information to obtain a classification model after incremental learning, and the classification model after incremental learning can perform classification and recognition based on the initial category and the target category. It can be understood that when training the classification model, the central sample data 509 corresponding to the target category can be determined by the average feature of the global features of each sample data under the target category during the last iterative training, and the central sample data corresponding to the target category can be saved, so that new incremental learning tasks can be performed based on the central sample data of the target category and the central sample data of the initial category saved previously.

[0126] For example, see Figure 6 , Figure 6 is a flow chart of an incremental learning method provided in an embodiment of the present application. Figure 6As shown, first, the basic classification model can be trained to obtain classification model 1 (step S601). If the classification category corresponding to the basic classification model can be category 0-4, the basic classification model is fully trained through the sample data under category 0-4, and the trained classification model can be classification model 1. In addition, in the process of training the basic classification model, the global features of each sample data under category 0-4 during the last iterative training can be recorded, and the average global features corresponding to each category can be determined by the average features of the global features of the sample data under each category in category 0-4, and then the global features of the sample data under each category are traversed, and the sample data with the smallest distance (such as Euclidean distance) between the global features and the average global features under each category is selected as the central sample data of each category. Then, after the classification model 1 is obtained through training, incremental learning can be performed on the basis of the classification model 1, that is, the target category can be added to the classification category corresponding to the classification model 1 (step S602). Specifically, the classification category can be added to the classification layer in the classification model 1. For example, the classification category corresponding to the classification model 1 is category 0-4 (i.e., 5 classification categories). After category 5 is added to the classification category, the dimension of the corresponding classification category is 6. It can be understood that the encoder and decoder of the classification model can load the model parameters of the classification model 1.

[0127] Further, when incremental learning is performed on the classification model 1, sample data belonging to the newly added target category (such as the second sample data described above) can be obtained, and the central sample data corresponding to each classification category of the classification model 1 can be obtained (step S603). The number of central sample data corresponding to each classification category can be one. Then, feature encoding can be performed on each central sample data by an encoder to obtain the central global feature corresponding to each central sample data, and the central global feature can be randomly sampled based on the feature variance to obtain the central sampling feature corresponding to each central global feature (step S604); feature encoding can be performed on the sample data corresponding to the target category by an encoder to obtain the global feature of each sample data corresponding to the target category (step S605); then the central sampling feature corresponding to each central global feature and the global feature of each sample data corresponding to the target category can be input into the classification layer for training (step S606), and specifically, the first reference information can be calculated by the central sampling feature corresponding to each central global feature, the global feature of each sample data corresponding to the target category, and the sample label of the corresponding sample data, and then the classification model 1 with the added classification category can be trained based on the first reference information, and the first reference information can be determined based on the cross entropy loss function. At the same time, the central global features corresponding to each central sample data and the global features corresponding to each sample data of the target category can be input into the decoder for training (step S607). Specifically, the decoder can determine the decoded data corresponding to the central global features corresponding to each central sample data and the global features corresponding to each sample data of the target category, and then determine the second reference information based on the similarity between the decoded data and the corresponding sample data, and then train the classification model 1 with the added classification category based on the second reference information. The second reference information can be determined based on MSEloss (a loss function). Incremental learning of classification model 1 can be achieved in this way. The classification model obtained after incremental learning of classification model 1 can be classification model 2. If incremental learning of classification model 2 is required, steps S602-S607 can be repeated. For example, if the classification model obtained by the final training is required to be able to classify and identify categories 0-9, and categories 0-4 are the classification categories corresponding to the basic classification model, then one category can be added to the incremental training each time until the classification category corresponding to the trained classification model is category 0-9.

[0128] See also Figure 7 , Figure 7 is a structural diagram of a data processing device provided in an embodiment of the present application. Optionally, the data processing device may be provided in the above-mentioned electronic device. Figure 7 As shown, the data processing device described in this embodiment may include:

[0129] The acquisition unit 701 is used to acquire a second classification model, and add a first target category to the classification category corresponding to the second classification model; the second classification model is obtained by training the first classification model based on the first sample data set, and the classification category corresponding to the second classification model includes N initial categories, and the N initial categories are different from the first target category; the first sample data set includes a plurality of first sample data under the N initial categories; N is a positive integer;

[0130] The acquisition unit 701 is further used to acquire the central sample data corresponding to the first sample data under each initial category, each central sample data refers to the first sample data having the smallest feature distance between the corresponding first global feature and the average global feature of the initial category to which it belongs, the first global feature of each first sample data is obtained by encoding the first sample data by an encoder based on a classification model, and the average global feature of each initial category refers to the average feature of the first global features of the corresponding plurality of first sample data;

[0131] The acquisition unit 701 is further configured to acquire a second sample data set, where the second sample data set includes a plurality of second sample data, each of which belongs to the first target category;

[0132] Processing unit 702 is used to iteratively train the second classification model with added classification categories based on the multiple second sample data and the central sample data corresponding to the N initial categories to obtain a third classification model, and use the third classification model to classify and identify the input data; the classification categories corresponding to the third classification model include the N initial categories and the first target category.

[0133] In one implementation, the processing unit 702 is specifically configured to:

[0134] The central global feature of the central sample data corresponding to each initial category is obtained by encoding the encoder of the second classification model by adding the classification category, and a random sampling process is performed based on each central global feature to obtain the central sampling feature corresponding to each central global feature;

[0135] Obtaining the second global feature of each second sample data by adding an encoder encoding of a second classification model of a classification category;

[0136] Determine the central sampling feature corresponding to each central global feature and the second global feature of each second sample data as target training features for a second classification model with increased classification categories;

[0137] By adding a second classification model of the classification category, each target training feature is classified and recognized respectively, so as to obtain a classification recognition result corresponding to each target training feature;

[0138] The second classification model with added classification categories is iteratively trained based on the classification recognition results corresponding to each target training feature to obtain a third classification model.

[0139] In one implementation, the processing unit 702 is specifically configured to:

[0140] Obtaining target average feature variances corresponding to the N initial categories; the target average feature variance is an average feature variance corresponding to the first global feature of the first sample data under the N initial categories;

[0141] Each central global feature is randomly sampled based on the target average feature variance to obtain a central sampling feature of each central global feature; the number of central sampling features of each central global feature is one or more.

[0142] In one implementation, the first classification model is obtained by training a basic classification model through a basic sample data set, and the second classification model is obtained by training the first classification model with a second target category added to the classification category; the classification category corresponding to the basic classification model includes M basic categories, the N initial categories include the M basic categories and the second target category, and M is a positive integer less than N; the sample data in the basic sample data set and the sample data used to train the first classification model with the added classification category are both the first sample data; the processing unit 702 is specifically used to:

[0143] Determine the average feature variance corresponding to the M basic categories based on the first global feature of the first sample data under each basic category, and determine the average feature variance corresponding to the N initial categories based on the first global feature of the first sample data under each initial category;

[0144] The target average feature variance is determined based on the average feature variances corresponding to the M basic categories and the average feature variances corresponding to the N initial categories.

[0145] In one implementation, the classification recognition result corresponding to any target training feature includes recognition probability information for the N initial categories and the first target category; each of the central sample data is associated with a sample label indicating the initial category to which it belongs, and each of the second sample data is associated with a sample label indicating that it belongs to the first target category; the processing unit 702 is specifically used to:

[0146] Determine first reference information based on the recognition probability information for the N initial categories and the first target category in the classification recognition result corresponding to each target training feature, and the difference between the sample labels associated with the sample data corresponding to each target training feature;

[0147] The second classification model with added classification categories is iteratively trained based on the first reference information to obtain the third classification model.

[0148] In one implementation, the second classification model further includes a decoder and a classification layer; each of the central sample data is associated with a sample label indicating the initial category to which it belongs, and each of the second sample data is associated with a sample label indicating that it belongs to the first target category; the processing unit 702 is specifically configured to:

[0149] Performing feature encoding processing on the central sample data corresponding to each initial category and each second sample data by an encoder in the second classification model with added classification categories, thereby obtaining target training features for the second classification model with added classification categories;

[0150] Performing classification and recognition processing on each target training feature by adding a classification layer in a second classification model of a classification category to obtain a classification and recognition result corresponding to each target training feature, and determining first reference information based on a difference between the classification and recognition result corresponding to each target training feature and a sample label associated with sample data corresponding to each target training feature;

[0151] Decoding the central global feature of each central sample data and the second global feature of each second sample data by adding a decoder in the second classification model of the classification category to obtain decoded data of each central sample data and each second sample data, and determining second reference information based on similarity between each decoded data obtained by decoding and the corresponding sample data;

[0152] The second classification model with added classification categories is iteratively trained based on the first reference information and the second reference information to obtain the third classification model.

[0153] In one implementation, the processing unit 702 is specifically configured to:

[0154] Acquire the first global feature of the first sample data under each of the initial categories, and determine the average global feature of each of the initial categories based on the average feature of the first global feature of the first sample data under each of the initial categories;

[0155] Calculating a feature distance between the first global feature of each first sample data and an average global feature of the initial category to which it belongs;

[0156] The first sample data with the smallest corresponding feature distance in the first sample data under each initial category is determined as the central sample data corresponding to each initial category.

[0157] In one implementation, the first sample data and the second sample data are both image data of a key part of an object; the third classification model includes an encoder and a classification layer, and the classification category corresponding to the third classification model includes the part category of the key part to which the input image data belongs; the processing unit 702 is specifically configured to:

[0158] Acquire image data to be processed, and obtain image global features of the image data to be processed based on encoder encoding in the third classification model;

[0159] The global features of the image are classified and identified through the classification layer in the third classification model to obtain a part identification result corresponding to the image data to be processed, and the part identification result is used to indicate the part category of the key part to which the image data to be processed belongs.

[0160] See also Figure 8 , Figure 8 801 and 802. The electronic device described in this embodiment includes: a processor 801 and a memory 802. Optionally, the electronic device may also include a network interface or a power supply module. The processor 801 and the memory 802 may exchange data.

[0161] The processor 801 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0162] The above-mentioned network interface may include an input device and / or an output device. For example, the input device may be a control panel, a microphone, a receiver, etc., and the output device may be a display screen, a transmitter, etc., which are not listed here one by one.

[0163] The memory 802 may include a read-only memory and a random access memory, and provide program instructions and data to the processor 801. A portion of the memory 802 may also include a non-volatile random access memory. When the processor 801 calls the program instructions, it is used to execute:

[0164] Obtain a second classification model, and add a first target category to the classification category corresponding to the second classification model; the second classification model is obtained by training the first classification model based on the first sample data set, the classification category corresponding to the second classification model includes N initial categories, and the N initial categories are different from the first target category; the first sample data set includes a plurality of first sample data under the N initial categories; N is a positive integer;

[0165] Obtaining central sample data corresponding to the first sample data under each initial category, each central sample data refers to the first sample data having the smallest feature distance between the corresponding first global feature and the average global feature of the initial category to which it belongs, the first global feature of each first sample data being obtained by encoding the first sample data by an encoder based on a classification model, and the average global feature of each initial category being an average feature of the first global features of the corresponding plurality of first sample data;

[0166] Acquire a second sample data set, where the second sample data set includes a plurality of second sample data, and each second sample data belongs to the first target category;

[0167] Based on the multiple second sample data and the central sample data corresponding to the N initial categories, the second classification model with added classification categories is iteratively trained to obtain a third classification model, and the third classification model is used to classify and identify the input data; the classification categories corresponding to the third classification model include the N initial categories and the first target category.

[0168] In one implementation, the processor 801 is specifically configured to:

[0169] The central global feature of the central sample data corresponding to each initial category is obtained by encoding the encoder of the second classification model by adding the classification category, and a random sampling process is performed based on each central global feature to obtain the central sampling feature corresponding to each central global feature;

[0170] Obtaining the second global feature of each second sample data by adding an encoder encoding of a second classification model of a classification category;

[0171] Determine the central sampling feature corresponding to each central global feature and the second global feature of each second sample data as target training features for a second classification model with increased classification categories;

[0172] By adding a second classification model of the classification category, each target training feature is classified and recognized respectively, so as to obtain a classification recognition result corresponding to each target training feature;

[0173] The second classification model with added classification categories is iteratively trained based on the classification recognition results corresponding to each target training feature to obtain a third classification model.

[0174] In one implementation, the processor 801 is specifically configured to:

[0175] Obtaining target average feature variances corresponding to the N initial categories; the target average feature variance is an average feature variance corresponding to the first global feature of the first sample data under the N initial categories;

[0176] Each central global feature is randomly sampled based on the target average feature variance to obtain a central sampling feature of each central global feature; the number of central sampling features of each central global feature is one or more.

[0177] In one implementation, the first classification model is obtained by training a basic classification model through a basic sample data set, and the second classification model is obtained by training the first classification model with a second target category added to the classification category; the classification category corresponding to the basic classification model includes M basic categories, the N initial categories include the M basic categories and the second target category, and M is a positive integer less than N; the sample data in the basic sample data set and the sample data used to train the first classification model with the added classification category are both the first sample data; the processor 801 is specifically used to:

[0178] Determine the average feature variance corresponding to the M basic categories based on the first global feature of the first sample data under each basic category, and determine the average feature variance corresponding to the N initial categories based on the first global feature of the first sample data under each initial category;

[0179] The target average feature variance is determined based on the average feature variances corresponding to the M basic categories and the average feature variances corresponding to the N initial categories.

[0180] In one implementation, the classification recognition result corresponding to any target training feature includes recognition probability information for the N initial categories and the first target category; each of the central sample data is associated with a sample label indicating the initial category to which it belongs, and each of the second sample data is associated with a sample label indicating that it belongs to the first target category; the processor 801 is specifically configured to:

[0181] Determine first reference information based on the recognition probability information for the N initial categories and the first target category in the classification recognition result corresponding to each target training feature, and the difference between the sample labels associated with the sample data corresponding to each target training feature;

[0182] The second classification model with added classification categories is iteratively trained based on the first reference information to obtain the third classification model.

[0183] In one implementation, the second classification model further includes a decoder and a classification layer; each of the center sample data is associated with a sample label indicating the initial category to which it belongs, and each of the second sample data is associated with a sample label indicating that it belongs to the first target category; the processor 801 is specifically configured to:

[0184] Performing feature encoding processing on the central sample data corresponding to each initial category and each second sample data by an encoder in the second classification model with added classification categories, thereby obtaining target training features for the second classification model with added classification categories;

[0185] Performing classification and recognition processing on each target training feature by adding a classification layer in a second classification model of a classification category to obtain a classification and recognition result corresponding to each target training feature, and determining first reference information based on a difference between the classification and recognition result corresponding to each target training feature and a sample label associated with sample data corresponding to each target training feature;

[0186] Decoding the central global feature of each central sample data and the second global feature of each second sample data by adding a decoder in the second classification model of the classification category to obtain decoded data of each central sample data and each second sample data, and determining second reference information based on similarity between each decoded data obtained by decoding and the corresponding sample data;

[0187] The second classification model with added classification categories is iteratively trained based on the first reference information and the second reference information to obtain the third classification model.

[0188] In one implementation, the processor 801 is specifically configured to:

[0189] Acquire the first global feature of the first sample data under each of the initial categories, and determine the average global feature of each of the initial categories based on the average feature of the first global feature of the first sample data under each of the initial categories;

[0190] Calculating a feature distance between the first global feature of each first sample data and an average global feature of the initial category to which it belongs;

[0191] The first sample data with the smallest corresponding feature distance in the first sample data under each initial category is determined as the central sample data corresponding to each initial category.

[0192] In one implementation, the first sample data and the second sample data are both image data of a key part of an object; the third classification model includes an encoder and a classification layer, and the classification category corresponding to the third classification model includes the part category of the key part to which the input image data belongs; the processor 801 is specifically configured to:

[0193] Acquire image data to be processed, and obtain image global features of the image data to be processed based on encoder encoding in the third classification model;

[0194] The global features of the image are classified and identified through the classification layer in the third classification model to obtain a part identification result corresponding to the image data to be processed, and the part identification result is used to indicate the part category of the key part to which the image data to be processed belongs.

[0195] Optionally, when the program instructions are executed by the processor, other steps of the method in the above embodiment may also be implemented, which will not be repeated here.

[0196] The present application also provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the above method, such as executing the method executed by the above electronic device, which is not repeated here.

[0197] Optionally, the storage medium involved in the present application, such as computer-readable storage medium, may be non-volatile or volatile.

[0198] Optionally, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of blockchain nodes, etc. Among them, the blockchain referred to in this application is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods, each of which contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. The blockchain may include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0199] It should be noted that, for the above-mentioned various method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0200] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0201] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction, and when the computer instruction is executed by a processor, some or all of the steps in the above method can be implemented. For example, the computer instruction is stored in a computer-readable storage medium. The processor of the computer device (i.e., the above-mentioned electronic device) reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction so that the computer device executes the steps performed in the embodiments of the above-mentioned methods. For example, the computer device can be a terminal, or it can be a server.

[0202] The above is a detailed introduction to a data processing method, device, electronic device and medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Obtain a second classification model, and add a first target category to the classification category corresponding to the second classification model; the second classification model is obtained by training the first classification model based on the first sample data set, the classification category corresponding to the second classification model includes N initial categories, and the N initial categories are different from the first target category; the first sample data set includes a plurality of first sample data under the N initial categories; N is a positive integer; Obtaining central sample data corresponding to the first sample data under each initial category, each central sample data refers to the first sample data having the smallest feature distance between the corresponding first global feature and the average global feature of the initial category to which it belongs, the first global feature of each first sample data being obtained by encoding the first sample data by an encoder based on a classification model, and the average global feature of each initial category being an average feature of the first global features of the corresponding plurality of first sample data; Acquire a second sample data set, where the second sample data set includes a plurality of second sample data, and each second sample data belongs to the first target category; Iteratively training the second classification model with added classification categories based on the plurality of second sample data and the central sample data corresponding to the N initial categories to obtain a third classification model, and classifying and identifying the input image data using the third classification model; the classification categories corresponding to the third classification model include the N initial categories and the first target category; The third classification model is obtained by iteratively training the second classification model using the target training features obtained by using the multiple second sample data and the central sample data corresponding to the N initial categories. The process of obtaining the target training features includes: obtaining the central global features of the central sample data corresponding to each initial category and the second global features of each second sample data by encoding the encoder of the second classification model with added classification categories; obtaining the target average feature variance corresponding to the N initial categories; the target average feature variance is the average feature variance corresponding to the first global features of the first sample data under the N initial categories; randomly sampling each central global feature based on the target average feature variance to obtain the central sampling feature of each central global feature; the number of central sampling features of each central global feature is one or more; determining the central sampling feature corresponding to each central global feature and the second global feature of each second sample data as the target training features for the second classification model with added classification categories.

2. The method according to claim 1, characterized in that The iterative training of the second classification model with added classification categories based on the plurality of second sample data and the central sample data corresponding to the N initial categories to obtain a third classification model includes: Obtaining target training features for a second classification model with increased classification categories based on the plurality of second sample data and the central sample data corresponding to the N initial categories; By adding a second classification model of the classification category, each target training feature is classified and recognized respectively, so as to obtain a classification recognition result corresponding to each target training feature; The second classification model with added classification categories is iteratively trained based on the classification recognition results corresponding to each target training feature to obtain a third classification model.

3. The method according to claim 1, characterized in that The first classification model is obtained by training a basic classification model through a basic sample data set, and the second classification model is obtained by training the first classification model with a second target category added to the classification category; the classification category corresponding to the basic classification model includes M basic categories, the N initial categories include the M basic categories and the second target category, and M is a positive integer less than N; the sample data in the basic sample data set and the sample data used to train the first classification model with the added classification category are all the first sample data; The obtaining of the target average feature variance corresponding to the N initial categories includes: Determine the average feature variance corresponding to the M basic categories based on the first global feature of the first sample data under each basic category, and determine the average feature variance corresponding to the N initial categories based on the first global feature of the first sample data under each initial category; The target average feature variance is determined based on the average feature variances corresponding to the M basic categories and the average feature variances corresponding to the N initial categories.

4. The method according to claim 2, characterized in that: The classification recognition result corresponding to any target training feature includes recognition probability information for the N initial categories and the first target category; each of the central sample data is associated with a sample label indicating the initial category to which it belongs, and each of the second sample data is associated with a sample label indicating that it belongs to the first target category; The iterative training of the second classification model with added classification categories based on the classification recognition results corresponding to each target training feature to obtain a third classification model includes: Determine first reference information based on the recognition probability information for the N initial categories and the first target category in the classification recognition result corresponding to each target training feature, and the difference between the sample labels associated with the sample data corresponding to each target training feature; The second classification model with added classification categories is iteratively trained based on the first reference information to obtain the third classification model.

5. The method according to claim 2, characterized in that: The second classification model also includes a decoder and a classification layer; each of the central sample data is associated with a sample label indicating the initial category to which it belongs, and each of the second sample data is associated with a sample label indicating that it belongs to the first target category; The iterative training of the second classification model with added classification categories based on the classification recognition results corresponding to each target training feature to obtain a third classification model includes: Determine first reference information based on a difference between a classification recognition result corresponding to each target training feature and a sample label associated with sample data corresponding to each target training feature; Decoding the central global feature of each central sample data and the second global feature of each second sample data by adding a decoder in the second classification model of the classification category to obtain decoded data of each central sample data and each second sample data, and determining second reference information based on similarity between each decoded data obtained by decoding and the corresponding sample data; The second classification model with added classification categories is iteratively trained based on the first reference information and the second reference information to obtain the third classification model.

6. The method according to claim 1, characterized in that The obtaining of central sample data corresponding to the first sample data under each initial category includes: Acquire the first global feature of the first sample data under each of the initial categories, and determine the average global feature of each of the initial categories based on the average feature of the first global feature of the first sample data under each of the initial categories; Calculating a feature distance between the first global feature of each first sample data and an average global feature of the initial category to which it belongs; The first sample data with the smallest corresponding feature distance in the first sample data under each initial category is determined as the central sample data corresponding to each initial category.

7. The method according to claim 1, characterized in that The first sample data and the second sample data are both image data of key parts of the object; the third classification model includes an encoder and a classification layer, and the classification category corresponding to the third classification model includes the part category of the key part to which the input image data belongs; The using the third classification model to classify and identify the input data includes: Acquire image data to be processed, and obtain image global features of the image data to be processed based on encoder encoding in the third classification model; The global features of the image are classified and identified through the classification layer in the third classification model to obtain a part identification result corresponding to the image data to be processed, and the part identification result is used to indicate the part category of the key part to which the image data to be processed belongs.

8. A data processing device, characterized in that: The device comprises: an acquisition unit, configured to acquire a second classification model, and add a first target category to the classification category corresponding to the second classification model; the second classification model is obtained by training the first classification model based on the first sample data set, the classification category corresponding to the second classification model includes N initial categories, and the N initial categories are different from the first target category; the first sample data set includes a plurality of first sample data under the N initial categories; N is a positive integer; The acquisition unit is further used to acquire the central sample data corresponding to the first sample data under each initial category, each central sample data refers to the first sample data having the smallest feature distance between the corresponding first global feature and the average global feature of the initial category to which it belongs, the first global feature of each first sample data is obtained by encoding the first sample data by an encoder based on a classification model, and the average global feature of each initial category refers to the average feature of the first global features of the corresponding plurality of first sample data; The acquisition unit is further used to acquire a second sample data set, where the second sample data set includes a plurality of second sample data, and each second sample data belongs to the first target category; a processing unit, configured to iteratively train the second classification model with added classification categories based on the plurality of second sample data and the central sample data corresponding to the N initial categories to obtain a third classification model, and classify and identify the input image data using the third classification model; the classification categories corresponding to the third classification model include the N initial categories and the first target category; The third classification model is obtained by iteratively training the second classification model using the target training features obtained by using the multiple second sample data and the central sample data corresponding to the N initial categories. The process of obtaining the target training features includes: obtaining the central global features of the central sample data corresponding to each initial category and the second global features of each second sample data by encoding the encoder of the second classification model with added classification categories; obtaining the target average feature variance corresponding to the N initial categories; the target average feature variance is the average feature variance corresponding to the first global features of the first sample data under the N initial categories; randomly sampling each central global feature based on the target average feature variance to obtain the central sampling feature of each central global feature; the number of central sampling features of each central global feature is one or more; determining the central sampling feature corresponding to each central global feature and the second global feature of each second sample data as the target training features for the second classification model with added classification categories.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product comprises computer instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images