Data classification model training method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a data classification model training method and device, electronic equipment and a storage medium. The method comprises the steps of performing enhancement processing on a training sample in training data to obtain a first enhanced sample and a second enhanced sample; respectively carrying out feature extraction on the first enhanced sample and the second enhanced sample to obtain a first feature and a second feature; clustering the training data to obtain a plurality of reference categories and a plurality of clustering centers; comparing the first feature and the second feature with each clustering center to obtain a first probability and a second probability; performing feature calculation according to the first feature and each first probability to obtain a first prototype, and performing feature calculation according to the second feature and each second probability to obtain a second prototype; and based on each first prototype and each second prototype, performing prototype comparison learning on the initial model to obtain a data classification model. The classification accuracy of the data classification model can be improved.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and specifically to a data classification model training method, device, electronic device and storage medium. Background Art

[0002] At present, machine learning has achieved relatively successful applications and practices in data classification tasks. Generally speaking, data classification tasks require a large amount of labeled data as training sets for machine learning models. Among them, classification tasks involve new category discovery, that is, finding new categories from unlabeled data.

[0003] However, currently in new category discovery scenarios, especially generalized category discovery scenarios, it is easy to confuse new categories with known categories, making it impossible to guarantee data classification accuracy. Summary of the invention

[0004] The embodiments of the present application provide a data classification model training method, device, electronic device and storage medium, which can improve the classification accuracy of the data classification model.

[0005] In a first aspect, an embodiment of the present application provides a data classification model training method, comprising:

[0006] Performing enhancement processing on the training samples in the training data to obtain a first enhanced sample and a second enhanced sample;

[0007] Extracting features from the first enhanced sample using the initial model to obtain a first feature, and extracting features from the second enhanced sample using the initial model to obtain a second feature;

[0008] Clustering the training data to obtain multiple reference categories and a cluster center of a cluster corresponding to each reference category;

[0009] Comparing the first feature with each cluster center to obtain a first probability that the first feature belongs to each reference category, and comparing the second feature with each cluster center to obtain a second probability that the first feature belongs to each reference category;

[0010] Performing feature calculation according to the first feature and each first probability to obtain a first prototype of each reference category, and performing feature calculation according to the second feature and each second probability to obtain a second prototype of each reference category;

[0011] Based on each first prototype and each second prototype, prototype comparison learning is performed on the initial model to obtain a data classification model.

[0012] In a second aspect, an embodiment of the present application further provides a data classification model training device, comprising:

[0013] An enhancement unit, configured to perform enhancement processing on the training samples in the training data to obtain a first enhanced sample and a second enhanced sample;

[0014] A feature extraction unit, configured to extract features from the first enhanced sample using an initial model to obtain a first feature, and to extract features from the second enhanced sample using the initial model to obtain a second feature;

[0015] A clustering unit, used to cluster the training data to obtain a plurality of reference categories and a cluster center of a cluster corresponding to each reference category;

[0016] a comparison unit, configured to compare the first feature with each cluster center to obtain a first probability that the first feature belongs to each reference category, and to compare the second feature with each cluster center to obtain a second probability that the first feature belongs to each reference category;

[0017] a prototype determination unit, configured to perform feature calculation according to the first feature and each first probability to obtain a first prototype of each reference category, and perform feature calculation according to the second feature and each second probability to obtain a second prototype of each reference category;

[0018] The training unit is used to perform prototype comparison learning on the initial model based on each first prototype and each second prototype to obtain a data classification model.

[0019] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory storing a plurality of instructions; the processor loads instructions from the memory to execute the steps in any one of the data classification model training methods provided in the embodiments of the present application.

[0020] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium, which stores a plurality of instructions, and the instructions are suitable for a processor to load to execute the steps in any data classification model training method provided in the embodiment of the present application.

[0021] In an embodiment of the present application, the training samples in the training data are enhanced to obtain the first enhanced sample and the second enhanced sample; the first enhanced sample is then subjected to feature extraction through the initial model to obtain the first feature, and the second enhanced sample is subjected to feature extraction through the initial model to obtain the second feature; then, the training data is clustered to obtain multiple reference categories and the cluster center of each reference category. The first feature is then compared with each cluster center to obtain the first probability that the first feature belongs to each reference category, and the second feature is compared with each cluster center to obtain the second probability that the first feature belongs to each reference category; feature calculation is performed based on the first feature and each first probability to obtain the first prototype of each reference category, and feature calculation is performed based on the second feature and each second probability to obtain the second prototype of each reference category; based on each first prototype and each second prototype, the initial model is subjected to prototype comparison learning to obtain a data classification model. Therefore, during the training of the data classification model, a plurality of generalized reference categories can be initially divided in the training data by clustering. Afterwards, since the first probability and the second probability are equivalent to the pseudo-labels added to each training sample by clustering, the first prototype of the reference category is determined according to the first probability and the first feature, and the second prototype of the reference category is determined according to the second probability and the second feature, which is equivalent to using the above-mentioned pseudo-labels to calculate the prototype required by the initial model in the prototype contrast learning. Therefore, pseudo-supervision signals can be assigned to all reference categories in the prototype contrast learning, so that all reference categories can get more balanced supervision in the prototype contrast learning process, avoiding the trained data classification model from being overly biased towards known categories, and ensuring the classification accuracy of the trained data classification model for generalized categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1a It is a scenario diagram of the data classification model training method provided in an embodiment of the present application;

[0024] Figure 1b It is a flowchart of a data classification model training method provided in an embodiment of the present application;

[0025] Figure 2a The data classification model training method provided in the embodiment of the present application is applied in Figure 1aA schematic diagram of the process flow in the scenario shown;

[0026] Figure 2b It is a model architecture diagram of the initial model provided in the embodiment of the present application;

[0027] Figure 3 It is a structural diagram of a data classification model training device provided in an embodiment of the present application;

[0028] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0030] In order to better illustrate the solution of the present application, the professional terms involved in this embodiment are first explained:

[0031] Novel Class Discovery (NCD) refers to the use of knowledge from known categories to identify new categories in unlabeled data. In this new category discovery task, not only unlabeled data is used, but also labeled data is required to provide information about the new concept types to be discovered.

[0032] Generalized Category Discovery (GCD) is a data mining technique that aims to discover multiple types of categories, patterns, or structures from a data set. This concept emphasizes the understanding of diversity, complexity, and multimodality in a data set, rather than just identifying a single type of category. The goal of generalized category discovery is to explore multiple aspects of the data through a variety of techniques and methods. It not only focuses on identifying a single category or pattern, but also strives to discover diversity, correlation, and implicit structures in the data. Through generalized category discovery, we can better understand the complexity in the data, discover the laws hidden behind the data, and use this knowledge to make more accurate predictions and decisions.

[0033] Semi-supervised learning (SSL) is a machine learning paradigm that uses a dataset containing both labeled and unlabeled data to train models. In semi-supervised learning, only a small portion of data samples have labels (their categories or outputs are known), while most data samples do not have label information. The goal is to use the information from these labeled and unlabeled data to build more accurate and generalizable models.

[0034] Pseudo-label is a technique in semi-supervised learning, and is one of the methods used to improve model performance by using unlabeled data. In the pseudo-label method, the model is first trained using labeled data, then the model is used to predict unlabeled data, and these prediction results are added to the training set as pseudo-labels, and finally the model is retrained.

[0035] Prototype Contrastive Learning is a semi-supervised learning method that uses unlabeled data and a small amount of labeled data to improve model performance. The core idea of ​​this method is to enhance the model's ability to represent and generalize data by clustering and prototyping data samples.

[0036] Epochs refers to the number of times the entire dataset is fully passed through once and used to train the model in machine learning or deep learning. In each training epoch, the model performs forward pass and backward pass on the entire dataset, and updates the model parameters based on the gradient calculated by the loss function. In deep learning, iterative training is generally used, and each training epoch can be divided into multiple mini-batch trainings. In each mini-batch training, the model performs forward and backward propagation on a small portion of the data and updates the model parameters. The entire training process is repeated for multiple training epochs until the conditions for stopping training are met (such as reaching the maximum number of training epochs, loss function convergence, etc.).

[0037] A training batch usually refers to the number of small batches into which a dataset is divided when training a neural network or other machine learning model. The size of the dataset contained in each training batch is the batch size, that is, the number of samples contained in each batch.

[0038] Embodiments of the present application provide a data classification model training method, device, electronic device and storage medium.

[0039] The data classification model training device can be integrated into an electronic device, which can be a terminal, a server, or other devices. The terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, or a personal computer (PC), etc. The server can be a single server or a server cluster composed of multiple servers.

[0040] In some embodiments, the data classification model training device can also be integrated into multiple electronic devices. For example, the data classification model training device can be integrated into multiple servers, and the data classification model training method of the present application can be implemented by multiple servers.

[0041] In some embodiments, the server may also be implemented in the form of a terminal.

[0042] For example, refer to Figure 1a , Figure 1a A schematic diagram of an application scenario of the data classification model training method provided in an embodiment of the present application is shown.

[0043] like Figure 1a As shown, the application scenario may include a server 110 and a client 120, wherein the server 110 may be connected to the client 120 through a network for communication. The number of the client 120 may be one or more, which is not limited here.

[0044] The server 110 may have a data classification function, which may include new category discovery. In practical applications, the server 110 may receive the data to be classified sent by the client 120, and then identify the category of the data to be classified to obtain a classification result, and return the classification result to the client 120. Optionally, the data to be classified may be image data.

[0045] In the related art, when the server 110 discovers new categories, it is usually assumed that there is no data in the unlabeled data that belongs to the samples of the known categories in the labeled set. However, this assumption is too simplistic. In a generalized category discovery scenario that is closer to reality, the unlabeled data may contain samples of known categories and samples of new categories, which leads to inaccurate discovery of new categories in the related art. In addition, in the related art, the discovery of new categories mainly focuses on how to improve the method of category discovery tasks, such as using parameter classifier methods.

[0046] However, parameter-based classifier methods usually confuse new categories with known categories. To address this problem, this embodiment proposes to use prototype contrast learning in semi-supervised learning to obtain more accurate representations, thereby improving the performance of GCD classifier learning.

[0047] Specifically, when using prototype contrast learning, the prototype obtained by running the k-means algorithm on the entire dataset can be used to compare the high-level abstract feature representation of the dataset, and the classification model can be trained based on the feature representation, thereby improving the classification model's recognition accuracy for new categories.

[0048] However, the k-means algorithm can only perform pseudo-label assignment once in each model training round, which cannot provide the classification model with the latest clustering information for each training batch, nor can it directly backpropagate gradients through prototype contrastive learning loss, so the classification accuracy of the classification model cannot be guaranteed.

[0049] In view of this, this embodiment proposes a data classification model training method, which clusters the training data and uses the cluster center to determine the first probability corresponding to the first feature of each training sample and the second probability corresponding to the second feature of each training sample, which is equivalent to adding a pseudo label to each training sample, and then determining the first prototype of the reference category according to the first probability and the first feature, and determining the second prototype of the reference category according to the second probability and the second feature, which is equivalent to using the above pseudo label to calculate the prototype required by the initial model in the prototype contrast learning, so that pseudo supervision signals can be assigned to all reference categories in the prototype contrast learning, so that all reference categories can be supervised more evenly during the prototype contrast learning process, and avoid the trained data classification model from being overly inclined to known categories. In addition, by calculating pseudo labels and constructing prototypes for small batches of training samples in each training batch, compared with only one pseudo label assignment per model training round, the prototype can be used more effectively to learn the unbiased representation of generalized category discovery, so as to better discover new categories, thereby ensuring the classification accuracy when classifying the data to be classified by the data classification model.

[0050] It should be noted that the serial numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0051] It is understandable that in the specific implementation of the present application, related data such as data to be classified is involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0052] In this embodiment, a data classification model training method is provided. The data classification scheme can be applied to Figure 1b Servers in Figure 1b As shown, the specific process of the data classification model training method can be as follows:

[0053] 101. Perform enhancement processing on training samples in training data to obtain first enhanced samples and second enhanced samples.

[0054] The training data is a data set used to train a data classification model, and the training data may include multiple training samples. Optionally, the training sample may be image data, or other data that can be converted into a vector of a specified dimension by a feature extractor (such as a deep neural network), which is not limited here. It is understandable that the training sample can be associated with a specific category through a semantic description (such as an attribute vector or a word vector).

[0055] In some implementations, before performing enhancement processing on the training samples in the training data, the method may further include:

[0056] A training sample set that is pre-divided into training batches and a current training batch of the initial model are obtained; and a training sample corresponding to the current training batch is obtained from the training sample set as training data.

[0057] The initial model may be a classification model whose model parameters have not yet been determined, i.e., a data classification model to be trained. The data classification model is used to classify input data into one of two or more categories. Such a model can be used to solve classification problems, where the goal is to map input data to predefined category labels. Optionally, the initial model may be a brand new model or a pre-trained model that needs to be fine-tuned or improved, which is not limited here. After the initial model is trained, the trained initial model can be used as a data classification model.

[0058] As an implementation method, multiple training samples can be obtained from a training sample library, and the multiple training samples can be divided into a training sample set of multiple training batches. For example, the multiple training samples obtained from the training sample library are 6,000 images, and the 6,000 images are divided into 48 training batches, each training batch corresponding to a training sample set of 125 images.

[0059] Among them, each training batch can be bound to a batch number (such as training batch 1, training batch 2, training batch 3, etc.). When the model is trained, the training sample set can be input into the model as training data in the order of the batch number from small to large for training. For example, if it is detected that the current training batch of the initial model is training batch 3, the 125 images corresponding to training batch 3 can be used as training data and input into the initial model for training. When the initial model proceeds to the next training batch, the 125 images corresponding to training batch 4 can be used as training data and input into the initial model for training. After the initial model traverses all training batches, the initial model enters the next training round, and in the next training round, the training sample sets of all training batches are traversed again until the initial model training is completed.

[0060] In some implementations, each training sample in the training data may be enhanced in two different enhancement modes to obtain a first enhanced sample and a second enhanced sample.

[0061] Among them, the enhancement method refers to a technology or method applied in the process of data augmentation. Data augmentation is a data preprocessing technology that generates new samples by performing preprocessing operations on the original data. Optionally, taking the training sample as image data as an example, the preprocessing operation includes but is not limited to: random cropping to a uniform size (such as 224×224), random left and right flipping, random color jitter, random grayscale transformation, etc. It can be understood that for different types of training samples, enhancement methods related to the type of training samples can be used.

[0062] In this embodiment, two enhanced samples of the same training sample are obtained by performing enhancement processing in two different enhancement modes on each training sample, so as to facilitate subsequent comparative learning of the initial model using the two enhanced samples.

[0063] For example, for a training sample in a training batch (such as x i ), the training sample can be randomly flipped left and right to obtain the first enhanced sample (such as ), perform random color jitter on the training sample to obtain a second enhanced sample (such as ). Among them, the training sample set corresponding to the training batch is B, x i represents the i-th training sample in the training sample set B, where i is a positive integer.

[0064] 102. Perform feature extraction on the first enhanced sample using the initial model to obtain a first feature, and perform feature extraction on the second enhanced sample using the initial model to obtain a second feature.

[0065] In some embodiments, the initial model may include a feature extraction network. After obtaining the first enhanced sample and the second enhanced sample, feature extraction can be performed on the first enhanced sample through the feature extraction network to obtain a first feature, and feature extraction can be performed on the second enhanced sample through the feature extraction network to obtain a second feature.

[0066] Among them, the feature extraction network is used to extract features from the input raw data and output the feature representation extracted from the raw data.

[0067] Using the above example, for example, the first enhanced sample is extracted through the feature extraction network Perform feature extraction to get the first feature The second enhanced sample is extracted through the feature extraction network By performing feature extraction, the second feature can be obtained

[0068] 103. Cluster the training data to obtain multiple reference categories and the cluster center of each reference category corresponding to the cluster.

[0069] The cluster center of the cluster corresponding to the reference category may refer to the feature representation that best represents the reference category. The cluster center of the cluster corresponding to the reference category may also be called the prototype corresponding to the reference category.

[0070] In some implementations, in step 103, the specific implementation of clustering the training data may include:

[0071] The training data is clustered by a K-means clustering algorithm. Optionally, the K-means clustering algorithm may be a k-means++ algorithm (a clustering algorithm).

[0072] Exemplarily, for the training data corresponding to the current training batch, for example, the training sample set B corresponding to the training batch includes training sample x1, training sample x2, training sample x3, and training sample x4. After clustering the training sample x1, training sample x2, training sample x3, and training sample x4 using the K-means clustering algorithm, three clustering results are obtained, such as reference category 1 including only training sample x1, reference category 2 including only training sample x2, and reference category 3 including both training sample x3 and training sample x4. It can be understood that the reference category may be an unknown category, that is, a category that is only represented by a code and has no actual meaning.

[0073] 104. Compare the first feature with each cluster center to obtain a first probability that the first feature belongs to each reference category, and compare the second feature with each cluster center to obtain a second probability that the first feature belongs to each reference category.

[0074] Among them, the first probability that the first feature belongs to a reference category can specifically represent the probability that the first feature is assigned to the prototype corresponding to the reference category. When the probability that the first feature is assigned to the prototype corresponding to the reference category is 1, it indicates that the training sample corresponding to the first feature can be classified into the reference category. When the probability that the first feature belongs to the prototype corresponding to the reference category is 0, it indicates that the training sample corresponding to the first feature cannot be classified into the reference category. Similarly, the second probability that the second feature belongs to a reference category can specifically represent the probability that the first feature is assigned to the prototype corresponding to the reference category.

[0075] In some embodiments, after obtaining multiple reference categories, the first feature can be compared with the cluster center corresponding to each reference category. If the comparison result is that the first feature matches the cluster center corresponding to the reference category, the first probability that the first feature belongs to the reference category can be determined to be 1. If the comparison result is that the first feature does not match the cluster center corresponding to the reference category, the first probability that the first feature belongs to the reference category can be determined to be 0. As an embodiment, when the first feature is consistent with the cluster center corresponding to the reference category, the comparison result can be determined to be that the first feature matches the cluster center corresponding to the reference category. As another embodiment, when the feature similarity between the first feature and the cluster center corresponding to the reference category meets a preset similarity condition (such as the similarity is greater than or equal to the similarity threshold), the comparison result can be determined to be that the first feature matches the cluster center corresponding to the reference category.

[0076] Similarly, the method for determining the second probability can refer to the method for determining the first probability mentioned above, so it will not be repeated here.

[0077] Using the above example, for example, clustering the training data in the current training batch, multiple reference categories are obtained, there are K (such as L1, L2, ... Lk), each reference category corresponds to a prototype, therefore, multiple cluster centers corresponding to the multiple reference categories (hereinafter referred to as prototypes) include {c1, c2, ..., c K}. Where K is a positive integer.

[0078] Then the training sample x i The corresponding first feature Distribute to multiple prototypes prototype c j The first probability can be expressed as Among them, the prototype c j is the jth prototype among multiple prototypes, where j is a positive integer and j is less than or equal to K. Similarly, the second feature corresponding to the training sample xi is Distribute to multiple prototypes prototype c j The second probability can be expressed as

[0079] 105. Perform feature calculation based on the first feature and each first probability to obtain a first prototype of each reference category, and perform feature calculation based on the second feature and each second probability to obtain a second prototype of each reference category.

[0080] In some implementations, in step 105, a specific implementation of performing feature calculation according to the first feature and each first probability to obtain the first prototype of each reference category may include:

[0081] For each reference category, the first feature is multiplied by the first probability that the first feature belongs to the reference category to obtain a first product corresponding to the first feature.

[0082] All first products corresponding to the first features are accumulated to obtain a first accumulation result.

[0083] The first accumulated result is divided by the absolute value of the first accumulated result to obtain a first prototype of the reference category.

[0084] Using the above example, the first prototype can be calculated by the following expression:

[0085]

[0086] in, For reference category L j The corresponding first prototype, B is the training sample set corresponding to the current training batch, is the first enhanced sample in the training sample set B The corresponding first feature. The first feature Assigned to the first prototype The first probability.

[0087] is the first accumulated result. is the absolute value of the first accumulated result.

[0088] In some implementations, in step 105, a specific implementation of performing feature calculation based on the second feature and each second probability to obtain the second prototype of each reference category may include:

[0089] For each reference category, multiply the second feature by the second probability that the second feature belongs to the reference category to obtain a second product corresponding to the second feature;

[0090] Accumulate the second products corresponding to all the second features to obtain a second accumulation result;

[0091] The second accumulated result is divided by the absolute value of the second accumulated result to obtain a second prototype of the reference category.

[0092] Continuing with the above example, the second prototype can be calculated by the following expression:

[0093]

[0094] in, For reference category L j The corresponding second prototype, B is the training sample set corresponding to the current training batch, is the second enhanced sample in the training sample set B The corresponding second feature. The second feature Assigned to the second prototype The second probability.

[0095] is the second accumulated result. is the absolute value of the second accumulated result.

[0096] In this embodiment, since the prototype assignment is done using the k-means++ algorithm rather than the prediction of the model, k-means++ clustering tends to generate clusters of uniform size, so it can be expected that the prototype is unbiased. In addition, the visible category data can be compactly clustered and the training samples of the new category can be assigned to different clusters, including those of the new category clusters that have not been seen, so that the seen categories and the new categories can be effectively separated, the representation ability of the visible categories and the new categories can be improved, and the new category discovery performance can be improved.

[0097] 106. Based on each first prototype and each second prototype, perform prototype comparative learning on the initial model to obtain a data classification model.

[0098] In some implementations, in step 106, based on each first prototype and each second prototype, prototype comparison learning is performed on the initial model to obtain a data classification model. The specific implementation may include:

[0099] A1. Calculate the contrast loss for each first prototype and each second prototype to obtain the prototype contrast loss.

[0100] In some implementations, in step A1, the contrast loss is calculated for each first prototype and each second prototype, and a specific implementation of obtaining the prototype contrast loss may include:

[0101] A11. Select a reference category from multiple reference categories as a target reference category, and use reference categories other than the target reference category from the multiple reference categories as candidate reference categories.

[0102] Using the above example, the multiple reference categories include: reference category L1, reference category L2, ..., reference category L K For example, you can select reference category L from multiple reference categories. j As the target reference category, then remove the reference category L from the multiple reference categories. j Reference categories other than L are used as candidate reference categories. For example, the candidate reference category is reference category L i , where i is not equal to j.

[0103] A12, calculating the initial loss of the target reference category according to the first prototype corresponding to each candidate reference category, the second prototype corresponding to each candidate reference category, and the second prototype corresponding to the target reference category;

[0104] In some implementations, in step A12, according to the first prototype corresponding to each candidate reference category, the second prototype corresponding to each candidate reference category, and the second prototype corresponding to the target reference category, a specific implementation of calculating the initial loss of the target reference category may include:

[0105] A121. Perform vector calculation based on the second prototype corresponding to the target reference category and the first prototype corresponding to each candidate reference category to obtain a first target vector.

[0106] Continuing with the above example, since the first prototype and the second prototype of each reference category are obtained in step 105, after determining the target reference category and the candidate reference category, the first prototype corresponding to the candidate reference category can be screened out from the obtained multiple first prototypes, and the second prototype corresponding to the candidate reference category and the second prototype corresponding to the target reference category can be screened out from the obtained multiple second prototypes. For example, for the reference category Lj, its corresponding second prototype can be screened out. For the reference category L i , you can filter out the corresponding first prototype and the second prototype

[0107] In some implementations, in step A121, a specific implementation of performing vector calculation based on the second prototype corresponding to the target reference category and the first prototype corresponding to each candidate reference category to obtain the first target vector may include:

[0108] S1. Calculate the product between the second prototype corresponding to the target reference category and the first prototype corresponding to each candidate reference category to obtain multiple initial vectors, and calculate the quotient of each initial vector and a preset temperature hyperparameter to obtain multiple initial calculation vectors.

[0109] Among them, the temperature hyperparameter is a parameter used to improve model training and generalization performance. In deep learning, especially in classification tasks, it is usually used with the cross entropy loss function to adjust the softening degree of labels.

[0110] Using the above example, the target reference category is the reference category Lj, and the reference category Lj corresponds to the second prototype The candidate reference category is the reference category Li, and the first prototype corresponding to the reference category Li and the second prototype The default temperature hyperparameter is τ c For example, according to the calculation process of step S1, the expression of the initial calculation vector can be obtained as follows:

[0111] S2. Calculate each initial calculation vector using an exponential function to obtain a first calculation vector, and accumulate multiple first calculation vectors to obtain a first target vector.

[0112] Using the above example, we can get the expression of the first calculation vector as Where exp() is an exponential function. Furthermore, the expression of the first target vector is Here, K is the number of reference categories.

[0113] A122. Perform vector calculation based on the first prototype corresponding to each candidate reference category and the second prototype corresponding to each candidate reference category to obtain a second target vector.

[0114] In some implementations, in step A122, a specific implementation of performing vector calculation based on the first prototype corresponding to each candidate reference category and the second prototype corresponding to each candidate reference category to obtain the second target vector may include:

[0115] B1. For each candidate reference category, multiply the first prototype corresponding to the candidate reference category by the second prototype corresponding to the candidate reference category and divide the result by a preset temperature hyperparameter to obtain a second calculation vector.

[0116] Using the above example, according to the calculation process of step B1, the expression of the second calculation vector can be obtained as follows:

[0117] B2. Calculate the second calculation vector using an exponential function to obtain a second target vector.

[0118] Using the above example, through the calculation process of step B2, the expression of the second target vector can be obtained as follows:

[0119] A123. Calculate the ratio of the second target vector to the first target vector to obtain a target ratio.

[0120] Using the above example, according to the calculation process of A123, the expression of the target ratio can be obtained as follows:

[0121]

[0122] A124. A plurality of target logarithms are obtained by performing logarithmic processing on each target ratio through a logarithmic function, and the negative value of each target logarithm in the plurality of target logarithms is determined as the initial loss of the target reference category.

[0123] Continuing with the above example, according to the calculation process of A124, the initial loss can be expressed as follows:

[0124]

[0125] in, is the initial loss.

[0126] A13. Return to execute the step of selecting a reference category from multiple reference categories as the target reference category, until all reference categories are selected as target reference categories, and initial losses of multiple target reference categories are obtained.

[0127] A14. Fusion multiple initial losses to obtain prototype contrast loss.

[0128] Continuing with the above example, when multiple initial losses are fused, the multiple initial losses can be accumulated to obtain the prototype contrast loss. The expression of the obtained prototype contrast loss can be as follows:

[0129]

[0130] in, is the prototype contrast loss.

[0131] A2. Based on the prototype contrast loss, the initial model is converged to obtain the data classification model.

[0132] In some implementations, the initial model includes a projection network and a classification network. In step A2, based on the prototype contrast loss, the initial model is converged to obtain a data classification model. The specific implementation may include:

[0133] A21. Perform representation mapping on the training data through the projection network to obtain a mapping result, and determine the representation loss based on the mapping result.

[0134] In some embodiments, the training samples include labeled samples, wherein the labeled samples are training samples with pre-set category labels. In step A21, the training data is represented and mapped by a projection network to obtain a mapping result, and a specific embodiment of determining the representation loss based on the mapping result may include:

[0135] A211. For each training sample in the training data, a first feature of the training sample is represented and mapped by a projection network to obtain a first mapping representation, and a second feature of the training sample is represented and mapped by a projection network to obtain a second mapping representation.

[0136] Among them, the projection network is used to learn data embedding or dimensionality reduction. It accepts high-dimensional features or representations learned from other networks (such as representation networks) and maps them to a low-dimensional space, thereby achieving data dimensionality reduction or compression.

[0137] Using the above example, the first feature Performing the representation mapping can obtain the first mapping representation The second feature is transformed by the projection network Performing the representation mapping can obtain the first mapping representation

[0138] A212. Determine a first loss based on the first mapping representation and the second mapping representation;

[0139] As an example, the first loss can be calculated by the following formula:

[0140]

[0141] in, is the first loss, B is the training sample set corresponding to the current training batch, x i is the i-th training sample in the training sample set B. j is the jth training sample in the training sample set B, is the training sample x j The corresponding first mapping is represented by τ u is the specified temperature hyperparameter. The value of the specified temperature hyperparameter can be customized according to actual needs.

[0142] A213. For the labeled samples, a first feature of the labeled samples is represented and mapped by a projection network to obtain a third mapping representation, and a second feature of the labeled samples is represented and mapped by a projection network to obtain a fourth mapping representation.

[0143] A214. Determine a second loss based on the third mapping representation and the fourth mapping representation.

[0144] Using the above example, the second loss can be calculated by the following formula:

[0145]

[0146] Among them, B l is a training sample set consisting of labeled samples in training sample set B. is the training sample set B l With training sample x i The indices of all training examples with the same label. The fourth mapping representation is obtained by performing representation mapping based on the second feature corresponding to the training sample with the same label as the index p. It is a mapping representation obtained by performing representation mapping based on the second feature corresponding to the training sample with the same label as index n. Based on the training sample set B l The training samples x in i The corresponding first feature is represented by mapping to a third mapping representation.

[0147] A215. The first loss and the second loss are combined to obtain a representation loss.

[0148] In some implementations, a first weighted hyperparameter may be obtained, and then the first loss and the second loss are weighted summed based on the first weighted hyperparameter to obtain a representation loss. The first weighted hyperparameter may be customized according to actual needs.

[0149] For example, if the first weighted hyperparameter is λ, the expression representing the loss may be:

[0150]

[0151] in, To indicate loss.

[0152] A22. Classify the training data through the classification network to obtain the classification result, and determine the classification loss based on the classification result.

[0153] In some implementations, in step A22, the training samples include labeled samples, the training data are classified by a classification network to obtain a classification result, and a specific implementation method for determining the classification loss based on the classification result may include:

[0154] A221. For each training sample in the training data, a first feature of the training sample is classified by a classification network to obtain a first classification result, and a second feature of the training sample is classified by a classification network to obtain a second classification result.

[0155] Among them, a classification network is a network used to map input data to predefined categories or labels.

[0156] In some embodiments, a plurality of initial prototypes {t1, t2, ..., t K}, then the classification network can assign labels to each training sample using the following classification formula:

[0157]

[0158] in, The first feature The label obtained by classification processing, that is, the first classification result mentioned above, means that the first feature Assigned to prototype t j Similarly, the second classification result can also be obtained through the above formula That is, for the second feature Labels obtained after classification.

[0159] A222. Determine a third loss based on the first classification result and the second classification result.

[0160] Using the above example, after obtaining the first classification result and the second classification result, the third loss can be calculated by the following formula:

[0161]

[0162] in, is the third loss, B is the training sample set corresponding to the current training batch, |B| is the number of training samples in the training sample set B, where, in, Cross entropy loss calculation function.

[0163] A223. For the labeled samples, the first feature of the labeled samples is classified by the classification network to obtain a third classification result.

[0164] A224. Determine the fourth loss based on the third classification result.

[0165] For example, assume that the labeled sample is a training sample x i , training sample x i The pre-labeled label is label y i , we can use the above classification formula to classify the labeled samples x i The first characteristic Perform classification processing to obtain the third classification result Then, the label yi and the third classification result The fourth loss is calculated using the following formula:

[0166]

[0167] in, For the fourth loss, B l is a training sample set consisting of labeled samples in training sample set B, |B l | is the training sample set B l The number of training samples in .

[0168] A225. The third loss and the fourth loss are integrated to obtain the classification loss.

[0169] In some implementations, a second weighted hyperparameter may be obtained, and then the third loss and the fourth loss are weighted summed based on the second weighted hyperparameter to obtain the classification loss. The second weighted hyperparameter may be customized according to actual needs.

[0170] For example, if the second weighted hyperparameter is λ1, the expression representing the loss may be:

[0171]

[0172] in, is the classification loss.

[0173] A23. The prototype contrast loss, representation loss and classification loss are integrated to obtain the target loss, and the initial model is converged based on the target loss to obtain the data classification model.

[0174] In some embodiments, after obtaining the prototype contrast loss, representation loss, and classification loss, the preset weight ratios corresponding to the prototype contrast loss, representation loss, and classification loss can be obtained, and then based on the preset weight ratio, the prototype contrast loss, representation loss, and classification loss are weighted and summed to obtain the target loss.

[0175] For example, if the preset weight ratios of prototype contrast loss, representation loss, and classification loss are k1:k2:k3, then the expression of target loss can be as follows:

[0176]

[0177] in, is the target loss. Optionally, the preset weight ratios corresponding to the prototype contrast loss, representation loss, and classification loss can be customized according to actual needs and are not limited here.

[0178] In some other implementations, in step A23, the prototype contrast loss, the representation loss, and the classification loss are integrated to obtain the target loss. The specific implementation may include:

[0179] A231. Obtain the fusion weight of the prototype contrast loss. The fusion weight is greater than 0 and less than or equal to 1.

[0180] In some implementations, the fusion weight may be a fixed value or a dynamically changing value, which is not limited herein.

[0181] In some other implementations, in step A231, a specific implementation of obtaining the fusion weight of the prototype contrast loss may include:

[0182] The current training round of the initial model is obtained; and the fusion weight is determined according to the current training round, and the fusion weight is positively correlated with the current training round.

[0183] Exemplarily, for example, the fusion weight can be represented by ω(t), where t is the current training round, and the current training round will increase as the model training round increases. When the current training round increases, the fusion weight will also increase accordingly. Optionally, the relationship between the current training round t and the fusion weight ω(t) can be a linear relationship or a nonlinear relationship, which is not limited here.

[0184] Optionally, the initial value of the fusion weight may not exceed a weight threshold.

[0185] A232. Based on the fusion weight, the prototype contrast loss, representation loss and classification loss are fused to obtain the target loss.

[0186] Continuing with the above example, the expression of the target loss can be:

[0187] Taking into account that the prototype may be estimated inaccurately when the initial model is just started to be trained, in this embodiment, the current training round of the initial model is obtained, and the fusion weight is determined according to the current training round, wherein the fusion weight is positively correlated with the current training round, thereby reducing the impact of inaccurate early prototype comparison loss, thereby improving the model training efficiency.

[0188] It can be understood that in the process of training the initial model based on the target loss to obtain the data classification model, the initial model can be iterated once through the training samples corresponding to the current training batch, that is, the network parameters of the feature extraction network, projection network and classification network in the initial model are adjusted by the target loss calculated by the training samples corresponding to the current training batch, and then the initial model is iterated again based on the next batch after the current training batch. When the training samples corresponding to all training batches are traversed, it means that the initial model has completed a training round, and then the above process can be repeated to perform model training for the next training round until the training round of the initial model reaches the preset round, or the target loss of the initial model converges to the specified range, and the data classification model is obtained. Among them, each training batch can update the prototype corresponding to the reference type obtained by the preset clustering algorithm. Each training round can update the fusion weight of the prototype comparison loss.

[0189] In some implementations, after step 106, the method may further include:

[0190] When the data to be classified is received, the data to be classified is classified by the data classification model.

[0191] It can be seen that in this embodiment, the first enhanced sample and the second enhanced sample are obtained by enhancing the training samples in the training data; the first enhanced sample is then subjected to feature extraction through the initial model to obtain the first feature, and the second enhanced sample is subjected to feature extraction through the initial model to obtain the second feature; then, the training data is clustered to obtain multiple reference categories and the cluster center of each reference category. Then, the first feature is compared with each cluster center to obtain the first probability that the first feature belongs to each reference category, and the second feature is compared with each cluster center to obtain the second probability that the first feature belongs to each reference category; feature calculation is performed based on the first feature and each first probability to obtain the first prototype of each reference category, and feature calculation is performed based on the second feature and each second probability to obtain the second prototype of each reference category; based on each first prototype and each second prototype, the initial model is subjected to prototype comparison learning to obtain a data classification model. Therefore, in the process of data classification model training, multiple generalized reference categories can be initially divided in the training data by clustering. After that, since the first probability and the second probability are equivalent to the pseudo-labels added to each training sample by clustering, the first prototype of the reference category is determined according to the first probability and the first feature, and the second prototype of the reference category is determined according to the second probability and the second feature, which is equivalent to using the above pseudo-labels to calculate the prototype required by the initial model in prototype contrast learning, so that pseudo-supervision signals can be assigned to all reference categories in prototype contrast learning, so that all reference categories can be supervised more evenly in the process of prototype contrast learning, avoiding the trained data classification model from being overly inclined to known categories, and ensuring the classification accuracy of the trained data classification model for generalized categories. In addition, by calculating pseudo-labels and constructing prototypes for small batches of training samples in each training batch, compared with only one pseudo-label assignment per model training round, prototypes can be used more effectively to learn unbiased representations of generalized category discovery, so as to better discover new categories, thereby ensuring the classification accuracy when classifying the categorized data through the data classification model.

[0192] The method described in the above embodiment will be further described in detail below.

[0193] In this embodiment, the data classification model training method is applied to Figure 1a Taking the application scenario shown as an example, the method of the embodiment of the present application is described in detail.

[0194] like Figure 2a As shown, the specific process of a data classification model training method is as follows:

[0195] 201. The server obtains training samples of multiple training batches and an initial model, where the initial model includes a feature extraction network, a projection network, and a classification network.

[0196] 202. The server obtains, for each training sample of each training batch, a first enhanced sample and a second enhanced sample obtained under two different enhancement methods, and respectively extracts features from the first enhanced sample and the second enhanced sample through a feature extraction network to obtain a first feature and a second feature.

[0197] 203. The server clusters the training samples in the training batch by using a preset clustering algorithm to obtain multiple reference categories, and determines a first probability that the first feature is assigned to a prototype corresponding to each reference category and a second probability that the second feature is assigned to a prototype corresponding to each reference category.

[0198] The preset clustering algorithm may be a k-means++ algorithm.

[0199] 204. The server determines a first prototype of the reference category according to the first probability and the first feature, and determines a second prototype of the reference category according to the second probability and the second feature.

[0200] Determining the first prototype of the reference category according to the first probability and the first feature may include:

[0201] For each first feature, the first feature is multiplied by the first probability corresponding to the first feature to obtain a first product.

[0202] All first products are accumulated to obtain a first accumulated result.

[0203] The first accumulated result is divided by the absolute value of the first accumulated result to obtain a first prototype.

[0204] Accordingly, determining the second prototype of the reference category according to the second probability and the second feature may include:

[0205] For each second feature, the second feature is multiplied by the second probability corresponding to the second feature to obtain a second product.

[0206] All second products are accumulated to obtain a second accumulated result.

[0207] The second accumulated result is divided by the absolute value of the second accumulated result to obtain a second prototype.

[0208] 205. The server determines a prototype comparison loss based on the first prototype and the second prototype.

[0209] Wherein, in step 205, determining the prototype contrast loss based on the first prototype and the second prototype may include:

[0210] Any one of the multiple reference categories is used as a target reference category, and reference categories other than the target reference category among the multiple reference categories are used as candidate reference categories.

[0211] A first prototype corresponding to the candidate reference category, a second prototype corresponding to the candidate reference category, and a second prototype corresponding to the target reference category are obtained.

[0212] A prototype contrast loss is determined according to a first prototype corresponding to the candidate reference category, a second prototype corresponding to the candidate reference category, and a second prototype corresponding to the target reference category.

[0213] The specific implementation of the step of “determining the prototype contrast loss according to the first prototype corresponding to the candidate reference category, the second prototype corresponding to the candidate reference category, and the second prototype corresponding to the target reference category” may include:

[0214] Get the preset temperature hyperparameters.

[0215] The products of the second prototype corresponding to the target reference category and the first prototype corresponding to each candidate reference category are calculated to obtain multiple initial vectors, and the quotient of each initial vector and a preset temperature hyperparameter is calculated to obtain multiple initial calculation vectors.

[0216] Each initial calculation vector is calculated by using an exponential function to obtain a plurality of first calculation vectors, and the plurality of first calculation vectors are accumulated to obtain a first target vector.

[0217] For each candidate reference category, a first prototype corresponding to the candidate reference category is multiplied by a second prototype corresponding to the candidate reference category and then divided by a preset temperature hyperparameter to obtain a second calculation vector.

[0218] The second calculation vector is calculated by using an exponential function to obtain a second target vector, and the ratio of the second target vector to the first target vector is calculated to obtain a target ratio.

[0219] By performing logarithmic processing on each target ratio through a logarithmic function, multiple target logarithms are obtained, and the negative value of each target logarithm in the multiple target logarithms is accumulated to obtain the prototype contrast loss.

[0220] 206. The server obtains the representation loss of the projection network corresponding to the multiple training samples.

[0221] Wherein, the multiple training samples include labeled samples, and the specific implementation of the step of "obtaining the representation loss of the projection network corresponding to the multiple training samples" may include:

[0222] For each training sample in the multiple training samples, a first feature and a second feature of the training sample are represented and mapped by a projection network to obtain a first mapping representation and a second mapping representation, and a first loss is determined based on the first mapping representation and the second mapping representation.

[0223] For the labeled samples, a first feature and a second feature of the labeled samples are represented and mapped by a projection network to obtain a third mapping representation and a fourth mapping representation, and a second loss is determined based on the third mapping representation and the fourth mapping representation.

[0224] The first loss and the second loss are combined to obtain the representation loss.

[0225] 207. The server obtains classification losses of the classification network corresponding to multiple training samples;

[0226] Among them, the specific implementation of the step of "the multiple training samples include labeled samples, and the classification loss of the classification network corresponding to the multiple training samples" may include:

[0227] For each training sample in the multiple training samples, a first feature and a second feature of the training sample are classified by a classification network to obtain a first classification result and a second classification result, and a third loss is determined based on the first classification result and the second classification result.

[0228] For the labeled samples, the first feature of the labeled samples is classified by the classification network to obtain a third classification result, and the fourth loss is determined based on the third classification result.

[0229] The third loss and the fourth loss are combined to obtain the classification loss.

[0230] 208. The server fuses the prototype contrast loss, the representation loss, and the classification loss to obtain the target loss, and performs model training on the initial model based on the target loss to obtain a data classification model.

[0231] In step 208, the prototype contrast loss, the representation loss, and the classification loss are fused to obtain the target loss, which may include:

[0232] Get the fusion weight of the prototype contrast loss. The fusion weight is greater than 0 and less than or equal to 1.

[0233] Based on the fusion weight, the prototype contrast loss, representation loss and classification loss are fused to obtain the target loss.

[0234] In some embodiments, the step of “obtaining fusion weights of prototype contrast loss” may include:

[0235] Get the current training epoch of the Inception model.

[0236] The fusion weight is determined according to the current training round, and the fusion weight is positively correlated with the current training round.

[0237] 209. When receiving the data to be classified, the server classifies the data to be classified using the data classification model and returns the classification result to the client.

[0238] For example, in practical applications, the model training architecture of the initial model can be as follows: Figure 2b As shown, according to Figure 2b It can be seen that the initial model can include a first enhancement module, a second enhancement module, a feature extraction network, a projection network and a classification network.

[0239] When the initial model is trained based on the model training architecture, the training samples can be first input into the first enhancement module and the second enhancement module respectively, so as to perform data enhancement on the training samples in two different enhancement modes to obtain the first enhanced sample and the second enhanced sample. Then, the first enhanced sample and the second enhanced sample are subjected to feature extraction processing by the feature extraction network respectively to obtain the first feature and the second feature.

[0240] After obtaining the first feature and the second feature, on the one hand, the first feature and the second feature can be input into the projection network, and the first mapping representation and the second mapping representation can be obtained through the mapping processing of the projection network, and then the representation loss can be calculated through the first mapping representation and the second mapping representation. On the other hand, the first feature and the second feature can be input into the classification network, and the first classification result and the second classification result can be obtained through the classification processing of the classification network, and then the classification loss can be calculated through the first classification result and the second classification result. On the other hand, the training samples can be clustered by a preset clustering algorithm to obtain multiple clustering results, and then the first prototype and the second prototype are determined based on the first feature and the second feature and the multiple clustering results, and then the prototype contrast loss is calculated based on the first prototype and the second prototype.

[0241] Then, the target loss is determined by combining the prototype contrast loss, representation loss, and classification loss.

[0242] Finally, based on the target loss, the model parameters of the projection network, feature extraction network, and classification network are updated to achieve model training.

[0243] As a more specific example, the following Figure 2b The model architecture of FIG. 2 illustrates the model training process from step 201 to step 208.

[0244] First, data initialization can be performed. Taking images as training samples, data initialization can include the following steps:

[0245] Step 1: Load the image dataset, which can include unlabeled datasets and labeled datasets Based on a specific data size (such as 128), the labeled image data and the unlabeled image data are loaded into a processing batch in proportion to obtain batch data B (containing labeled data Unlabeled data ).

[0246] Step 2: For the image data in the batch data B, construct an image enhancement transformation set T, where transformation T includes common image enhancement methods, including random cropping to a uniform size (such as 224×224), random left and right flipping, random color jittering, random grayscale transformation, etc.

[0247] Then, the initial model is initialized, and the model initialization may include the following steps:

[0248] Step 1: Before model optimization, build a feature extraction network f(·) based on the basic feature extraction backbone network (resnet residual network, VGG network, ViT network, etc.). Initialize the pre-trained network such as ImageNet in the initial model and freeze some parameters in the pre-trained network.

[0249] Step 2: Construct a projection network g(·), where the projection network may include a feature linear connection layer, a relu activation function layer, a batch normalization layer, and a target linear connection layer).

[0250] Step 3: Construct a classification network q(·), where the classification network may include a feature linear connection layer, a relu activation function layer, a batch normalization layer, and a target linear connection layer.

[0251] Step 4: Initialize the optimizer (stochastic gradient optimizer sgd, or adaptive momentum optimizer adam, etc.) and the sampled cosine annealing function.

[0252] Step 5: Initialize the temperature hyperparameters in each network.

[0253] Step 6: Initialize the prototype through k-means++

[0254] Finally, the model training process of the initial model may include the following steps:

[0255] Step 1: For each processing batch B in the data set (containing labeled data Unlabeled data ) for a given image instance x i, through two random data augmentation transformations T a and T b Enhance the data to obtain the converted data containing both labeled and unlabeled data and

[0256] Step 2: Through the feature extraction encoding network f(·), we get the feature representation pairs of two different views. in The feature representation pair for characterization learning is further obtained through the projection network g(·) in

[0257] Step 3: Calculate the instance-based self-supervised contrast loss for all representations based on formula one Then, based on Formula 2, the supervised comparison loss based on label data and instances is calculated. Further calculations indicate learning loss

[0258] Among them, formula 1 is as follows:

[0259]

[0260] Among them, formula 2 is as follows:

[0261]

[0262] Step 4: Based on formula 3, the soft label assignment of K categories is obtained through the classification network q(·) in

[0263] Among them, formula three is as follows:

[0264]

[0265] Step 5: Calculate the unlabeled classification loss based on the soft label assignment of all categories based on Formula 5 Then calculate the supervised classification loss based on formula 4 and label data Further calculation to obtain the classification learning loss

[0266] Among them, formula 4 is as follows:

[0267]

[0268] Among them, formula five is as follows:

[0269]

[0270] Step 6: Prototype-based And formula 6 calculates the prototype within the batch and The prototype contrast loss is further calculated based on formula 7

[0271] Among them, formula six is ​​as follows:

[0272]

[0273] Among them, formula seven is as follows:

[0274]

[0275] Step 7: Calculate the total loss based on Formula 8

[0276] Among them, formula eight is as follows:

[0277]

[0278] Step 8: Based on the optimizer (such as stochastic gradient optimizer sgd, or adaptive momentum optimizer adam, etc.), the cosine annealing function is sampled to update the learning rate lr in real time to minimize the total loss. The back-propagation algorithm is used to perform derivation and update the network parameters of the feature extraction network f(·), projection network g(·), and classification network q(·).

[0279] For the description of each character in Formula 1 to Formula 8, please refer to the explanation of each character in the above embodiment, so it will not be repeated here.

[0280] Step 9: After the training batch is updated, the prototype is updated according to the updated model parameters and the training set corresponding to the updated training batch.

[0281] Step 10: Perform a specific number of rounds of optimization to obtain the final model, and save the feature extraction network f(·) and classification network q(·) for image classification and clustering tasks.

[0282] In summary, the data classification model training method provided in this embodiment solves the problem of classifier bias to seen categories by introducing prototype contrast learning loss. First, pseudo labels are generated using k-means++ and assigned to each sample in the training data set, and then these pseudo labels are used to calculate the feature average of each training mini-batch as the prototype of the prototype contrast loss. Since all categories are assigned pseudo-supervisory signals, more balanced supervision can be provided for all categories to avoid excessive inclination to visible categories. On the other hand, the prototype is calculated using the relevant pseudo-labels of each mini-batch, rather than fixing the prototype for each round, so that more suitable supervision can be provided for each batch and the training process can be stabilized. Since the model needs to learn unbiased representation features, a prototype-based loss function is adopted. In the learning process of comparing the prototype, because the prototype is the average value of the features in each mini-batch, this gradient can be used to update the model, thereby learning better representation features.

[0283] In order to better implement the above method, the embodiment of the present application also provides a data classification model training device, which can be integrated in an electronic device, and the electronic device can be a terminal, a server, etc. Among them, the terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, a personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers.

[0284] For example, in this embodiment, the method of the embodiment of the present application will be described in detail by taking the data classification model training device specifically integrated in the server as an example.

[0285] For example, Figure 3 As shown, the data classification model training device may include: an enhancement unit 301, a feature extraction unit 302, a clustering unit 303, a comparison unit 304, a prototype determination unit 305 and a training unit 306, as follows:

[0286] An enhancement unit 301 is used to enhance the training samples in the training data to obtain a first enhanced sample and a second enhanced sample;

[0287] A feature extraction unit 302 is used to extract features from the first enhanced sample using an initial model to obtain a first feature, and to extract features from the second enhanced sample using the initial model to obtain a second feature;

[0288] A clustering unit 303 is used to cluster the training data to obtain multiple reference categories and a cluster center corresponding to each reference category;

[0289] A comparison unit 304 is used to compare the first feature with each cluster center to obtain a first probability that the first feature belongs to each reference category, and compare the second feature with each cluster center to obtain a second probability that the first feature belongs to each reference category;

[0290] A prototype determination unit 305 is used to perform feature calculation according to the first feature and each first probability to obtain a first prototype of each reference category, and perform feature calculation according to the second feature and each second probability to obtain a second prototype of each reference category;

[0291] The training unit 306 is used to perform prototype comparison learning on the initial model based on each first prototype and each second prototype to obtain a data classification model.

[0292] In some implementations, the training unit 306 includes:

[0293] A loss calculation subunit, used for performing a comparison loss calculation on each first prototype and each second prototype to obtain a prototype comparison loss;

[0294] The convergence subunit is used to converge the initial model based on the prototype contrast loss to obtain the data classification model.

[0295] In some embodiments, the loss calculation subunit is specifically configured to:

[0296] Selecting a reference category from the multiple reference categories as a target reference category, and selecting reference categories other than the target reference category from the multiple reference categories as candidate reference categories;

[0297] Calculate the initial loss of the target reference category according to the first prototype corresponding to each candidate reference category, the second prototype corresponding to each candidate reference category, and the second prototype corresponding to the target reference category;

[0298] Returning to the step of selecting a reference category from the multiple reference categories as the target reference category, until all the reference categories are selected as the target reference categories, and initial losses of multiple target reference categories are obtained;

[0299] Multiple initial losses are fused to obtain the prototype contrast loss.

[0300] In some implementations, the loss calculation subunit is further configured to:

[0301] Perform vector calculation based on the second prototype corresponding to the target reference category and the first prototype corresponding to each candidate reference category to obtain a first target vector;

[0302] Performing vector calculation based on the first prototype corresponding to each candidate reference category and the second prototype corresponding to each candidate reference category to obtain a second target vector;

[0303] Calculating a ratio of the second target vector to the first target vector to obtain a target ratio;

[0304] A plurality of target logarithms are obtained by performing logarithm processing on each target ratio through a logarithmic function, and the negative value of each target logarithm in the plurality of target logarithms is determined as the initial loss of the target reference category.

[0305] In some implementations, the loss calculation subunit is further configured to:

[0306] Calculate the product of the second prototype corresponding to the target reference category and the first prototype corresponding to each candidate reference category to obtain a plurality of initial vectors, and calculate the quotient of each of the initial vectors and a preset temperature hyperparameter to obtain a plurality of initial calculation vectors;

[0307] Calculate each of the initial calculation vectors using an exponential function to obtain a first calculation vector, and accumulate the multiple first calculation vectors to obtain a first target vector;

[0308] The step of performing vector calculation based on the first prototype corresponding to each candidate reference category and the second prototype corresponding to each candidate reference category to obtain a second target vector includes:

[0309] For each candidate reference category, multiply the first prototype corresponding to the candidate reference category by the second prototype corresponding to the candidate reference category and divide the result by the preset temperature hyperparameter to obtain a second calculation vector;

[0310] The second calculation vector is calculated by using an exponential function to obtain a second target vector.

[0311] In some embodiments, the initial model includes a projection network and a classification network, and a convergence subunit, specifically for:

[0312] Performing representation mapping on the training data through the projection network to obtain a mapping result, and determining a representation loss based on the mapping result;

[0313] The training data is classified by the classification network to obtain a classification result, and a classification loss is determined based on the classification result;

[0314] The prototype contrast loss, the representation loss and the classification loss are fused to obtain the target loss, and the initial model is converged based on the target loss to obtain the data classification model.

[0315] In some embodiments, the convergence subunit is further configured to:

[0316] Get the fusion weight of the prototype contrast loss, which is greater than 0 and less than or equal to 1;

[0317] Based on the fusion weight, the prototype contrast loss, representation loss and classification loss are fused to obtain the target loss.

[0318] In some embodiments, it is also used for:

[0319] Get the current training round of the initial model;

[0320] The fusion weight is determined according to the current training round, and the fusion weight is positively correlated with the current training round.

[0321] In some implementations, the training sample includes a labeled sample, and the convergence subunit is specifically further used for:

[0322] For each training sample in the training data, a first feature of the training sample is represented and mapped by the projection network to obtain a first mapping representation, and a second feature of the training sample is represented and mapped by the projection network to obtain a second mapping representation;

[0323] determining a first loss based on the first mapping representation and the second mapping representation;

[0324] For the labeled sample, the projection network is used to perform representation mapping on the first feature of the labeled sample to obtain a third mapping representation, and the projection network is used to perform representation mapping on the second feature of the labeled sample to obtain a fourth mapping representation;

[0325] determining a second loss based on the third mapping representation and the fourth mapping representation;

[0326] The first loss and the second loss are fused to obtain the representation loss.

[0327] In some implementations, the training sample includes a labeled sample, and the convergence subunit is specifically further used for:

[0328] For each training sample in the training data, classify the first feature of the training sample through the classification network to obtain a first classification result, and classify the second feature of the training sample through the classification network to obtain a second classification result;

[0329] Determining a third loss based on the first classification result and the second classification result;

[0330] For the labeled sample, classify the first feature of the labeled sample through the classification network to obtain a third classification result;

[0331] Based on the third classification result, determining a fourth loss;

[0332] The third loss and the fourth loss are fused to obtain the classification loss.

[0333] In some implementations, the prototype determination unit 305 includes a first prototype determination subunit and a second prototype determination subunit, wherein the first prototype determination subunit is used to:

[0334] For each of the reference categories, multiplying the first feature by a first probability that the first feature belongs to the reference category to obtain a first product corresponding to the first feature;

[0335] Accumulate the first products corresponding to all the first features to obtain a first accumulation result;

[0336] The first accumulated result is divided by the absolute value of the first accumulated result to obtain a first prototype of the reference category.

[0337] The first prototype determination subunit is used to: for each of the reference categories, multiply the second feature by the second probability that the second feature belongs to the reference category to obtain a second product corresponding to the second feature;

[0338] Accumulate the second products corresponding to all the second features to obtain a second accumulation result;

[0339] The second accumulated result is divided by the absolute value of the second accumulated result to obtain a second prototype of the reference category.

[0340] In some implementations, the clustering unit 303 is specifically configured to:

[0341] The training data is clustered using the K-means clustering algorithm.

[0342] In some embodiments, the device may further include:

[0343] The batch acquisition unit is used to acquire a set of training samples that have been pre-divided into training batches and a current training batch of the initial model.

[0344] The training data acquisition unit is used to acquire the training samples corresponding to the current training batch from the training sample set as the training data.

[0345] In this embodiment, the electronic device of this embodiment will be taken as an example for detailed description. Figure 4 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0346] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, an input module 404, and a communication module 405. Those skilled in the art will appreciate that Figure 4 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0347] The processor 401 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 402 and calling data stored in the memory 402, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. In some embodiments, the processor 401 may include one or more processing cores; in some embodiments, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0348] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, a data processing function, etc.); the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0349] The electronic device also includes a power supply 403 for supplying power to various components. In some embodiments, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 403 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0350] The electronic device may further include an input module 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0351] The electronic device may further include a communication module 405. In some embodiments, the communication module 405 may include a wireless module. The electronic device may perform short-range wireless transmission through the wireless module of the communication module 405, thereby providing the user with wireless broadband Internet access. For example, the communication module 405 may be used to help the user send and receive emails, browse web pages, and access streaming media.

[0352] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402 to implement various functions.

[0353] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0354] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0355] To this end, an embodiment of the present application provides a computer-readable storage medium, which stores multiple instructions, and the instructions can be loaded by a processor to execute the steps in any data classification model training method provided in the embodiment of the present application.

[0356] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0357] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the above embodiment.

[0358] Since the instructions stored in the storage medium can execute the steps in any data classification model training method provided in the embodiments of the present application, the beneficial effects that can be achieved by any data classification model training method provided in the embodiments of the present application can be achieved. Please see the previous embodiments for details and will not be repeated here.

[0359] The above is a detailed introduction to a data classification model training method, device, electronic device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data classification model training method, characterized in that: include: Performing enhancement processing on the training samples in the training data to obtain a first enhanced sample and a second enhanced sample; Extracting features from the first enhanced sample using the initial model to obtain a first feature, and extracting features from the second enhanced sample using the initial model to obtain a second feature; Clustering the training data to obtain multiple reference categories and a cluster center of a cluster corresponding to each reference category; Comparing the first feature with each cluster center to obtain a first probability that the first feature belongs to each reference category, and comparing the second feature with each cluster center to obtain a second probability that the first feature belongs to each reference category; Performing feature calculation according to the first feature and each first probability to obtain a first prototype of each reference category, and performing feature calculation according to the second feature and each second probability to obtain a second prototype of each reference category; Based on each first prototype and each second prototype, prototype comparison learning is performed on the initial model to obtain a data classification model.

2. The data classification model training method according to claim 1, characterized in that: The method of performing prototype comparison learning on the initial model based on each first prototype and each second prototype to obtain a data classification model includes: Calculate the contrast loss for each first prototype and each second prototype to obtain the prototype contrast loss; Based on the prototype contrast loss, the initial model is converged to obtain the data classification model.

3. The data classification model training method according to claim 2, characterized in that: The step of calculating the contrast loss for each first prototype and each second prototype to obtain the prototype contrast loss includes: Selecting a reference category from the multiple reference categories as a target reference category, and selecting reference categories other than the target reference category from the multiple reference categories as candidate reference categories; Calculating the initial loss of the target reference category according to the first prototype corresponding to each candidate reference category, the second prototype corresponding to each candidate reference category, and the second prototype corresponding to the target reference category; Returning to the step of selecting a reference category from the multiple reference categories as the target reference category, until all the reference categories are selected as the target reference categories, and initial losses of the multiple target reference categories are obtained; A plurality of the initial losses are fused to obtain the prototype contrast loss.

4. The data classification model training method according to claim 3, characterized in that: The calculating the initial loss of the target reference category according to the first prototype corresponding to each candidate reference category, the second prototype corresponding to each candidate reference category, and the second prototype corresponding to the target reference category includes: Performing vector calculation based on the second prototype corresponding to the target reference category and the first prototype corresponding to each candidate reference category to obtain a first target vector; Performing vector calculation based on the first prototype corresponding to each candidate reference category and the second prototype corresponding to each candidate reference category to obtain a second target vector; Calculating a ratio of the second target vector to the first target vector to obtain a target ratio; A plurality of target logarithms are obtained by performing logarithmic processing on each of the target ratios through a logarithmic function, and the negative value of each target logarithm in the plurality of target logarithms is determined as the initial loss of the target reference category.

5. The data classification model training method according to claim 4, characterized in that: The step of performing vector calculation based on the second prototype corresponding to the target reference category and the first prototype corresponding to each candidate reference category to obtain a first target vector includes: Calculating the product between the second prototype corresponding to the target reference category and the first prototype corresponding to each candidate reference category to obtain a plurality of initial vectors, and calculating the quotient of each of the initial vectors and a preset temperature hyperparameter to obtain a plurality of initial calculation vectors; Calculating each of the initial calculation vectors by using an exponential function to obtain a first calculation vector, and accumulating the multiple first calculation vectors to obtain a first target vector; The step of performing vector calculation based on the first prototype corresponding to each candidate reference category and the second prototype corresponding to each candidate reference category to obtain a second target vector includes: For each candidate reference category, multiply the first prototype corresponding to the candidate reference category by the second prototype corresponding to the candidate reference category and divide the result by the preset temperature hyperparameter to obtain a second calculation vector; The second calculation vector is calculated by using an exponential function to obtain a second target vector.

6. The data classification model training method according to claim 2, characterized in that: The initial model includes a projection network and a classification network. Based on the prototype contrast loss, the initial model is converged to obtain the data classification model, including: Performing representation mapping on the training data through the projection network to obtain a mapping result, and determining a representation loss based on the mapping result; Classify the training data through the classification network to obtain a classification result, and determine the classification loss based on the classification result; The prototype contrast loss, the representation loss and the classification loss are fused to obtain a target loss, and the initial model is converged based on the target loss to obtain the data classification model.

7. The data classification model training method according to claim 6, characterized in that: The step of fusing the prototype contrast loss, the representation loss, and the classification loss to obtain the target loss includes: Obtaining a fusion weight of the prototype contrast loss, where the fusion weight is greater than 0 and less than or equal to 1; Based on the fusion weight, the prototype contrast loss, the representation loss and the classification loss are fused to obtain the target loss.

8. The data classification model training method according to claim 7, characterized in that: The obtaining of the fusion weight of the prototype contrast loss includes: Obtaining the current training round of the initial model; The fusion weight is determined according to the current training round, and the fusion weight is positively correlated with the current training round.

9. The data classification model training method according to claim 6, characterized in that: The training samples include labeled samples, and the representation mapping is performed on the training data by the projection network to obtain a mapping result, and the representation loss is determined based on the mapping result, including: For each training sample in the training data, a first feature of the training sample is represented and mapped by the projection network to obtain a first mapping representation, and a second feature of the training sample is represented and mapped by the projection network to obtain a second mapping representation; determining a first loss based on the first mapping representation and the second mapping representation; For the labeled sample, the projection network is used to perform representation mapping on the first feature of the labeled sample to obtain a third mapping representation, and the projection network is used to perform representation mapping on the second feature of the labeled sample to obtain a fourth mapping representation; determining a second loss based on the third mapping representation and the fourth mapping representation; The first loss and the second loss are combined to obtain the representation loss.

10. The data classification model training method according to claim 6, characterized in that: The training samples include labeled samples, and the classification network is used to classify the training data to obtain a classification result, and the classification loss is determined based on the classification result, including: For each training sample in the training data, classify the first feature of the training sample through the classification network to obtain a first classification result, and classify the second feature of the training sample through the classification network to obtain a second classification result; determining a third loss based on the first classification result and the second classification result; For the labeled sample, classify the first feature of the labeled sample through the classification network to obtain a third classification result; Determining a fourth loss based on the third classification result; The third loss and the fourth loss are combined to obtain the classification loss.

11. The data classification model training method according to any one of claims 1 to 10, characterized in that: The performing feature calculation according to the first feature and each first probability to obtain a first prototype of each reference category includes: For each of the reference categories, multiplying the first feature by a first probability that the first feature belongs to the reference category to obtain a first product corresponding to the first feature; Accumulate the first products corresponding to all the first features to obtain a first accumulation result; The first accumulated result is divided by the absolute value of the first accumulated result to obtain a first prototype of the reference category. The performing feature calculation according to the second feature and each second probability to obtain a second prototype of each reference category includes: For each of the reference categories, multiplying the second feature by a second probability that the second feature belongs to the reference category to obtain a second product corresponding to the second feature; Accumulate the second products corresponding to all the second features to obtain a second accumulation result; The second accumulated result is divided by the absolute value of the second accumulated result to obtain a second prototype of the reference category.

12. The data classification model training method according to any one of claims 1 to 10, characterized in that: The clustering of the training data comprises: The training data is clustered using a K-means clustering algorithm.

13. The data classification model training method according to any one of claims 1 to 10, characterized in that: Before performing enhancement processing on the training samples in the training data, the method further includes: Obtaining a training sample set that is pre-divided into training batches and a current training batch of the initial model; A training sample corresponding to the current training batch is obtained from the training sample set as the training data.

14. A data classification model training device, characterized in that: include: An enhancement unit, configured to perform enhancement processing on the training samples in the training data to obtain a first enhanced sample and a second enhanced sample; A feature extraction unit, configured to extract features from the first enhanced sample using an initial model to obtain a first feature, and to extract features from the second enhanced sample using the initial model to obtain a second feature; A clustering unit, used to cluster the training data to obtain a plurality of reference categories and a cluster center of a cluster corresponding to each reference category; a comparison unit, configured to compare the first feature with each cluster center to obtain a first probability that the first feature belongs to each reference category, and to compare the second feature with each cluster center to obtain a second probability that the first feature belongs to each reference category; a prototype determination unit, configured to perform feature calculation according to the first feature and each first probability to obtain a first prototype of each reference category, and perform feature calculation according to the second feature and each second probability to obtain a second prototype of each reference category; The training unit is used to perform prototype comparison learning on the initial model based on each first prototype and each second prototype to obtain a data classification model.

15. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the steps in the data classification model training method as described in any one of claims 1 to 12.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, which are suitable for loading by a processor to execute the steps in the data classification model training method according to any one of claims 1 to 12.

Images