Sample determination method and apparatus

Abstract

Embodiments of the present specification provide a sample determination method and device, wherein the sample determination method comprises: obtaining a to-be-processed sample set, wherein the to-be-processed sample set comprises a first initial sample set and a second initial sample set, the number of second initial samples in the second initial sample set is greater than the number of first initial samples in the first initial sample set; determining sampling probabilities corresponding to the first initial samples and the second initial samples in the to-be-processed sample set respectively; sampling the first initial samples and the second initial samples in the to-be-processed sample set based on the sampling probabilities respectively to obtain first to-be-determined samples corresponding to the first initial samples and second to-be-determined samples corresponding to the second initial samples, wherein the number of the first to-be-determined samples is the same as the number of the second to-be-determined samples; and performing data enhancement on the first to-be-determined samples and the second to-be-determined samples to obtain a target sample set.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of artificial intelligence technology, and in particular to a method for determining samples. Background Technology

[0002] With the development of science and technology, artificial intelligence technology is increasingly penetrating people's daily lives, such as the application of various neural networks. For the same neural network, it is often necessary to train multiple different tasks. However, due to the differences in datasets, the similarity between new and old data is low, which may lead to catastrophic forgetting problems in the trained network model, that is, the network model forgets old knowledge after learning new knowledge.

[0003] To address this issue, incremental learning is often used to train network models. Incremental learning-like tasks aim to more closely resemble real-world data learning patterns, characterized by a continuous flow of data. This requires models to not only learn new class knowledge effectively but also retain knowledge from older classes. The challenge lies in the fact that models often only have abundant training samples of new classes at the current stage, while storage capacity and cost limitations often limit the storage of only a small number of older class samples. Furthermore, because models impose stricter restrictions on learning new class knowledge and maintain older class knowledge through methods like distillation, the old model exhibits varying degrees of preference for different new classes. These two factors lead to an imbalance in incremental learning, impacting its effectiveness.

[0004] Therefore, a solution to the aforementioned technical problems is urgently needed. Summary of the Invention

[0005] In view of this, embodiments of this specification provide a sample determination method. One or more embodiments of this specification also relate to a sample determination apparatus, a method and apparatus for training a classification model, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0006] According to a first aspect of the embodiments of this specification, a sample determination method is provided, comprising:

[0007] Obtain a set of samples to be processed, wherein the set of samples to be processed includes a first initial sample set and a second initial sample set, and the number of second initial samples in the second initial sample set is greater than the number of first initial samples in the first initial sample set;

[0008] Determine the sampling probabilities corresponding to the first initial sample and the second initial sample in the set of samples to be processed;

[0009] Based on the sampling probability, the first initial sample and the second initial sample are sampled from the set of samples to be processed to obtain the first sample to be determined corresponding to the first initial sample and the second sample to be determined corresponding to the second initial sample, wherein the number of the first sample to be determined is the same as the number of the second sample to be determined.

[0010] Data augmentation is performed on the first and second samples to be determined to obtain the target sample set.

[0011] According to a second aspect of the embodiments of this specification, a sample determining apparatus is provided, comprising:

[0012] The acquisition module is configured to acquire a set of samples to be processed, wherein the set of samples to be processed includes a first initial sample set and a second initial sample set, and the number of second initial samples in the second initial sample set is greater than the number of first initial samples in the first initial sample set;

[0013] The determination module is configured to determine the sampling probabilities corresponding to the first initial sample and the second initial sample in the set of samples to be processed, respectively.

[0014] The sampling module is configured to sample the first initial sample and the second initial sample in the set of samples to be processed based on the sampling probability, and obtain the first sample to be determined corresponding to the first initial sample and the second sample to be determined corresponding to the second initial sample, wherein the number of the first sample to be determined is the same as the number of the second sample to be determined.

[0015] The enhancement module is configured to perform data enhancement on the first sample to be determined and the second sample to be determined to obtain a target sample set.

[0016] According to a third aspect of the embodiments of this specification, a method for training a classification model is provided, comprising:

[0017] Receive sample training data and the sample categories corresponding to the sample training data;

[0018] The sample training data is input into the classification model to obtain the predicted category of the sample training data;

[0019] The loss value of the classification model is calculated based on the sample category and the predicted category;

[0020] The model parameters of the classification model are adjusted based on the loss value, and the classification model is trained continuously until the training stopping condition is met.

[0021] According to a fourth aspect of the embodiments of this specification, a training apparatus for a classification model is provided, comprising:

[0022] The receiving module is configured to receive sample training data and the sample categories corresponding to the sample training data;

[0023] The input module is configured to input the sample training data into the classification model to obtain the predicted category of the sample training data;

[0024] The calculation module is configured to calculate the loss value of the classification model based on the sample category and the predicted category;

[0025] The training module is configured to adjust the model parameters of the classification model based on the loss value and continue training the classification model until the training stopping condition is met.

[0026] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0027] Memory and processor;

[0028] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the above-described sample determination method or classification model training method.

[0029] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the sample determination method or the training method of the classification model described above.

[0030] According to a seventh aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described sample determination method or classification model training method.

[0031] The sample determination method provided in this specification includes: obtaining a sample set to be processed, wherein the sample set to be processed includes a first initial sample set and a second initial sample set, wherein the number of second initial samples in the second initial sample set is greater than the number of first initial samples in the first initial sample set; determining the sampling probabilities corresponding to the first initial samples and the second initial samples in the sample set to be processed, respectively; sampling the first initial samples and the second initial samples in the sample set to be processed based on the sampling probabilities, respectively, to obtain a first undetermined sample corresponding to the first initial sample and a second undetermined sample corresponding to the second initial sample, wherein the number of the first undetermined samples and the number of the second undetermined samples are the same; performing data augmentation on the first undetermined samples and the second undetermined samples to obtain a target sample set.

[0032] One embodiment of this specification implements two sampling operations on a first initial sample and a second initial sample in a set of samples to be processed. By setting the same initial sampling probability, both the first initial sample and the second initial sample in the set of samples to be processed can participate in the subsequent sampling and training process. Then, based on the model parameters of the classification model in the previous stage and the number of sample categories, the sampling probability corresponding to different samples is determined, so that different initial samples have different sampling probabilities. Sampling is performed based on different sampling probabilities to achieve a balanced sample distribution between the first initial sample set and the second initial sample set. Attached Figure Description

[0033] Figure 1 This is a schematic diagram illustrating the application of a sample determination method provided in one embodiment of this specification;

[0034] Figure 2 This is a flowchart of a sample determination method provided in one embodiment of this specification;

[0035] Figure 3 This is a flowchart illustrating the processing steps of a sample determination method provided in one embodiment of this specification.

[0036] Figure 4 This is a schematic diagram of the structure of a sample determination device provided in one embodiment of this specification;

[0037] Figure 5 This is a flowchart illustrating a training method for a classification model provided in one embodiment of this specification;

[0038] Figure 6 This is a schematic diagram of the structure of a training device for a classification model provided in one embodiment of this specification;

[0039] Figure 7This is a flowchart illustrating an image classification method provided in one embodiment of this specification;

[0040] Figure 8 This is a schematic diagram of the structure of an image classification device provided in one embodiment of this specification;

[0041] Figure 9 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0042] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0043] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0044] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0045] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0046] Class Incremental Learning: This requires the model to retain previously learned knowledge while learning the knowledge at the current stage.

[0047] Imbalance learning is an imbalance phenomenon in incremental learning caused by the old model's preference for different new classes and the discrepancy in the number of samples between the old and new classes and between the new classes.

[0048] Catastrophic forgetting: In a sequence of tasks that are unlabeled, may switch randomly, and may not be repeated for a long time, when an AI learns the current task B, it suddenly loses its knowledge of the previous task A.

[0049] Downsampling: For an imbalanced dataset, make the sample size in the target value the same, and use the sample size of the side with less data as the benchmark.

[0050] In practical applications, artificial intelligence technology is increasingly penetrating people's daily lives, for example, through the application of various neural networks. For the same neural network, it is often necessary to train it for multiple different tasks. However, due to the differences in datasets, the low similarity between new and old data can lead to catastrophic forgetting problems in the trained network model; that is, the network model forgets old knowledge after learning new knowledge.

[0051] To address this issue, incremental learning is often used to train network models. Incremental learning-like tasks aim to more closely resemble real-world data learning patterns, characterized by a continuous flow of data. This requires models to not only learn new class knowledge effectively but also retain knowledge from older classes. The challenge lies in the fact that models often only have abundant training samples of new classes at the current stage, while storage capacity and cost limitations often limit the storage of only a small number of older class samples. Furthermore, because models impose stricter restrictions on learning new class knowledge and maintain older class knowledge through methods like distillation, the old model exhibits varying degrees of preference for different new classes. These two factors lead to an imbalance in incremental learning, impacting its effectiveness.

[0052] This specification provides a sample determination method, and also relates to a sample determination apparatus, a method and apparatus for training a classification model, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0053] See Figure 1 , Figure 1 A schematic diagram illustrating an application of a sample determination method according to an embodiment of this specification is shown. Figure 1 As shown, the sampling probability is calculated for each sample in the sample set to be processed. Based on the corresponding sampling probability, sampling is performed in the sample set to obtain the first and second samples to be determined. Then, data augmentation algorithms are used to augment the first and second samples to obtain the target sample set.

[0054] Furthermore, the classification model at the current stage is incrementally trained based on the target sample set. Specifically, the classification model at the current stage is adjusted accordingly by calculating the distillation loss function and the classification loss function, until the training conditions of the model are met.

[0055] The sample determination method provided in this specification involves sampling the first and second initial samples in the sample set to be processed twice. By setting the same initial sampling probability, both the first and second initial samples in the sample set to be processed can participate in the subsequent sampling and training process. Then, based on the model parameters of the classification model in the previous stage and the number of sample categories, the sampling probability corresponding to different samples is determined, so that different initial samples have different sampling probabilities. Sampling is performed based on different sampling probabilities to achieve a balanced sample distribution between the first and second initial sample sets.

[0056] See Figure 2 , Figure 2 A flowchart of a sample determination method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0057] Step 202: Obtain the sample set to be processed, wherein the sample set to be processed includes a first initial sample set and a second initial sample set, and the number of second initial samples in the second initial sample set is greater than the number of first initial samples in the first initial sample set.

[0058] The set of samples to be processed refers to the set of samples waiting to be processed, specifically the training data used to train the classification model in the current stage, such as sample images. The set of samples to be processed includes a first initial sample set and a second initial sample set. The first initial sample set can be understood as the training data used to train the classification model in the previous stage; the second initial sample set can be understood as newly added training data based on the first initial sample set, used to train the classification model in the current stage together with the first initial sample set.

[0059] For example, in an image classification task, the training data of sample images used to train the image classification model of the previous stage is called the first initial sample, and the set of training data of sample images used to train the image classification model of the previous stage is called the first initial sample set. Based on the first initial sample set, the newly added image sample training data used to train the image classification model of the current stage is called the second initial sample, and the set of newly added sample image training data is called the second initial sample set.

[0060] Specifically, obtaining the set of samples to be processed involves obtaining the first initial sample set and the second initial sample set. Due to the model's own storage capacity and to reduce storage costs, the number of second initial samples will be greater than the number of first initial samples.

[0061] Step 204: Determine the sampling probabilities corresponding to the first initial sample and the second initial sample in the set of samples to be processed.

[0062] Here, sampling probability refers to the probability that the first and second initial samples in the set of samples to be processed can be collected and applied to the training process of the classification model. For example, if the sampling probability of the first initial sample a is 30%, it means that there is a 30% chance that the first initial sample a will be collected and used to train the classification model.

[0063] Since the number of second initial samples in the second initial sample set is much larger than the number of first initial samples in the first initial sample set, in order to ensure the balance of training samples, the number of second initial samples needs to be the same as the number of first initial samples. Therefore, different sampling probabilities need to be set for different samples to be processed.

[0064] In one embodiment provided in this specification, determining the sampling probabilities corresponding to the first initial sample and the second initial sample in the sample set to be processed includes:

[0065] Set initial sampling probabilities for the first initial sample and the second initial sample respectively;

[0066] The first initial sample and the second initial sample are sampled respectively based on the initial sampling probability;

[0067] Based on the sampling results, determine the sampling weights corresponding to the first initial sample and the second initial sample, respectively;

[0068] The sampling probability is determined based on the sampling weight.

[0069] Specifically, the initial sampling probability refers to the sampling probability set when performing the first sampling on the first and second initial samples in the sample set to be processed after obtaining the sample set. It is important to note that the initial sampling probability set for the first and second initial samples is the same, that is, the first sampling on the first and second initial samples is performed with equal probability.

[0070] Taking image samples as an example, the acquired set of image samples to be processed includes a first initial image sample set A and a second initial image sample set B. The first initial image sample set A contains 20 first initial image samples, and the second initial image sample set B contains 50 second initial image samples, resulting in a total of 70 image samples to be processed. These 70 image samples are given the same initial sampling probability of 60%, meaning there is a 60% probability that each of these 70 image samples can be collected and used in the subsequent training process. Based on this 60% initial sampling probability, these 70 image samples are sampled to obtain sampled image samples, and the number of sampled image samples is 70.

[0071] Weights refer to the relative importance of the first and second initial samples in the set of samples to be processed. Sampling weights, on the other hand, refer to the relative importance of the first and second initial samples in subsequent training of the classification model. Higher sampling weights indicate higher relative importance, and lower sampling weights indicate lower relative importance. Therefore, to ensure that samples with high sampling weights are collected, they need to have a high sampling probability; for example, a sampling weight of 0.5 with a sampling probability of 50%, or a sampling weight of 0.8 with a sampling probability of 80%, etc.

[0072] Since the samples are independently distributed and the number of samples in each sample set is not the same and the distribution is uneven, it is necessary to set different sampling weights for different samples and determine the sampling probability of the sample based on the sampling weights.

[0073] Furthermore, to ensure a uniform distribution of the number of initial samples between the first and second initial samples, the samples with fewer samples need to have a higher sampling probability, so that the fewer samples are collected as much as possible. Continuing the previous example, the second initial image sample set B contains 50 second initial image samples. These 50 second initial image samples include four sample categories: "birds," "jellyfish," "crabs," and "lions." The second initial image samples corresponding to the "birds" category are 20, those corresponding to the "jellyfish" category are 1, those corresponding to the "crabs" category are 5, and those corresponding to the "lions" category are 24. Therefore, the sampling probabilities of the second initial image samples corresponding to the four sample categories, from highest to lowest, are: "jellyfish," "crabs," "birds," and "lions."

[0074] Furthermore, taking 100 sampling times as an example, the sampling probability of the second initial image sample corresponding to the "jellyfish" sample category is set to 25% (1 image), the sampling probability of the second initial image sample corresponding to the "crab" sample category is set to 5% (5 images), the sampling probability of the second initial image sample corresponding to the "bird" sample category is set to 1.25% (20 images), and the sampling probability of the second initial image sample corresponding to the "lion" sample category is set to 1% (24 images). Sampling is performed based on the sampling probabilities of these four corresponding second initial image samples, resulting in 25 second initial image samples for each of the four sample categories: "birds," "jellyfish," "crab," and "lion."

[0075] Specifically, by setting initial sampling probabilities for the first and second initial samples, and sampling is performed on the first and second initial samples based on these initial sampling probabilities, the first and second initial samples can be collected with equal probability for subsequent use. Based on the sampled first and second initial samples, corresponding sampling weights are determined, which establishes the relative importance of each initial sample. The sampling probabilities are then determined based on these sampling weights, ensuring that the sampling weights are collected as much as possible according to the relative importance of the initial samples.

[0076] The process of determining the sampling weights can be implemented in the following way:

[0077] In one embodiment provided in this specification, determining the sampling weights corresponding to the first initial sample and the second initial sample based on the sampling results includes:

[0078] Based on the sampling results, a first initial sample of the target is determined in the first initial sample, and a second initial sample of the target is determined in the second initial sample;

[0079] Determine the number of sample categories corresponding to the first initial sample category in the first initial sample set;

[0080] Obtain the model parameters of the classification model from the previous training phase;

[0081] The target first sampling weight and the target second sampling weight corresponding to the target first initial sample are determined based on the number of sample categories and the model parameters of the classification model in the previous training phase.

[0082] Here, the target first initial sample refers to the first initial sample in the first initial sample set that requires a determination of its sampling weight, and the target second initial sample refers to the second initial sample in the second initial sample set that requires a determination of its sampling weight. For example, in the first initial sample set A, if the first initial sample a needs to have its sampling weight determined, then the first initial sample a is the target first initial sample. Similarly, in the second initial sample set B, if the second initial sample b needs to have its sampling weight determined, then the second initial sample b is the target second initial sample.

[0083] A classification model is a model that classifies samples based on samples from different stages. Specifically, the model parameters of the classification model from the previous training stage refer to the classifier parameters, feature extractor parameters, and temperature parameters.

[0084] The first sampling weight of the target refers to the sampling weight corresponding to the first initial sample of the target, and the second sampling weight of the target refers to the sampling weight corresponding to the second initial sample of the target.

[0085] Specifically, after sampling the first initial sample and the second initial sample in the sample set to be processed according to the initial sampling probability, so as to obtain the first initial sample and the second initial sample, the first initial sample and the second initial sample that need to be sampled are determined in the sampled first initial sample set and the second initial sample set, that is, the target first initial sample and the target second initial sample are determined.

[0086] Furthermore, the number of sample categories corresponding to the first initial sample in the first initial sample set is determined. Based on this, when determining the predicted sample categories corresponding to the target first initial sample and the target second initial sample, the model parameters of the classification model in the previous training stage are used. Based on the model parameters of the classification model in the previous training stage, the target first sampling weight corresponding to the target first initial sample and the target second sampling weight corresponding to the target second initial sample are calculated.

[0087] For example, to determine the sampling weight corresponding to the first initial sample x in the first initial sample set, the first initial sample x is determined as the target first initial sample. If the first initial sample set includes four sample categories: "cats," "geese," "fish," and "birds," then the number of sample categories in the first initial sample set is determined to be 4. When determining the predicted sample category corresponding to the first initial sample x, the model parameters of the classification model in the previous training stage are used. Based on the number of categories and the model parameters of the classification model in the previous stage, the energy score corresponding to the first initial sample x is calculated. Then, the sampling weight corresponding to the first initial sample x is determined based on the energy score.

[0088] Furthermore, the sampling weights corresponding to the initial samples can be calculated using the following formula (1).

[0089]

[0090] Where E is the energy score of the initial sample in the previous stage classification model, x is the initial sample input to the previous stage classification model, and θ is the parameter of the feature extractor of the previous stage classification model. Here, T represents the parameters of the classifier in the previous stage classification model, T is the temperature parameter (preferably set to 1 in this specification), i is the sample category corresponding to the initial sample, and k is the number of sample categories, with i ranging from {1, 2, 3, ..., k}. The parameters of the classifier are used to predict the initial sample x as class i in the previous stage classification model.

[0091] Furthermore, after calculating the energy scores corresponding to the first initial sample and the second initial sample, the first sampling weight corresponding to the first initial sample and the second sampling weight corresponding to the second initial sample can be determined based on the energy scores.

[0092] The sampling weights are calculated using the following formula (2):

[0093] ω=1-E (2)

[0094] Where ω is the sampling weight.

[0095] Specifically, after determining the energy score of the initial sample, the sampling weight of the initial sample can be determined according to the above formula (2).

[0096] By determining the energy scores of the first and second initial samples in the classification model of the previous training stage, the imbalance between the first and second initial samples in the sample set to be processed can be identified. Further determining the sampling weights corresponding to the first and second initial samples allows us to know the relative importance of different samples in the sample set to be processed, thereby setting different sampling probabilities for different samples for subsequent sampling processes.

[0097] The sample determination method provided in this specification involves sampling the first and second initial samples in the sample set to be processed twice. By setting the same initial sampling probability, both the first and second initial samples in the sample set to be processed can participate in the subsequent sampling and training process. Then, based on the model parameters of the classification model in the previous stage and the number of sample categories, the sampling probability corresponding to different samples is determined, so that different initial samples have different sampling probabilities. Sampling is performed based on different sampling probabilities to achieve a balanced sample distribution between the first and second initial sample sets.

[0098] Step 206: Based on the sampling probability, sample the first initial sample and the second initial sample in the set of samples to be processed to obtain the first undetermined sample corresponding to the first initial sample and the second undetermined sample corresponding to the second initial sample, wherein the number of the first undetermined sample and the number of the second undetermined sample are the same.

[0099] Wherein, the first sample to be determined refers to the sample data obtained after sampling the first initial sample based on the sampling probability; the second sample to be determined refers to the sample data obtained after sampling the second initial sample based on the sampling probability.

[0100] Specifically, after determining the sampling probability corresponding to different sample data based on the sampling weights corresponding to different sample data, the first initial sample and the second initial sample in the sample set to be processed can be downsampled according to the sampling probability, so that the number of the first sample to be determined is the same as the number of the second sample to be determined.

[0101] Step 208: Perform data augmentation on the first sample to be determined and the second sample to be determined to obtain the target sample set.

[0102] The target sample set refers to the sample data obtained after data augmentation of the first and second samples to be determined.

[0103] The number of first and second undetermined samples obtained by sampling based on sampling probability is the same, and the number of first undetermined samples is the same across all sample categories included in the first undetermined samples. Correspondingly, the number of second undetermined samples is the same across all sample categories included in the second undetermined samples. Furthermore, the number of first undetermined samples corresponding to the sample categories in the first undetermined samples is the same as the number of second undetermined samples corresponding to the sample categories in the second undetermined samples.

[0104] In other words, this specification achieves a balance in sample distribution between the first and second samples to be determined, and among the sample categories of the second sample. However, the first and second samples to be determined belong to different sample data, which may result in differences in training difficulty. Therefore, after achieving a balance in sample distribution between the first and second samples to be determined, and among the sample categories of the second sample, it is still necessary to overcome the differences in sample features between the first and second samples to be determined.

[0105] Therefore, in one embodiment provided in this specification, the step of performing data augmentation on the first sample to be determined and the second sample to be determined to obtain a target sample set includes:

[0106] Two adjacent target samples to be determined are identified from the first sample to be determined and the second sample to be determined, wherein the two adjacent target samples to be determined correspond to the same sample category;

[0107] Obtain the preset correction coefficient;

[0108] Based on the preset correction coefficient and the two adjacent target samples to be determined, determine the target samples corresponding to the two adjacent target samples to be determined;

[0109] Add the target sample to the target sample set.

[0110] Specifically, the two adjacent target samples to be determined refer to the sample data of the same sample category that are determined in the first and second samples to be determined.

[0111] Since the first and second undetermined samples are already evenly distributed, there is no need to distinguish between them during data augmentation. That is, there is no need to differentiate whether the undetermined sample was used to train the classification model from the previous stage, or whether it is newly added to the existing data for training the current stage's classification model. Therefore, two adjacent undetermined samples can be two first undetermined samples, two second undetermined samples, or one first undetermined sample and one second undetermined sample.

[0112] Specifically, two adjacent target samples of the same category are identified from the first and second undetermined samples, and the target samples corresponding to the two adjacent target samples are obtained according to the preset correction coefficient.

[0113] The interpolation algorithm will be used as an example for explanation. For the specific calculation method, please refer to the following formula (3):

[0114] x new =λx j +(1-λ)x j+1 (3)

[0115] Where, x j For the j-th sample to be determined, x j+1 Let x be the (j+1)th sample to be determined, λ be the preset correction coefficient, and x be the value of x. new This refers to the target sample obtained after data augmentation of the j-th and (j+1)-th samples to be determined. Correspondingly, the sample category labels corresponding to the first and second samples to be determined can also be obtained using the algorithm described above.

[0116] By performing data augmentation on the first and second samples to be determined after sampling, more homogeneous sample data can be obtained, reducing the training difficulty between the first and second samples. It should be noted that, regarding data augmentation of the first and second samples to be determined, and not limited to the interpolation algorithm described above, other methods can also be selected according to actual needs, such as extrapolation algorithms, etc., and this specification does not impose any limitations here.

[0117] Preferably, the process of sampling the first initial sample and the second initial sample according to the sampling probability, and the process of data augmentation of the first sample to be determined and the second sample to be determined, can be implemented by constructing a dual-branch network model.

[0118] Specifically, the first initial sample and the second initial sample are input into the first branch network of the dual-branch network model to obtain the sampling probabilities corresponding to the first initial sample and the second initial sample output by the first branch network. Based on the sampling probabilities, the first initial sample and the second initial sample are sampled in the second branch network to obtain the first undetermined sample and the second undetermined sample. Data augmentation is then performed on the first undetermined sample and the second undetermined sample to obtain the target sample output by the second branch network model.

[0119] By constructing a dual-branch network model, it is also possible to achieve balance between the first and second initial samples and between different sample categories, thus solving the imbalance phenomenon in class incremental learning.

[0120] The sample determination method provided in this specification first determines the energy scores corresponding to the first and second initial samples. The energy scores are used to identify any imbalance between the first and second initial samples. Based on the energy scores, sampling weights are calculated for different samples to determine their relative importance. Then, sampling probabilities are determined according to these sampling weights, allowing downsampling of the first and second initial samples to achieve balance between them and between different sample categories. Finally, data augmentation is performed on the sampled first and second initial samples to reduce the training difficulty between them.

[0121] In one embodiment provided in this specification, after performing data augmentation on the first sample to be determined and the second sample to be determined to obtain a target sample set, the method further includes:

[0122] Obtain the target sample category set corresponding to the target sample set and the classification model from the previous training stage;

[0123] The classification model for the current training stage is trained based on the target sample set, the target sample category set, and the classification model from the previous training stage.

[0124] The target sample category set refers to the set of actual sample categories corresponding to the target samples in the target sample set.

[0125] After determining the target sample set, the classification model for the current stage can be trained based on the target sample set. Specifically, the target sample category set corresponding to the target sample set and the classification model from the previous stage are obtained, and the classification model for the current training stage is trained based on the target sample set, the target sample category set, and the classification model from the previous training stage.

[0126] Furthermore, in one embodiment provided in this specification, training the classification model for the current training stage based on the target sample set, the target sample category set, and the classification model from the previous training stage includes:

[0127] A reference target sample is determined from the target sample set, and a reference target sample category corresponding to the reference target sample is determined from the target sample category set.

[0128] The reference target sample is input into the classification model of the current training stage to obtain the second classification sample feature vector and the predicted target sample category output by the classification model of the current training stage, wherein the classification model of the current training stage is used to classify the sample categories of the first initial sample set and the second initial sample set.

[0129] The model loss value is calculated based on the reference target sample, the feature vector of the second classification sample, the predicted target sample category, and the reference target sample category.

[0130] The classification model in the current training phase is adjusted based on the model loss value until the training stops.

[0131] Here, the reference target sample refers to the sample data determined in the target sample set for training the classification model in the current stage, and the reference target sample category refers to the sample category corresponding to the reference target sample determined in the target sample category set. For example, if a target sample set R is obtained, and the sample data used for training the classification model in the current stage is determined to be target sample 1 in the target sample set R, then target sample 1 is the reference target sample; based on target sample 1, the sample category corresponding to target sample 1 is determined to be L in the target sample category set, then sample category L is the reference target sample category.

[0132] The second classification sample feature vector specifically refers to the feature vector of the reference target sample extracted by the classification model at the current stage; the predicted target sample category is the sample category output by inputting the reference target sample into the classification model at the current stage.

[0133] Specifically, after obtaining the reference target sample, the feature vector of the second classification sample, the category of the reference target sample, the category of the predicted target sample, and the category of the reference target sample, the model loss value of the classification model at the current stage can be calculated using the reference target sample, the feature vector of the second classification sample, the category of the reference target sample, the category of the predicted target sample, and the category of the reference target sample. Based on the model loss value, the classification model at the current stage can continue to be trained.

[0134] Furthermore, in one embodiment provided in this specification, calculating the model loss value based on the reference target sample, the feature vector of the second classification sample, the predicted target sample category, and the reference target sample category includes:

[0135] The reference target sample is input into the classification model of the previous training stage to obtain the feature vector of the first classification sample;

[0136] The distillation loss value of the classification model in the current training phase is calculated based on the feature vectors of the first and second classification samples.

[0137] The classification loss value of the classification model in the current training phase is calculated based on the predicted target sample category and the reference target sample category.

[0138] Wherein, the first classification sample feature vector refers to the feature vector of the reference target sample extracted by the classification model of the previous stage; the distillation loss value is used to limit the changes between the model parameters of the current stage classification model and the model parameters of the previous stage classification model based on the first classification sample feature vector and the second classification sample feature vector; the classification loss value refers to the difference between the predicted target sample category and the reference target sample category, and is used to measure the difference between the predicted target sample category and the reference target sample category.

[0139] Specifically, the reference target sample is input into the classification model of the previous stage to obtain the first classification sample feature vector output by the classification model of the previous training stage. Based on the first and second classification sample feature vectors, the distillation loss value of the current stage classification model is calculated. In practical applications, the loss function for calculating the distillation loss value can be the knowledge distillation loss function. Further, the classification loss value of the current stage classification model is calculated based on the predicted target sample category and the reference target sample category. In practical applications, the loss function for calculating the classification loss value can be the 0-1 loss function, the squared loss function, the cross-entropy loss function, etc. In this specification, the cross-entropy function is preferably selected as the loss function for calculating the classification loss value. Since the current stage classification model is not yet fully trained, there will be a deviation between the identified predicted target sample category and the actual reference target sample category. Therefore, the model parameters of the current stage classification model need to be adjusted accordingly until the model training stopping condition is met.

[0140] The sample determination method provided in this specification includes: obtaining a sample set to be processed, wherein the sample set to be processed includes a first initial sample set and a second initial sample set, wherein the number of second initial samples in the second initial sample set is greater than the number of first initial samples in the first initial sample set; determining the sampling probabilities corresponding to the first initial samples and the second initial samples in the sample set to be processed, respectively; sampling the first initial samples and the second initial samples in the sample set to be processed based on the sampling probabilities, respectively, to obtain a first undetermined sample corresponding to the first initial sample and a second undetermined sample corresponding to the second initial sample, wherein the number of the first undetermined samples and the number of the second undetermined samples are the same; performing data augmentation on the first undetermined samples and the second undetermined samples to obtain a target sample set.

[0141] One embodiment of this specification implements two sampling operations on a first initial sample and a second initial sample in a set of samples to be processed. By setting the same initial sampling probability, both the first initial sample and the second initial sample in the set of samples to be processed can participate in the subsequent sampling and training process. Then, based on the model parameters of the classification model in the previous stage and the number of sample categories, the sampling probability corresponding to different samples is determined, so that different initial samples have different sampling probabilities. Sampling is performed based on different sampling probabilities to achieve a balanced sample distribution between the first initial sample set and the second initial sample set.

[0142] The following is in conjunction with the appendix Figure 3 Taking the application of the sample determination method provided in this specification in an image classification scenario as an example, the sample determination method will be further explained. Among other things, Figure 3 The present specification shows a flowchart of a sample determination method according to an embodiment, which specifically includes the following steps.

[0143] Step 302: Obtain a set of image samples to be processed, wherein the set of image samples to be processed includes a first initial image sample set and a second initial image sample set, and the number of second initial image samples in the second initial image sample set is greater than the number of first initial image samples in the first initial image sample set.

[0144] Specifically, obtain a set of image samples A to be processed. The set of image samples A includes the old class image sample set from the previous training stage and the new class image sample set from the current training stage. The number of new class image samples in the new class image sample set is greater than the number of old class image samples in the old class image sample set.

[0145] Step 304: Set initial sampling probabilities for the first initial image sample and the second initial image sample respectively, and sample the first initial image sample and the second initial image sample respectively based on the initial sampling probabilities.

[0146] Specifically, to ensure that both old and new class image samples can participate in subsequent training, the same initial sampling probability can be set for both types of image samples, allowing them to participate in the training process with equal probability. For example, setting the initial sampling probability to 80% means that each image sample has an 80% chance of being sampled. Furthermore, based on the set initial sampling probability, old and new class image samples are sampled separately, ensuring that all image samples in the set A to be processed can participate in subsequent training with equal probability.

[0147] Step 306: Based on the sampling results, determine the target first initial image sample in the first initial image sample, and determine the target second initial image sample in the second initial image sample.

[0148] Specifically, since the initial sampling probabilities of the old class image samples and the new class image samples are the same, the results obtained based on the initial sampling probabilities are the same as the image sample set A to be processed, meaning that all image samples in the image sample set A to be processed can be collected. Based on this, the target old class initial image sample is determined from the old class image samples, and the target new class initial image sample is determined from the new class image samples.

[0149] Step 308: Determine the number of sample categories of the first initial image sample category corresponding to the first initial image sample set.

[0150] Specifically, if the old class image sample set is determined to have three image sample categories: "birds", "crabs", and "lions", then the number of old class image sample categories corresponding to the old class image sample set is 3.

[0151] Step 310: Obtain the model parameters of the classification model in the previous training stage, and determine the target first sampling weight and the target second sampling weight corresponding to the target first initial image sample and the target second initial image sample based on the number of sample categories of the first initial image sample category and the model parameters of the classification model in the previous training stage.

[0152] Specifically, the model parameters of the classification model from the previous training stage are obtained. Based on the number of old class image sample categories being 3 and the obtained model parameters of the classification model from the previous training stage, the sampling weights corresponding to the initial image samples of the target old class and the initial image samples of the target new class are calculated.

[0153] Step 312: Determine the sampling probability corresponding to the first initial image sample of the target based on the first sampling weight of the target, and determine the sampling probability corresponding to the second initial image sample of the target based on the second sampling weight of the target.

[0154] Specifically, the sampling probability of the initial image sample of the target old class is calculated based on the sampling weight corresponding to the initial image sample of the target old class, and the sampling probability of the initial image sample of the target new class is calculated based on the sampling weight corresponding to the initial image sample of the target new class.

[0155] Step 314: Based on different sampling probabilities, sample the first initial image sample and the second initial image sample in the set of image samples to be processed to obtain the first image sample to be determined corresponding to the first initial image sample and the second image sample to be determined corresponding to the second initial image sample, wherein the number of the first image sample to be determined is the same as the number of the second image sample to be determined.

[0156] Specifically, after calculating the sampling probabilities corresponding to different image samples, the corresponding image samples are sampled according to the sampling probabilities between different image samples to obtain the corresponding image samples, so as to make the distribution of the new and old class image samples and the new class image sample categories balanced.

[0157] Step 316: Determine two adjacent target image samples from the first image sample to be determined and the second image sample to be determined, wherein the two adjacent target image samples to be determined correspond to the same sample category.

[0158] Specifically, because the representations of new and old image samples differ, the classification model faces training difficulties due to these inconsistencies. Therefore, after sampling the new and old image samples, data augmentation is necessary to reduce the training difficulty between them.

[0159] In the sampled new and old class image samples, two adjacent target image samples of the same sample category are selected as target image samples to be determined.

[0160] Step 318: Obtain a preset correction coefficient, and determine the target image sample corresponding to the two adjacent target image samples based on the preset correction coefficient and the two adjacent target image samples to be determined.

[0161] Specifically, a preset correction coefficient is obtained, and the image interpolation method is used to calculate the target image sample after data augmentation for two adjacent target image samples to be determined.

[0162] Step 320: Obtain the target sample category corresponding to the target image sample and the classification model from the previous training stage.

[0163] Step 322: Input the target image sample into the classification model of the current training stage to obtain the second classification sample feature vector and the predicted target sample category output by the classification model of the current training stage.

[0164] Specifically, the target image sample is input into the classification model in the current training stage, and the image feature vector output by the classification model in the current training stage and the predicted image sample category of the target image sample are obtained.

[0165] Step 324: Input the target image sample into the classification model of the previous training stage to obtain the first classification sample feature vector, and calculate the distillation loss value of the classification model of the current training stage based on the first classification sample feature vector and the second classification sample feature vector.

[0166] Specifically, the target image sample is input into the classification model of the previous training stage, and the image feature vector output by the classification model of the previous training stage is obtained. Furthermore, based on the image feature vector output by the classification model of the current training stage and the image feature vector output by the classification model of the previous training stage, the distillation loss value of the classification model of the current training stage is calculated.

[0167] Step 326: Calculate the classification loss value of the classification model in the current training phase based on the predicted target sample category and the target sample category.

[0168] Specifically, the classification loss value of the classification model in the current training phase is calculated based on the predicted image sample category of the target image sample output by the classification model in the current training phase and the actual sample category of the target image sample.

[0169] Step 328: Adjust the classification model in the current training phase according to the distillation loss value and the classification loss value until the training stopping condition is met.

[0170] One embodiment of this specification implements two sampling operations on a first initial sample and a second initial sample in a set of samples to be processed. By setting the same initial sampling probability, both the first initial sample and the second initial sample in the set of samples to be processed can participate in the subsequent sampling and training process. Then, based on the model parameters of the classification model in the previous stage and the number of sample categories, the sampling probability corresponding to different samples is determined, so that different initial samples have different sampling probabilities. Sampling is performed based on different sampling probabilities to achieve a balanced sample distribution between the first initial sample set and the second initial sample set.

[0171] The sample determination method provided in this specification enables sample balancing in incremental learning, thereby resolving the problem of uneven distribution of training samples between old and new classes, and between different categories of the new class, which leads to imbalance in incremental learning and affects its effectiveness. Therefore, the image samples obtained using the sample determination method provided in this specification can be applied to image classification, image segmentation, image detection, and other related fields.

[0172] Corresponding to the above method embodiments, this specification also provides embodiments of a sample determination device. Figure 4 A schematic diagram of a sample determination device according to one embodiment of this specification is shown. Figure 4 As shown, the device includes:

[0173] The acquisition module 402 is configured to acquire a set of samples to be processed, wherein the set of samples to be processed includes a first initial sample set and a second initial sample set, and the number of second initial samples in the second initial sample set is greater than the number of first initial samples in the first initial sample set;

[0174] The determining module 404 is configured to determine the sampling probabilities corresponding to the first initial sample and the second initial sample in the set of samples to be processed, respectively;

[0175] The sampling module 406 is configured to sample the first initial sample and the second initial sample in the set of samples to be processed based on the sampling probability, and obtain the first sample to be determined corresponding to the first initial sample and the second sample to be determined corresponding to the second initial sample, wherein the number of the first sample to be determined is the same as the number of the second sample to be determined.

[0176] The enhancement module 408 is configured to perform data enhancement on the first sample to be determined and the second sample to be determined to obtain a target sample set.

[0177] Optionally, the determining module 404 is further configured to:

[0178] Set initial sampling probabilities for the first initial sample and the second initial sample respectively;

[0179] The first initial sample and the second initial sample are sampled respectively based on the initial sampling probability;

[0180] Based on the sampling results, determine the sampling weights corresponding to the first initial sample and the second initial sample, respectively;

[0181] The sampling probability is determined based on the sampling weight.

[0182] Optionally, the determining module 404 is further configured to:

[0183] Based on the sampling results, a first initial sample of the target is determined in the first initial sample, and a second initial sample of the target is determined in the second initial sample;

[0184] Determine the number of sample categories corresponding to the first initial sample category in the first initial sample set;

[0185] Obtain the model parameters of the classification model from the previous training phase;

[0186] The target first sampling weight and the target second sampling weight corresponding to the target first initial sample are determined based on the number of sample categories and the model parameters of the classification model in the previous training phase.

[0187] Optionally, the enhancement module 408 is further configured to:

[0188] Two adjacent target samples to be determined are identified from the first sample to be determined and the second sample to be determined, wherein the two adjacent target samples to be determined correspond to the same sample category;

[0189] Obtain the preset correction coefficient;

[0190] Based on the preset correction coefficient and the two adjacent target samples to be determined, determine the target samples corresponding to the two adjacent target samples to be determined;

[0191] Add the target sample to the target sample set.

[0192] Optionally, the device further includes:

[0193] The model acquisition module is configured to acquire the target sample category set corresponding to the target sample set and the classification model of the previous training stage;

[0194] The model training module is configured to train the classification model for the current training stage based on the target sample set, the target sample category set, and the classification model from the previous training stage.

[0195] Optionally, the model training module is further configured as follows:

[0196] A reference target sample is determined from the target sample set, and a reference target sample category corresponding to the reference target sample is determined from the target sample category set.

[0197] The reference target sample is input into the classification model of the current training stage to obtain the second classification sample feature vector and the predicted target sample category output by the classification model of the current training stage, wherein the classification model of the current training stage is used to classify the sample categories of the first initial sample set and the second initial sample set.

[0198] The model loss value is calculated based on the reference target sample, the feature vector of the second classification sample, the predicted target sample category, and the reference target sample category.

[0199] The classification model in the current training phase is adjusted based on the model loss value until the training stops.

[0200] Optionally, the model training module is further configured as follows:

[0201] The reference target sample is input into the classification model of the previous training stage to obtain the feature vector of the first classification sample;

[0202] The distillation loss value of the classification model in the current training phase is calculated based on the feature vectors of the first and second classification samples.

[0203] The classification loss value of the classification model in the current training phase is calculated based on the predicted target sample category and the reference target sample category.

[0204] The sample determination apparatus provided in this specification includes: an acquisition module configured to acquire a sample set to be processed, wherein the sample set to be processed includes a first initial sample set and a second initial sample set, and the number of second initial samples in the second initial sample set is greater than the number of first initial samples in the first initial sample set; a determination module configured to determine the sampling probabilities corresponding to the first initial samples and the second initial samples in the sample set to be processed, respectively; a sampling module configured to sample the first initial samples and the second initial samples in the sample set to be processed based on the sampling probabilities, respectively, to obtain a first sample to be determined corresponding to the first initial samples and a second sample to be determined corresponding to the second initial samples, wherein the number of the first sample to be determined is the same as the number of the second sample to be determined; and an enhancement module configured to perform data enhancement on the first sample to be determined and the second sample to be determined to obtain a target sample set.

[0205] One embodiment of this specification implements two sampling operations on a first initial sample and a second initial sample in a set of samples to be processed. By setting the same initial sampling probability, both the first initial sample and the second initial sample in the set of samples to be processed can participate in the subsequent sampling and training process. Then, based on the model parameters of the classification model in the previous stage and the number of sample categories, the sampling probability corresponding to different samples is determined, so that different initial samples have different sampling probabilities. Sampling is performed based on different sampling probabilities to achieve a balanced sample distribution between the first initial sample set and the second initial sample set.

[0206] The above is a schematic scheme of a sample determination device according to this embodiment. It should be noted that the technical solution of this sample determination device and the technical solution of the sample determination method described above belong to the same concept. For details not described in detail in the technical solution of the sample determination device, please refer to the description of the technical solution of the sample determination method described above.

[0207] See Figure 5 , Figure 5 A flowchart is shown of a training method for a classification model according to an embodiment of this specification, which specifically includes the following steps.

[0208] Step 502: Receive the sample training data and the sample categories corresponding to the sample training data.

[0209] Step 504: Input the sample training data into the classification model to obtain the predicted category of the sample training data.

[0210] Step 506: Calculate the loss value of the classification model based on the sample category and the predicted category.

[0211] Step 508: Adjust the model parameters of the classification model according to the loss value, and continue training the classification model until the training stopping condition is met.

[0212] Here, sample training data refers to the training data obtained from the sample training data set, which is the training sample of the classification model; the sample training data set refers to the set of images obtained by collecting images; the sample category refers to the actual category corresponding to the sample training data; the predicted topic category refers to the category output by the classification model when the sample training data is input; and the loss value refers to the difference between the sample category and the predicted category, which is used to measure the difference between the sample category and the predicted category.

[0213] Specifically, the sample training data is obtained through the above-described method for obtaining the sample set to be processed. The sample training data is then input into the classification model, which is used to identify the category of the sample training data. At this point, the classification model is not yet fully trained, and there will be a deviation between the predicted category and the actual sample category. The model parameters of the classification model need to be adjusted accordingly. Specifically, the loss value of the classification model is calculated based on the output predicted category and the sample category. In practical applications, the loss function for calculating the loss value can be the 0-1 loss function, the squared loss function, the cross-entropy loss function, etc. In this specification, the cross-entropy function is preferably selected as the loss function for calculating the loss value. The model parameters of the classification model are adjusted based on the loss value. The adjusted model parameters are then used to continue training the classification model with the next batch of sample training data until the model training stops.

[0214] Specifically, the model training stopping conditions include the model loss value being less than a preset threshold and / or the number of training epochs reaching a preset number of epochs.

[0215] In one specific implementation provided in this specification, the training stopping condition is taken as when the model loss value is less than a preset threshold. The preset threshold is 0.3. When the model loss value is less than 0.3, the classification model is considered to have completed training.

[0216] In another specific embodiment provided in this specification, taking a preset number of training rounds as the training stopping condition as an example, the preset number of training rounds is 30 rounds. When the training rounds of the sample training data reach 30 rounds, the classification model is considered to have completed training.

[0217] In another specific embodiment provided in this specification, two training stopping conditions are set: a preset threshold and a preset number of training rounds. At the same time, the loss value and the number of training rounds are monitored. When either the model loss value or the number of training rounds meets the training stopping condition, the classification model is considered to have completed training.

[0218] The training method for the classification model provided in this specification includes: receiving sample training data and the sample categories corresponding to the sample training data; inputting the sample training data into the classification model to obtain the predicted categories of the sample training data; calculating the loss value of the classification model based on the sample categories and the predicted categories; adjusting the model parameters of the classification model based on the loss value, and continuing to train the classification model until the training stopping condition is met.

[0219] One embodiment of this specification implements training a classification model using sample training data, thereby improving the accuracy and efficiency of the classification model in classifying sample training data.

[0220] Corresponding to the above method embodiments, this specification also provides embodiments of a training device for a classification model. Figure 6 A schematic diagram of a training apparatus for a classification model according to one embodiment of this specification is shown. Figure 6 As shown, the device includes:

[0221] The receiving module 602 is configured to receive sample training data and the sample categories corresponding to the sample training data;

[0222] Input module 604 is configured to input the sample training data into the classification model to obtain the predicted category of the sample training data;

[0223] The calculation module 606 is configured to calculate the loss value of the classification model based on the sample category and the predicted category;

[0224] Training module 608 is configured to adjust the model parameters of the classification model based on the loss value and continue training the classification model until the training stop condition is met.

[0225] The classification model training apparatus provided in this specification includes: a receiving module configured to receive sample training data and sample categories corresponding to the sample training data; an input module configured to input the sample training data into the classification model to obtain the predicted category of the sample training data; a calculation module configured to calculate the loss value of the classification model based on the sample category and the predicted category; and a training module configured to adjust the model parameters of the classification model based on the loss value and continue training the classification model until the training termination condition is met.

[0226] One embodiment of this specification implements training a classification model using sample training data, thereby improving the accuracy and efficiency of the classification model in classifying sample training data.

[0227] The above is a schematic scheme of a classification model training device according to this embodiment. It should be noted that the technical solution of this classification model training device and the technical solution of the classification model training method described above belong to the same concept. For details not described in detail in the technical solution of the classification model training device, please refer to the description of the technical solution of the classification model training method described above.

[0228] See Figure 7 , Figure 7 A flowchart of an image classification method according to an embodiment of this specification is shown below, in conjunction with the accompanying drawings. Figure 7 Taking the image classification method provided in this specification as an example in the application of image classification scenarios, the image classification method will be further explained, specifically including the following steps.

[0229] Step 702: Obtain the image to be recognized.

[0230] Step 704: Input the image to be identified into the image classification model.

[0231] Step 706: Obtain the image category output by the image classification model.

[0232] Step 708: Determine the classification result of the image to be identified based on the image category.

[0233] The image to be identified refers to the image that needs to be classified and is waiting to be identified. The image to be identified can be any image from the old and new image samples. The image classification model is the model used to determine the image category corresponding to the image to be identified.

[0234] Specifically, the image to be identified is obtained based on the above sample determination method. The obtained image to be identified is input into the image classification model to obtain the image category corresponding to the image to be identified, and the classification result corresponding to the image to be identified is determined according to the image category corresponding to the image to be identified.

[0235] For example, if the image to be identified, a, is obtained based on the above sample determination method, and the image to be identified, a, is input into the image classification model, and the image category corresponding to the image to be identified, a, is obtained as "birds", then the classification result of the image to be identified, a, is determined to be "birds", and the image to be identified, a, is classified into the image category of "birds".

[0236] The image classification method provided in this specification uses images to be classified based on the sample determination method described above. The image to be classified can be any image from the old and new class samples. When the image to be classified is input into the image classification model, the image category corresponding to the image to be classified can be accurately identified, and then the image to be classified can be classified according to the image category corresponding to the image to be classified.

[0237] Corresponding to the above method embodiments, this specification also provides embodiments of image classification devices. Figure 8 A schematic diagram of an image classification device according to one embodiment of this specification is shown. Figure 8 As shown, the device includes:

[0238] Image acquisition module 802 is configured to acquire an image to be recognized;

[0239] Image input module 804 is configured to input the image to be recognized into an image classification model;

[0240] The category acquisition module 806 is configured to acquire the image category output by the image classification model;

[0241] The classification module 808 is configured to determine the classification result of the image to be identified based on the image category.

[0242] The image classification apparatus provided in this specification includes: an image acquisition module configured to acquire an image to be identified; an image input module configured to input the image to be identified into an image classification model; a category acquisition module configured to acquire the image category output by the image classification model; and a classification module configured to determine the classification result of the image to be identified based on the image category.

[0243] The image classification device provided in this specification uses images to be classified based on the sample determination method described above. The images to be classified can be any one of the old and new image samples. When the images to be classified are input into the image classification model, the image category corresponding to the images to be classified can be accurately identified, and then the images to be classified can be classified according to the image category corresponding to the images to be classified.

[0244] The above is a schematic scheme of an image classification device according to this embodiment. It should be noted that the technical solution of this image classification device and the technical solution of the image classification method described above belong to the same concept. For details not described in detail in the technical solution of the image classification device, please refer to the description of the technical solution of the image classification method described above.

[0245] Figure 9 A structural block diagram of a computing device 900 according to one embodiment of this specification is shown. The components of the computing device 900 include, but are not limited to, a memory 910 and a processor 920. The processor 920 is connected to the memory 910 via a bus 930, and a database 950 is used to store data.

[0246] The computing device 900 also includes an access device 940, which enables the computing device 900 to communicate via one or more networks 960. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 940 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0247] In one embodiment of this specification, the above-described components of the computing device 900 and Figure 9 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 9 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0248] The computing device 900 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 900 can also be a mobile or stationary server.

[0249] The processor 920 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described sample determination method or classification model training method. The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above-described sample determination method or classification model training method belong to the same concept. Details not described in detail in the technical solution of the computing device can be found in the description of the technical solution of the above-described sample determination method or classification model training method.

[0250] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described sample determination method or classification model training method.

[0251] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the sample determination method or the classification model training method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the sample determination method or the classification model training method described above.

[0252] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described sample determination method or classification model training method.

[0253] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program belongs to the same concept as the technical solutions of the aforementioned sample determination method or classification model training method. Details not described in detail in the computer program's technical solution can be found in the descriptions of the technical solutions of the aforementioned sample determination method or classification model training method.

[0254] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0255] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0256] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0257] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0258] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for determining a sample, comprising: Obtain a set of samples to be processed, wherein the set of samples to be processed includes a first initial sample set and a second initial sample set, wherein the number of second initial samples in the second initial sample set is greater than the number of first initial samples in the first initial sample set, and the samples to be processed include image samples, wherein the first initial samples are old class image samples, and the second initial samples are new class image samples; The sampling probabilities corresponding to the first initial sample and the second initial sample in the set of samples to be processed are determined respectively, wherein the sampling probabilities are determined by: sampling the first initial sample and the second initial sample respectively based on the initial sampling probabilities, and determining the sampling probabilities based on the sampling weights obtained from the sampling results; Based on the sampling probability, the first initial sample and the second initial sample are sampled from the set of samples to be processed to obtain the first sample to be determined corresponding to the first initial sample and the second sample to be determined corresponding to the second initial sample, wherein the number of the first sample to be determined is the same as the number of the second sample to be determined. Data augmentation is performed on the first and second samples to be determined to obtain the target sample set.

2. The method according to claim 1, wherein determining the sampling probabilities corresponding to the first initial sample and the second initial sample in the sample set to be processed includes: Set initial sampling probabilities for the first initial sample and the second initial sample respectively; The first initial sample and the second initial sample are sampled respectively based on the initial sampling probability; Based on the sampling results, determine the sampling weights corresponding to the first initial sample and the second initial sample, respectively; The sampling probability is determined based on the sampling weight.

3. The method according to claim 2, wherein determining the sampling weights corresponding to the first initial sample and the second initial sample based on the sampling results comprises: Based on the sampling results, a first initial sample of the target is determined in the first initial sample, and a second initial sample of the target is determined in the second initial sample; Determine the number of sample categories corresponding to the first initial sample category in the first initial sample set; Obtain the model parameters of the classification model from the previous training phase; The target first sampling weight and the target second sampling weight corresponding to the target first initial sample are determined based on the number of sample categories and the model parameters of the classification model in the previous training phase.

4. The method according to claim 1, wherein performing data augmentation on the first sample to be determined and the second sample to be determined to obtain a target sample set includes: Two adjacent target samples to be determined are identified from the first sample to be determined and the second sample to be determined, wherein the two adjacent target samples to be determined correspond to the same sample category; Obtain the preset correction coefficient; Based on the preset correction coefficient and the two adjacent target samples to be determined, determine the target samples corresponding to the two adjacent target samples to be determined; Add the target sample to the target sample set.

5. The method according to claim 1, after performing data augmentation on the first sample to be determined and the second sample to be determined to obtain a target sample set, the method further includes: Obtain the target sample category set corresponding to the target sample set and the classification model from the previous training stage; The classification model for the current training stage is trained based on the target sample set, the target sample category set, and the classification model from the previous training stage.

6. The method according to claim 5, wherein training the classification model for the current training stage based on the target sample set, the target sample category set, and the classification model from the previous training stage comprises: A reference target sample is determined from the target sample set, and a reference target sample category corresponding to the reference target sample is determined from the target sample category set. The reference target sample is input into the classification model of the current training stage to obtain the second classification sample feature vector and the predicted target sample category output by the classification model of the current training stage, wherein the classification model of the current training stage is used to classify the sample categories of the first initial sample set and the second initial sample set. The model loss value is calculated based on the reference target sample, the feature vector of the second classification sample, the predicted target sample category, and the reference target sample category. The classification model in the current training phase is adjusted based on the model loss value until the training stops.

7. The method according to claim 6, wherein calculating the model loss value based on the reference target sample, the second classification sample feature vector, the predicted target sample category, and the reference target sample category comprises: The reference target sample is input into the classification model of the previous training stage to obtain the feature vector of the first classification sample; The distillation loss value of the classification model in the current training phase is calculated based on the feature vectors of the first and second classification samples. The classification loss value of the classification model in the current training phase is calculated based on the predicted target sample category and the reference target sample category.

8. A method for training a classification model, comprising: The system receives sample training data and the sample categories corresponding to the sample training data. The sample training data is a target sample set obtained through the following method: obtaining a sample set to be processed, wherein the sample set to be processed includes a first initial sample set and a second initial sample set, the number of second initial samples in the second initial sample set being greater than the number of first initial samples in the first initial sample set, the samples to be processed including image samples, the first initial samples being old-class image samples, and the second initial samples being new-class image samples; determining the sampling probabilities corresponding to the first initial samples and the second initial samples in the sample set to be processed, wherein the determination of the sampling probabilities includes: sampling the first initial samples and the second initial samples based on the initial sampling probabilities, determining the sampling probabilities based on the sampling weights obtained from the sampling results, sampling the first initial samples and the second initial samples in the sample set to be processed based on the sampling probabilities to obtain a first undetermined sample corresponding to the first initial sample and a second undetermined sample corresponding to the second initial sample, wherein the number of the first undetermined samples is the same as the number of the second undetermined samples; performing data augmentation on the first undetermined samples and the second undetermined samples to obtain a target sample set; The sample training data is input into the classification model to obtain the predicted category of the sample training data; The loss value of the classification model is calculated based on the sample category and the predicted category; The model parameters of the classification model are adjusted based on the loss value, and the classification model is trained continuously until the training stopping condition is met.

9. A sample determination device, comprising: The acquisition module is configured to acquire a set of samples to be processed, wherein the set of samples to be processed includes a first initial sample set and a second initial sample set, the number of second initial samples in the second initial sample set is greater than the number of first initial samples in the first initial sample set, and the samples to be processed include image samples, wherein the first initial samples are old class image samples and the second initial samples are new class image samples; The determining module is configured to determine the sampling probabilities corresponding to the first initial sample and the second initial sample in the set of samples to be processed, wherein the method for determining the sampling probabilities includes: sampling the first initial sample and the second initial sample based on the initial sampling probabilities, and determining the sampling probabilities based on the sampling weights obtained from the sampling results; The sampling module is configured to sample the first initial sample and the second initial sample in the set of samples to be processed based on the sampling probability, and obtain a first sample to be determined corresponding to the first initial sample and a second sample to be determined corresponding to the second initial sample, wherein the number of the first sample to be determined is the same as the number of the second sample to be determined. The enhancement module is configured to perform data enhancement on the first sample to be determined and the second sample to be determined to obtain a target sample set.

10. A training device for a classification model, comprising: A receiving module is configured to receive sample training data and the sample categories corresponding to the sample training data. The sample training data is a target sample set obtained through the following methods: acquiring a sample set to be processed, wherein the sample set to be processed includes a first initial sample set and a second initial sample set, the number of second initial samples in the second initial sample set being greater than the number of first initial samples in the first initial sample set, the samples to be processed including image samples, the first initial samples being old-class image samples, and the second initial samples being new-class image samples; determining the sampling probabilities corresponding to the first initial samples and the second initial samples in the sample set to be processed, wherein the determination of the sampling probabilities includes: sampling the first initial samples and the second initial samples based on the initial sampling probabilities; determining the sampling probabilities based on the sampling weights obtained from the sampling results; sampling the first initial samples and the second initial samples in the sample set to be processed based on the sampling probabilities to obtain a first undetermined sample corresponding to the first initial sample and a second undetermined sample corresponding to the second initial sample, wherein the number of the first undetermined samples is the same as the number of the second undetermined samples; performing data augmentation on the first undetermined samples and the second undetermined samples to obtain a target sample set; The input module is configured to input the sample training data into the classification model to obtain the predicted category of the sample training data; The calculation module is configured to calculate the loss value of the classification model based on the sample category and the predicted category; The training module is configured to adjust the model parameters of the classification model based on the loss value and continue training the classification model until the training stopping condition is met.

11. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 7 or 8.

12. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7 or 8.

Images