Class-incremental learning method based on long-tail distribution scene and related product

Abstract

The application discloses a class incremental learning method based on a long-tail distribution scene and related products, and the method comprises the following steps: constructing a sample data set presenting a long-tail distribution; wherein the sample data set is divided into old-class sample data and new-class sample data; extracting a feature vector of the old-class sample data as a first feature vector, and generating a second feature vector by using the first feature vector; and training an incremental learning model by using a preset total loss function, the sample data set and the second feature vector, so as to obtain a trained incremental learning model. The embodiment of the application can improve the model classification accuracy.

Description

Technical Field

[0001] This application relates to the field of model optimization technology, and in particular to a category incremental learning method and related products based on long-tailed distribution scenarios. Background Technology

[0002] Image classification has always been a focus of attention in the field of computer vision. With the continuous development of hardware resources, how to enable models to continuously learn new knowledge from new data like humans, while ensuring that previously learned knowledge is not forgotten, i.e., category incremental learning, has become a new research direction.

[0003] In actual data collection, significant differences often exist in sample distribution: some categories (such as popular videos and common animal pictures) have a sufficient number of samples, while other categories (such as less popular videos and rare animal pictures) have scarce samples. This phenomenon is defined as a long-tail distribution, specifically manifested in the fact that some categories in the dataset have a large number of samples (i.e., head classes), while some categories have only a small number of samples (i.e., tail classes). In such scenarios, due to limited memory space, the number of samples in older categories is generally very small, easily leading to an imbalance in the number of samples between older and newer categories, thus causing bias in model classification. Summary of the Invention

[0004] This application provides a category incremental learning method and related products based on long-tail distribution scenarios to solve the problem that the imbalance between the number of new and old category samples in the existing technology can easily lead to model classification bias.

[0005] To achieve the above objectives, embodiments of this application provide a category incremental learning method based on long-tail distribution scenarios, including: Construct a sample dataset exhibiting a long-tail distribution; wherein the sample dataset is divided into old category sample data and new category sample data; Extract the feature vector of the old category sample data as the first feature vector, and use the first feature vector to generate the second feature vector; The incremental learning model is trained using a preset total loss function, the sample dataset, and the second feature vector to obtain a trained incremental learning model. The incremental learning model comprises an input layer, a feature extraction layer, and a classification layer connected in sequence. The total loss function includes a balance loss function and a knowledge distillation loss function. The balance loss function is used to balance the learning of the target old category sample data and the new category sample data. The target old category sample data includes the old category sample data and the second feature vector. The knowledge distillation loss function is used to measure the difference between the output results of the feature extraction layer of the old incremental learning model and the output results of the feature extraction layer of the new incremental learning model.

[0006] As an improvement to the above scheme, the balancing loss function includes: The first loss function is used to measure the classification layer output of the incremental learning model when the old category sample data is used as input; The second loss function is used to measure the classification layer output of the incremental learning model when the second feature vector is used as input.

[0007] As an improvement to the above scheme, the balancing loss function further includes: The third loss function is used to measure the difference between the class center feature vector of the old category sample data and the feature vector of the new category sample data output by the feature extraction layer of the incremental learning model.

[0008] As an improvement to the above scheme, the incremental learning model also includes a correction layer connected to the classification layer; The total loss function also includes: The corrective loss function is used to measure the difference between the output of the classification layer and the output of the corrective layer of the new category sample data in the same incremental learning model.

[0009] As an improvement to the above scheme, the step of generating a second feature vector using the first feature vector includes: Based on the first feature vector, obtain the class center vector of the old category sample data; Using the class center vector as a condition and the first feature vector as training data, the adversarial network is trained using random noise to obtain a trained adversarial network. The second feature vector is generated using the trained adversarial network.

[0010] As an improvement to the above scheme, the construction of a sample dataset exhibiting a long-tailed distribution includes: For the first task, the sample data of each type within the first task will be distributed in a balanced manner; For non-first tasks, the sample data of each type within the non-first task are presented as a long-tail distribution.

[0011] To achieve the above objectives, embodiments of this application also provide a category incremental learning device based on long-tail distribution scenarios, comprising: A construction module is used to build a sample dataset that exhibits a long-tail distribution; wherein, the sample dataset is divided into old category sample data and new category sample data; The generation module is used to extract the feature vector of the old category sample data as the first feature vector, and use the first feature vector to generate the second feature vector; The learning module is used to train the incremental learning model using a preset total loss function, the sample dataset, and the second feature vector to obtain a trained incremental learning model. The incremental learning model comprises an input layer, a feature extraction layer, and a classification layer connected in sequence. The total loss function includes a balance loss function and a knowledge distillation loss function. The balance loss function is used to balance the learning of the target old category sample data and the new category sample data. The target old category sample data includes the old category sample data and the second feature vector. The knowledge distillation loss function is used to measure the difference between the output results of the feature extraction layer of the old incremental learning model and the output results of the feature extraction layer of the new incremental learning model.

[0012] To achieve the above objectives, this application also provides a category incremental learning device based on a long-tailed distribution scenario, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the category incremental learning method based on the long-tailed distribution scenario as described above.

[0013] To achieve the above objectives, embodiments of this application also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the category incremental learning method based on the long-tail distribution scenario described above.

[0014] To achieve the above objectives, embodiments of this application also provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the category incremental learning method based on long-tailed distribution scenarios as described above.

[0015] Compared with existing technologies, this application provides a category incremental learning method and related products based on long-tail distribution scenarios. This method constructs a sample dataset exhibiting a long-tail distribution. The sample dataset is divided into old category sample data and new category sample data. Feature vectors from the old category sample data are extracted as a first feature vector. A second feature vector is generated using the first feature vector. An incremental learning model is trained using a preset total loss function, the sample dataset, and the second feature vector to obtain a trained incremental learning model. The incremental learning model includes an input layer, a feature extraction layer, and a classification layer connected sequentially. The total loss function includes a balance loss function and a knowledge distillation loss function. The balance loss function is used to balance the learning of the target old category sample data and the new category sample data. The target old category sample data includes the old category sample data and the second feature vector. The knowledge distillation loss function measures the difference between the output results of the feature extraction layer of the old incremental learning model and the output results of the feature extraction layer of the new incremental learning model. Therefore, the embodiments of this application utilize the old feature vector (i.e., the first feature vector) of the old category sample data to generate a new feature vector (i.e., the second feature vector) of the old category sample data, which can overcome the prediction bias caused by the imbalance between the number of old and new category samples and can no longer rely solely on the old samples in memory. Attached Figure Description

[0016] Figure 1 This is a flowchart of a category incremental learning method based on a long-tail distribution scenario provided in an embodiment of this application; Figure 2 This is a schematic diagram of the distribution of a sample dataset provided in an embodiment of this application; Figure 3 This is a structural block diagram of a category incremental learning device based on a long-tail distribution scenario provided in an embodiment of this application; Figure 4 This is a structural block diagram of a category incremental learning device based on a long-tail distribution scenario provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0018] In the description of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0019] In this application description, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0020] In this application description, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The term "based on" means "at least partially based on." The term "according to" means "at least partially according to." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments." The term "and / or" means at least one of the connected objects, such as A and / or B, indicating three cases: including only A, only B, and both A and B. Unless otherwise stated, the term "multiple" means two or more.

[0021] See Figure 1 , Figure 1 This is a flowchart illustrating a category incremental learning method for long-tailed distribution scenarios provided in this application embodiment. The category incremental learning method for long-tailed distribution scenarios includes: S1. Construct a sample dataset exhibiting a long-tail distribution; wherein, the sample dataset is divided into old category sample data and new category sample data; S2. Extract the feature vector of the old category sample data as the first feature vector, and use the first feature vector to generate the second feature vector; S3. Train the incremental learning model using the preset total loss function, the sample dataset, and the second feature vector to obtain the trained incremental learning model; The incremental learning model comprises an input layer, a feature extraction layer, and a classification layer connected in sequence. The total loss function includes a balance loss function and a knowledge distillation loss function. The balance loss function is used to balance the learning of the target old category sample data and the new category sample data. The target old category sample data includes the old category sample data and the second feature vector. The knowledge distillation loss function is used to measure the difference between the output results of the feature extraction layer of the old incremental learning model and the output results of the feature extraction layer of the new incremental learning model.

[0022] It is worth noting that, according to the current task, the sample dataset can be divided into old category sample data and new category sample data. Old category refers to categories that do not exist in the current task but appeared in previous tasks. Therefore, old category sample data consists of category sample data that does not exist in the current task but appeared in previous tasks. New category refers to categories that exist in the current task but did not appear in previous tasks. Therefore, new category sample data consists of category sample data that exists in the current task but did not appear in previous tasks. The concepts of old and new are relative and are specific to a particular task.

[0023] By generating a large number of second feature vectors (i.e. new feature vectors of old category sample data) during training, the balance between old and new category sample data is ensured. The embodiments of this application use a balanced loss function combined with a knowledge distillation loss function for model training, which can overcome the prediction bias caused by the imbalance between the number of old and new category sample data and no longer rely solely on old sample data in memory.

[0024] In one optional embodiment, the balance loss function includes: The first loss function is used to measure the classification layer output of the incremental learning model when the old category sample data is used as input; The second loss function is used to measure the classification layer output of the incremental learning model when the second feature vector is used as input.

[0025] In this embodiment, the first loss function is used to update the entire model, and the second loss function is used to update the classification layer parameters.

[0026] In an optional embodiment, the balance loss function further includes: The third loss function is used to measure the difference between the class center feature vector of the old category sample data and the feature vector of the new category sample data output by the feature extraction layer of the incremental learning model.

[0027] In order to make the new category features extracted by the model more discriminative with the old category features, this embodiment of the application uses a third loss function to measure the difference between the class center feature vector of the old category sample data and the feature vector of the new category sample data output by the feature extraction layer of the incremental learning model. Specifically, the distance between the two can be used as a measure, and it is desirable for the two to be as far apart as possible.

[0028] In an optional embodiment, the incremental learning model further includes a correction layer connected to the classification layer; The total loss function also includes: The corrective loss function is used to measure the difference between the output of the classification layer and the output of the corrective layer of the new category sample data in the same incremental learning model.

[0029] It is worth noting that in long-tail distribution scenarios, imbalance within the new category sample data can occur, which will inevitably lead to a further decrease in the model's classification accuracy. This application proposes a method of outputting correction for each category, that is, adding a correction layer after the model's classification layer. This layer contains multiple sets of parameters, and the final training of the model is completed by training these parameters. It is important to note that when training the parameters of the correction layer, the parameters of the model's feature extraction layer and classification layer are fixed; that is, the parameters of the correction layer are trained only after the parameters of the feature extraction layer and classification layer are trained.

[0030] In an optional embodiment, generating a second feature vector using the first feature vector includes: Based on the first feature vector, obtain the class center vector of the old category sample data; Using the class center vector as a condition and the first feature vector as training data, the adversarial network is trained using random noise to obtain a trained adversarial network. The second feature vector is generated using the trained adversarial network.

[0031] It is worth noting that the class center vector refers to the center vector of a certain class of data. The class center vector of the old class sample data is the center vector of the old class sample data. In this embodiment, the class center vector is used to generate a new feature vector, namely the second feature vector, for the old class sample data. This makes incremental learning no longer solely dependent on the old samples in memory, thereby alleviating the problem of imbalance between the number of new and old class samples and helping the model avoid serious prediction bias.

[0032] In one alternative embodiment, constructing the sample dataset exhibiting a long-tailed distribution includes: For the first task, the sample data of each type within the first task will be distributed in a balanced manner; For non-first tasks, the sample data of each type within the non-first task are presented as a long-tail distribution.

[0033] It's worth noting that the first task refers to the initial task in the entire incremental learning process. The training samples for this task are class-balanced, meaning each class has an equal number of samples. New samples arrive in batches, and incremental learning is performed for each batch. In this process, each batch of data constitutes a new task, which is considered a non-first task.

[0034] For example, the entire incremental learning process proceeds sequentially through Task 0, Task 1, and Task 2. Task 0 is the first task, while Tasks 1 and 2 are subsequent tasks. For Task 0, each class of sample data is new class data and exhibits a balanced distribution. For Task 1, the data from Task 0 is old class sample data, and the new data from Task 1 is new class sample data, exhibiting a long-tailed distribution. For Task 2, the data from Task 0 and Task 1 are old class sample data, and the new data from Task 2 is new class sample data, also exhibiting a long-tailed distribution.

[0035] In this embodiment, for the first task, the sample data of each class is distributed in a balanced manner, that is, the number of sample data of each class in the first task is the same; for non-first tasks, the sample data of each class is distributed in a long-tailed manner, which can better simulate the real scene and improve the model performance.

[0036] In one specific embodiment, the steps of the category incremental learning method based on long-tail distribution scenarios are as follows: 1. Problem Definition First, there are a total of [number] training sessions throughout the entire training process. Each task (phase) can be represented as . No. New category sample data for each task ,in For the input sample, such as an input image, For the labels corresponding to the input samples, For the first The number of samples for each task. Additionally, there is memory used to store old category sample data, represented as... The number of old category samples in memory is fixed and relatively small. However, as the learning task progresses, the number of old category samples continuously increases, leading to a larger total number of samples in memory. Gradually increase; The input samples are stored in memory. This is the label corresponding to the input sample. Indicates the first The number of categories in the new category sample data for each task Indicates the first The number of categories in the old category sample data for each task, then the relationship between the two can be expressed as: , Indicates the first The number of categories in the new category sample data for each task. The final evaluation criterion for the model's classification ability is whether it can make correct classification predictions on all known category samples.

[0037] 2. Dataset Partitioning To construct a sample dataset that conforms to a long-tailed distribution, the ImageNet100 dataset is modified as follows: Figure 2 The entire dataset was divided into six subsets, corresponding to six tasks. The first task involved 50 classes with balanced sample sizes. During the subsequent incremental learning process, each subset contained 10 classes and exhibited a long-tailed distribution. The long-tailed distribution was constructed using a common decay exponent, i.e. , Representing the i The number of samples in each category, take , =1300, substitute it into ,get =65, then for non-first tasks, the sample size will be reduced from 1300 to 65, with the reduction method following the exponential decay method. For example, the sample size of various types of data in non-first tasks will be 1300, 931, 668, 478, 343, 246, 176, 126, 90, and 65 respectively.

[0038] Furthermore, in addition to training the model using the data normally for the first task, the subsequent five tasks underwent preprocessing before training the model: for new category sample data... The dataset is divided into a validation set and a training set. The same applies to the old class sample data in memory, which are represented here as follows: and Therefore, we can obtain the first... One task is used to train the feature extraction layer. and validation data for training the correction layer. .

[0039] 3. Feature Vector Generation When in the first After the model for each task is trained, all the data from the current task will be used. By inputting these features into the trained model, we can obtain the first feature vector for each input data point (input image). Here, the class center vector for each category's feature vector can be calculated as follows:

[0040] in, Representing the The first task The class center vectors of each category, Representing the The first task The number of samples in each category Representing the The first task The first feature vector of each category. This yields a set of class centers. .

[0041] After obtaining the class center vector for each category, use it as a condition. The training data (specifically, the set of feature vectors formed by the first feature vector) and random noise are used to train the adversarial network (including: the first feature vector). Generator of tasks and discriminator This allows the adversarial network to learn the distribution of feature vectors for all current categories. The specific loss function is:

[0042] in, Expressing expectations, Indicates random noise. This represents the real data, specifically the first feature vector. Represents the class center vector. Represents a generator. Indicates the discriminator, The distribution representing the first eigenvector is used for sampling during training to form the set of eigenvectors. , This represents a prior noise distribution, commonly a Gaussian or uniform distribution. Thus, in the... Each task will reward you with a generator. The generator of this adversarial network is fed with random noise as input. and specific class center vectors This will generate a feature vector (i.e., the second feature vector) for that category, which can be represented as:

[0043] In this way, the generator can continuously generate new feature vectors, or second feature vectors, of old category sample data to help the model avoid biasing the prediction results towards the new category.

[0044] 4. Loss Calculation and Prediction Correction use The model is trained using the second feature vector.

[0045] The balance loss function can be specifically expressed as:

[0046] in, It is the first The number of old category sample data for each task It is the first The number of new category sample data for each task and It is an indicator function; the positions where a sample belongs to the true class are marked as 1, and the rest are marked as 0. This refers to the classification layer output of the incremental learning model when the sample dataset is used as input; for example, it refers to the classification layer output probability when an image is used as input. It is the second eigenvector When used as input, the classification layer output of the incremental learning model, for example, the probability output of the classification layer when the second feature vector is used as input. It is the feature vector of the new category sample data output by the feature extraction layer of the model. Representing the The first task The class center vectors of each category.

[0047] Knowledge distillation loss function Using the same input image, the distance between the feature extraction layer output of the old incremental learning model and the feature extraction layer output of the new incremental learning model is measured. The smaller the distance, the better, i.e., the lower the loss, the better.

[0048] As the number of old categories grows larger, older knowledge becomes more important. Therefore, the balancing loss is multiplied by an automatically adjusting weight. In addition, combined with The weights are adjusted using two methods, which are then fused and averaged to achieve better training results. The overall loss function is then defined as follows:

[0049] Furthermore, a method of output correction for each category is proposed. This involves adding a correction layer after the model output. After the model's feature extractor and classification layer parameters are trained, the parameters of the correction layer are trained. Since the old samples stored in memory are class-balanced, and the generator also generates feature vectors for the old categories, the model's output for the old categories remains unchanged.

[0050]

[0051] In the formula, It is the first The output results of each category in the classification layer It is the first The output of each category after passing through the correction layer. For the output of the new category, a method using two sets of training parameters is proposed. and They are corrected one by one, that is:

[0052] in It is clear that the number of correction parameters for each task is [number missing]. The correction loss function used during training can be the cross-entropy loss function, which measures the difference between the output of the classification layer and the output of the correction layer of the same incremental learning model for new class sample data. 5. Model Reasoning After each training stage, a model is obtained that can predict the class of test samples with all known classes. Known classes refer to all classes that have appeared in the training data up to the current stage. The better the model's classification ability in the final stage, the better it learns new knowledge during incremental learning while retaining knowledge learned from previous tasks.

[0053] See Figure 3 , Figure 3 This is a structural block diagram of a category incremental learning device 10 based on a long-tailed distribution scenario provided in an embodiment of this application. The category incremental learning device based on a long-tailed distribution scenario includes: Module 11 is used to construct a sample dataset that presents a long-tail distribution; wherein the sample dataset is divided into old category sample data and new category sample data; The generation module 12 is used to extract the feature vector of the old category sample data as the first feature vector, and use the first feature vector to generate the second feature vector; Learning module 13 is used to train the incremental learning model using a preset total loss function, the sample dataset and the second feature vector, to obtain a trained incremental learning model. The incremental learning model comprises an input layer, a feature extraction layer, and a classification layer connected in sequence. The total loss function includes a balance loss function and a knowledge distillation loss function. The balance loss function is used to balance the learning of the target old category sample data and the new category sample data. The target old category sample data includes the old category sample data and the second feature vector. The knowledge distillation loss function is used to measure the difference between the output results of the feature extraction layer of the old incremental learning model and the output results of the feature extraction layer of the new incremental learning model.

[0054] Optionally, the balancing loss function includes: The first loss function is used to measure the classification layer output of the incremental learning model when the old category sample data is used as input; The second loss function is used to measure the classification layer output of the incremental learning model when the second feature vector is used as input.

[0055] Optionally, the balancing loss function further includes: The third loss function is used to measure the difference between the class center feature vector of the old category sample data and the feature vector of the new category sample data output by the feature extraction layer of the incremental learning model.

[0056] Optionally, the incremental learning model also includes a correction layer connected to the classification layer; The total loss function also includes: The corrective loss function is used to measure the difference between the output of the classification layer and the output of the corrective layer of the new category sample data in the same incremental learning model.

[0057] Optionally, generating a second feature vector using the first feature vector includes: Based on the first feature vector, obtain the class center vector of the old category sample data; Using the class center vector as a condition and the first feature vector as training data, the adversarial network is trained using random noise to obtain a trained adversarial network. The second feature vector is generated using the trained adversarial network.

[0058] Optionally, the construction of the sample dataset exhibiting a long-tailed distribution includes: For the first task, the sample data of each type within the first task will be distributed in a balanced manner; For non-first tasks, the sample data of each type within the non-first task are presented as a long-tail distribution.

[0059] It is worth noting that the working process of each module in the category incremental learning device 10 based on long-tail distribution scenarios described in this application embodiment can refer to the working process of the category incremental learning method based on long-tail distribution scenarios described in the above embodiment, and achieve the same beneficial effect, which will not be repeated here.

[0060] Furthermore, this application also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the category incremental learning method based on the long-tail distribution scenario as described in any of the above embodiments.

[0061] Furthermore, this application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the category incremental learning method based on long-tailed distribution scenarios as described in any of the above embodiments.

[0062] See Figure 4 , Figure 4 This is a structural block diagram of a category incremental learning device 20 based on a long-tailed distribution scenario provided in this application embodiment. The category incremental learning device 20 based on a long-tailed distribution scenario includes: a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, it implements the steps in the above-described embodiment of the category incremental learning method based on a long-tailed distribution scenario. Alternatively, when the processor 21 executes the computer program, it implements the functions of each module / unit in the above-described device embodiments.

[0063] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the category incremental learning device 20 based on long-tail distribution scenarios.

[0064] The category incremental learning device 20 based on long-tail distribution scenarios may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the schematic diagram is merely an example of the category incremental learning device 20 based on long-tail distribution scenarios and does not constitute a limitation on the device. It may include more or fewer components than illustrated, or combine certain components, or use different components. For example, the category incremental learning device 20 based on long-tail distribution scenarios may also include input / output devices, network access devices, buses, etc.

[0065] The processor 21 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 21 is the control center of the category incremental learning device 20 based on the long-tail distribution scenario, connecting all parts of the device 20 using various interfaces and lines.

[0066] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements various functions of the category incremental learning device 20 based on long-tail distribution scenarios by running or executing the computer programs and / or modules stored in the memory 22 and calling the data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0067] The modules / units integrated in the category incremental learning device 20 based on the long-tail distribution scenario, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above-described embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 21, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0068] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0069] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. A category incremental learning method based on long-tail distribution scenarios, characterized in that, include: Construct a sample dataset exhibiting a long-tail distribution; wherein the sample dataset is divided into old category sample data and new category sample data; Extract the feature vector of the old category sample data as the first feature vector, and use the first feature vector to generate the second feature vector; The incremental learning model is trained using a preset total loss function, the sample dataset, and the second feature vector to obtain a trained incremental learning model. The incremental learning model comprises an input layer, a feature extraction layer, and a classification layer connected in sequence. The total loss function includes a balance loss function and a knowledge distillation loss function. The balance loss function is used to balance the learning of the target old category sample data and the new category sample data. The target old category sample data includes the old category sample data and the second feature vector. The knowledge distillation loss function is used to measure the difference between the output results of the feature extraction layer of the old incremental learning model and the output results of the feature extraction layer of the new incremental learning model.

2. The category incremental learning method based on long-tail distribution scenarios as described in claim 1, characterized in that, The balance loss function includes: The first loss function is used to measure the classification layer output of the incremental learning model when the old category sample data is used as input; The second loss function is used to measure the classification layer output of the incremental learning model when the second feature vector is used as input.

3. The category incremental learning method based on long-tail distribution scenarios as described in claim 1, characterized in that, The balance loss function also includes: The third loss function is used to measure the difference between the class center feature vector of the old category sample data and the feature vector of the new category sample data output by the feature extraction layer of the incremental learning model.

4. The category incremental learning method based on long-tail distribution scenarios as described in claim 1, characterized in that, The incremental learning model also includes a correction layer connected to the classification layer; The total loss function also includes: The corrective loss function is used to measure the difference between the output of the classification layer and the output of the corrective layer of the new category sample data in the same incremental learning model.

5. The category incremental learning method based on long-tail distribution scenarios as described in claim 1, characterized in that, The step of generating a second feature vector using the first feature vector includes: Based on the first feature vector, obtain the class center vector of the old category sample data; Using the class center vector as a condition and the first feature vector as training data, the adversarial network is trained using random noise to obtain a trained adversarial network. The second feature vector is generated using the trained adversarial network.

6. The category incremental learning method based on long-tail distribution scenarios as described in claim 1, characterized in that, The construction of the sample dataset exhibiting a long-tailed distribution includes: For the first task, the sample data of each type within the first task will be distributed in a balanced manner; For non-first tasks, the sample data of each type within the non-first task are presented as a long-tail distribution.

7. A category incremental learning device based on long-tail distribution scenarios, characterized in that, include: A construction module is used to build a sample dataset that exhibits a long-tail distribution; wherein, the sample dataset is divided into old category sample data and new category sample data; The generation module is used to extract the feature vector of the old category sample data as the first feature vector, and use the first feature vector to generate the second feature vector; The learning module is used to train the incremental learning model using a preset total loss function, the sample dataset, and the second feature vector to obtain a trained incremental learning model. The incremental learning model comprises an input layer, a feature extraction layer, and a classification layer connected in sequence. The total loss function includes a balance loss function and a knowledge distillation loss function. The balance loss function is used to balance the learning of the target old category sample data and the new category sample data. The target old category sample data includes the old category sample data and the second feature vector. The knowledge distillation loss function is used to measure the difference between the output results of the feature extraction layer of the old incremental learning model and the output results of the feature extraction layer of the new incremental learning model.

8. A category incremental learning device based on long-tail distribution scenarios, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the category incremental learning method based on a long-tailed distribution scenario as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program; wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the category incremental learning method based on the long-tail distribution scenario as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes a computer program / instruction that, when executed by a processor, implements the category incremental learning method based on long-tailed distribution scenarios as described in any one of claims 1 to 6.

Images