A data processing method and device based on class incremental learning model

Abstract

The embodiment of the present application discloses a data processing method and device based on a class incremental learning model. The technical solution provided by the embodiment of the present application obtains a second label recognition model by adding a second label parameter on the basis of the original first label recognition model, and trains the second label recognition model using the first label data and the second label data, thereby ensuring the old label recognition capability and achieving better coverage of the new label recognition capability, effectively reducing the complexity and time consumption of model training, ensuring that the second label recognition model will not deviate too much from the original first label recognition model, thereby ensuring the recognition effect of the original first label data and improving the recognition effect of the label category.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to a data processing method and device based on a class incremental learning model. Background Art

[0002] With the rapid development of short video, live broadcast, news and information platforms, the amount of video and audio data is increasing day by day, which puts higher requirements on the review of audio and video content. At present, the review of audio and video content is mainly based on machine review, which filters out a large amount of invalid and illegal audio and video data through machine review, and only pushes a small amount of potential illegal data for manual review.

[0003] In the audit system, the audio and video data are generally identified and analyzed through the trained classification recognition model, and the corresponding label category is output. However, when a new audit type is proposed, it is often necessary to add new sample data and retrain a classification recognition model. The training process is cumbersome and inefficient. Summary of the invention

[0004] The embodiments of the present application provide a data processing method and device based on a class incremental learning model to solve the technical problems in the prior art that when a new audit type is proposed, the classification recognition model needs to be retrained, the training process is cumbersome, and the efficiency is low, thereby improving the recognition effect of label categories.

[0005] In a first aspect, an embodiment of the present application provides a data processing method based on a class incremental learning model, comprising:

[0006] Get the pending data for review;

[0007] Input the data to be processed into a second label recognition model, and the second label recognition model outputs the data label category corresponding to the data to be processed, the second label recognition model is obtained by adding a second label parameter to the first label recognition model and training with the first label data and the second label data, and the first label recognition model is obtained by training based on the first label data;

[0008] From the data to be processed, target data whose data label category is consistent with the set target category is screened out.

[0009] In a second aspect, an embodiment of the present application provides a data processing device based on a class incremental learning model, including a data acquisition module, a label recognition module and a data screening module, wherein:

[0010] The data acquisition module is used to acquire the data to be processed and to be reviewed;

[0011] The label recognition module is used to input the data to be processed into a second label recognition model, and the second label recognition model outputs the data label category corresponding to the data to be processed, the second label recognition model is obtained by adding a second label parameter to the first label recognition model, and training with the first label data and the second label data, and the first label recognition model is obtained by training based on the first label data;

[0012] The data screening module is used to screen out target data whose data label category is consistent with a set target category from the data to be processed.

[0013] In a third aspect, an embodiment of the present application provides a data processing device based on a class incremental learning model, including: a memory and one or more processors;

[0014] The memory is used to store one or more programs;

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method based on the class incremental learning model as described in the first aspect.

[0016] In a fourth aspect, an embodiment of the present application provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the data processing method based on the class incremental learning model as described in the first aspect.

[0017] The embodiment of the present application obtains a second label recognition model by adding a second label parameter to the original first label recognition model, and trains the second label recognition model using the first label data and the second label data. While ensuring the old label recognition capability, it achieves better coverage of the new label recognition capability, effectively reduces the complexity and time of model training, and ensures that the second label recognition model will not deviate too much from the original first label recognition model, thereby ensuring the recognition effect of the original first label data and improving the optimization efficiency of the label recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flow chart of a data processing method based on a class incremental learning model provided in an embodiment of the present application;

[0019] Figure 2 is a flowchart of another data processing method based on a class incremental learning model provided in an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a first tag recognition model provided in an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of a second tag recognition model provided in an embodiment of the present application;

[0022] Figure 5 is a flowchart of another data processing method based on a class incremental learning model provided in an embodiment of the present application;

[0023] Figure 6 A schematic diagram of a process for training a second tag recognition model provided in an embodiment of the present application;

[0024] Figure 7 It is a structural schematic diagram of a data processing device based on a class incremental learning model provided in an embodiment of the present application;

[0025] Figure 8 It is a structural diagram of a data processing device based on a class incremental learning model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0027] Figure 1 A flowchart of a data processing method based on a class incremental learning model provided in an embodiment of the present application is given. The data processing method based on a class incremental learning model provided in an embodiment of the present application can be executed by a data processing device based on a class incremental learning model. The data processing device based on a class incremental learning model can be implemented by hardware and / or software and integrated in a data processing device based on a class incremental learning model.

[0028] The following description is made by taking a data processing device based on a class incremental learning model to perform a data processing method based on a class incremental learning model as an example. Figure 1 , the data processing method based on the class incremental learning model includes:

[0029] S101: Obtaining data to be processed and to be reviewed.

[0030] The data to be processed for review can be audio, video, pictures and other data provided by short video, live broadcast, news and information platforms. There may be illegal content in these data to be processed, and it is necessary to use the label recognition model to conduct preliminary screening of the data to be processed, screen out the target data that may contain illegal content from the data to be processed, and then submit the target data to the manual review process for more accurate review.

[0031] Exemplarily, the acquisition of data to be processed can be performed when a user uploads new data (including complete data and real-time data stream). For example, when a user uploads new data to a short video, live broadcast, news information and other platforms, the new data uploaded by the user is used as content to be reviewed. The acquisition of data to be processed can also be performed by randomly extracting user-uploaded data from short video, live broadcast, news information and other platforms as content to be reviewed. Optionally, for data that has been identified by a label category, the status label corresponding to the data can be updated from unidentified to identified. When acquiring data to be processed, only data centers with a status label of unidentified are selected.

[0032] S102: The data to be processed is input into the second label recognition model, and the second label recognition model outputs the data label category corresponding to the data to be processed. The second label recognition model is obtained by adding a second label parameter to the first label recognition model and training with the first label data and the second label data. The first label recognition model is obtained by training based on the first label data.

[0033] Exemplarily, after the data to be processed is obtained, the data to be processed is input into the second label recognition model, the second label recognition model analyzes and processes the data to be processed, and outputs the data label category corresponding to the data to be processed.

[0034] The first label recognition model provided in this embodiment is trained based on the first label data, and the first label data records the corresponding first label category. For example, different first label categories are set according to different illegal contents, and the first label data are collected according to different first label categories. A first label recognition model is established based on a neural learning network, and the first parameter of the first label recognition model is set. The first label recognition model is trained using the first label data to optimize the first parameter. After the data to be processed is input into the first label recognition model, the first label recognition model will output the data label category corresponding to the data to be processed (the identifiable label category only includes the first label category).

[0035] Furthermore, a second label parameter is added to the first parameter of the first label recognition model to obtain a second parameter including the first parameter and the second label parameter, and the first label recognition model with the second label parameter added is used as the second label recognition model, and the second label recognition model is trained using the first label data and the second label data, thereby optimizing the second parameter, and finally obtaining a second label recognition model that can identify the first label category and the second label category. The second label data records the corresponding second label category. For example, the corresponding second label category is set according to the newly added illegal content as needed, the second label data is collected according to the second label category, and the second label data is added on the basis of the first parameter of the first label recognition model. At this time, the first parameter is updated to the second parameter, and the first label recognition model is updated to the second label recognition model with the second parameter, and the second label recognition model is trained using the first label data and the second label data to optimize the second parameter. After the data to be processed is input into the second label recognition model, the second label recognition model will output the data label category corresponding to the data to be processed (the identifiable label categories include the first label category and the second label category).

[0036] It is understandable that when the data to be processed contains multiple types of target content, the first label recognition model or the second label recognition model will output data label categories corresponding to multiple target contents. When the next illegal content is added and the label recognition model is updated again, the second label recognition model, the first label data and the second label data, the first label category and the second label category, and the second parameter completed in this training will be used as the first label recognition model, the first label data, the first label category, and the first parameter in the next data processing process based on the class incremental learning model.

[0037] S103: Filter out target data whose data label category is consistent with the set target category from the data to be processed.

[0038] After determining the data label category of the data to be processed, the data label category is compared with the set target category, and the data to be processed whose data label category is consistent with the set target category is screened out, and these data to be processed are used as target data. Further, the screened target data can be submitted to the manual review process for manual review operations. It can be understood that when the second label recognition model will output multiple data label categories, as long as there is a data label category that matches the target category threshold, the data to be processed can be screened as the target data.

[0039] In one embodiment, the target category can be a collection of the first label category and the second label category, or a collection of parts of the first label category and the second label category (for example, in subsequent reviews, if part of the target content is no longer set as illegal content, the corresponding target category will be deleted accordingly).

[0040] In the above, the second label recognition model is obtained by adding the second label parameter on the basis of the original first label recognition model, and the second label recognition model is trained using the first label data and the second label data. While ensuring the old label recognition capability, the new label recognition capability is better covered, which effectively reduces the complexity and time of model training, and ensures that the deviation of the second label recognition model relative to the original first label recognition model is not too large, thereby ensuring the recognition effect of the original first label data and improving the recognition effect of the label category.

[0041] Based on the above embodiments, Figure 2 A flowchart of another data processing method based on a class incremental learning model provided in an embodiment of the present application is given. The data processing method based on a class incremental learning model is a specific implementation of the data processing method based on the class incremental learning model described above. Figure 2 , the data processing method based on the class incremental learning model includes:

[0042] S201: Adding a second label parameter to a first label recognition model having a first parameter to obtain a second label recognition model, wherein a second parameter of the second label recognition model includes the first parameter and the second label parameter, and the first parameter includes a backbone network parameter and the first label parameter.

[0043] The first parameter is obtained by training the first label recognition model based on the first label data. Figure 3 A schematic diagram of a first tag recognition model provided in an embodiment of the present application is given, such as Figure 3 As shown, the first parameter set in the first tag recognition model provided in this embodiment includes a backbone network parameter (w b ) and the first label parameter (w o ), wherein the backbone network (backbone in the figure) is the basic network used to extract features in the first label recognition model, and the first label parameters are set according to the first label category (old label in the figure) to be recognized (the first label parameters correspond to the number of first label categories). After the first label recognition model is built, the first label recognition model is trained using the first label data to optimize the backbone network parameters and the first label parameters, wherein the first label data records the corresponding first label category (e.g., the content category contained in the audio and video data, such as people, animals, patterns, actions, item types, etc.), and the first label data may correspond to one or more different first label categories.

[0044] It is understandable that the trained first label recognition model can be put online in the machine review process, and the data to be processed (such as audio and video data, pictures, etc.) waiting to be recognized is input into the first label recognition model, and the first label recognition model analyzes and processes the data to be processed, and outputs the corresponding label recognition results. The label recognition results include one or more first label categories, which respectively reflect the content categories contained in the data to be processed. For example, a video containing people and animals is input into the first label recognition model, and the first label recognition model will output a label recognition result containing the first label categories corresponding to people and animals. According to the label recognition results, the content categories contained in the data to be processed can be determined.

[0045] In one embodiment, a target first label category that needs to be submitted to a manual review process is preset, and when the label recognition result corresponding to the data to be processed includes the target first label category, the data to be processed is pushed to the manual review process for further review.

[0046] When a new first label category needs to be added, the second label parameter is added to the original first label recognition model to obtain a second label recognition model. Figure 4 A schematic diagram of a second tag recognition model provided in an embodiment of the present application is given, combined with Figure 3 and Figure 4 , the second label recognition model adds a branch corresponding to the new first label category (new label in the figure) based on the original first label recognition model, and sets the parameters of the newly added branch as the second label parameter. Then, the second parameters set in the second label recognition model include the backbone network parameters and the first label parameters in the original first label recognition model, as well as the newly added second label parameters (w n ).

[0047] S202: Fix the backbone network parameters and the first label parameters, and train the second label recognition model using the second label data to optimize the second label parameters.

[0048] After obtaining the second label recognition model, the first parameter of the original first label recognition model in the second label recognition model is fixed, that is, the backbone network parameter and the first label parameter in the second label recognition model are fixed, and in the process of training the second label recognition model in step S202, the backbone network parameter and the first label parameter are kept unchanged. The second label data records the corresponding first label category, and the second label data corresponds to the first label category that needs to be added currently.

[0049] Further, the second label data is used to train the second label recognition model, that is, the second label recognition model is warmed up and initialized (warm up, a method of warming up the model is conducive to the convergence of subsequent model training) until the second label parameters in the second label recognition model converge, so as to achieve warm-up optimization of the second label parameters, so that the newly added second label parameters are better initialized, which is conducive to the subsequent optimization and convergence of the second label recognition model, and ensures the recognition performance of the new label category (second label category). Optionally, the warm-up initialization of the second label recognition model can be random initialization, Xavier initialization, Kaiming initialization, etc., which are not limited in this embodiment.

[0050] S203: Train a second label recognition model using the first label data and the second label data to optimize the second parameter.

[0051] Exemplarily, after completing the warm-up initialization of the second label recognition model, the second label recognition model is trained using the first label data and the second label data, that is, the first label data and the second label data are used as input, and the first label category corresponding to each label data of the first label data and the second label data is used as output to train the second label recognition model until the set model loss function is less than the set loss function threshold.

[0052] It can be understood that training the second label recognition model is a process of optimizing the second parameters, that is, the process of optimizing the backbone network parameters, the first label parameters and the second label parameters. When the set model loss function is less than the set loss function threshold, the backbone network parameters, the first label parameters and the second label parameters converge.

[0053] After completing the training of the second label recognition model, the second label recognition model can be put online in the machine review process, and the data to be processed to be recognized is input into the first label recognition model. The second label recognition model analyzes and processes the data to be processed, and outputs the corresponding label recognition result. At this time, the second label recognition model retains the recognition ability of the original first label recognition model for the old label category (first label category), and adds the recognition ability of the new label category (second label category). For example, the first label category that the original first label recognition model can recognize is people and animals. Then, after adding the second label parameter to the first label recognition model to obtain the second label recognition model, the second label recognition model is warmed up and initialized using the second label data containing the first label category of plants, and the second label recognition model is trained using the original first label data and the newly added second label data. After the training, the second label recognition model will have the ability to recognize the first label categories corresponding to people, animals and plants.

[0054] S204: Obtain the data to be processed and to be reviewed.

[0055] S205: Input the data to be processed into the second label recognition model, and the second label recognition model outputs the data label category corresponding to the data to be processed.

[0056] S206: Filter out target data whose data label category is consistent with the set target category from the data to be processed.

[0057] In the above, the second label recognition model is obtained by adding the second label parameter on the basis of the original first label recognition model, and the first label parameter and the backbone network parameter obtained by training the first label recognition model are fixed, and the second label recognition model is trained with the second label data to warm up and initialize the second label parameter, and then the second label recognition model is trained with the first label data and the second label data to optimize the backbone network parameters, the first label parameters and the second label parameters, and to achieve better coverage of the new label recognition capability while ensuring the old label recognition capability, and to achieve better initialization of the newly added second label parameter through warm-up and fine-tuning of the second label recognition model, so as to improve the subsequent convergence ability of the second label recognition model, effectively reduce the complexity and time of model training, and ensure that the second label recognition model will not deviate too much from the original first label recognition model, thereby ensuring the recognition effect of the original first label data and improving the optimization efficiency of the label recognition model.

[0058] Based on the above embodiments, Figure 5 A flowchart of another data processing method based on a class incremental learning model provided in an embodiment of the present application is given. The data processing method based on a class incremental learning model is a specific implementation of the data processing method based on the class incremental learning model described above. Figure 5 , the data processing method based on the class incremental learning model includes:

[0059] S301: Filtering first tag data corresponding to different first tag categories according to a set tag quantity.

[0060] Among them, different first label data correspond to different first label categories (each first label data records its corresponding first label category). It can be understood that when training the first label recognition model, the amount of first label data used for training is not limited. If the second label recognition model is still trained using the original amount of first label data, there will be a situation where the training efficiency of the second label recognition model is too low due to the excessive amount of the original first label data. Based on this, before training the second label recognition model after warm-up initialization, the first label data corresponding to different first label categories are also screened according to the set number of labels, so that the number of first label data under each first label category remains within the set number of labels.

[0061] It is understandable that when the number of first label data under a certain first label category is within the set label number, there is no need to screen the first label data under the first label category. When the screening of the first label data under a certain first label category is completed, the first label data under the first label category can be kept unchanged, and in the subsequent data processing process based on the class incremental learning model, the first label data under the newly added first label category can be screened. After completing the data processing process based on the class incremental learning model, the second label data in this process will become the first label data, and the second label recognition model will become the first label recognition model and be added to the next data processing process based on the class incremental learning model. The original first label data has been screened, and the newly added first label data (i.e., the second label data in this process) can be screened in the next data processing process based on the class incremental learning model. By screening the first label data, the amount of training data of the old label (first label data) can be controlled, and the main features of the old label can be retained.

[0062] In one embodiment, the first label data corresponding to different first label categories are screened by using a feature clustering screening method. Based on this, when the first label data corresponding to different first label categories are screened, the steps S3011-S3012 are specifically included:

[0063] S3011: Input the first label data into the first label recognition model to obtain a feature vector of each first label data.

[0064] Specifically, for first label categories whose data volume of first label data exceeds the set data volume, all first labels under these first label categories are input into the first label recognition model to obtain the feature vector output by the first label recognition model when processing each first label data. For example, all first labels under each first label category are passed through the first label recognition model to obtain the intermediate layer features (such as the one-dimensional vector output in the fully connected layer of the first label data, the size of the one-dimensional vector is 1x1024) output by the first label recognition model when processing each first label data.

[0065] S3012: Filtering first label data corresponding to different first label categories according to feature distances between feature vectors.

[0066] For each first label category, the first label data under each first label category is screened according to the characteristic distance between the characteristic vectors of each first label data. The first label data can be screened according to the characteristic distance by using a distance metric (such as the nearest neighbor). Based on this, step S3012 specifically includes steps S30121-S30122:

[0067] S30121: Determine the vector mean between the feature vectors of each first label data in each first label category.

[0068] S30122: For each first label category, the first label data are screened according to the feature distance between the feature vector of each first label data and the vector mean.

[0069] Specifically, for each first label category, determine the vector mean between the feature vectors corresponding to each first label data in each first label category. Furthermore, calculate the feature distance between the feature vector of each first label data and the vector mean of the corresponding first label category in each first label category, and filter out the first label data with a set number of labels in order of feature distance from small to large. The first label data filtered out by feature measurement retains most of the main features of the original first label data, reduces the complexity and time of training the subsequent second label recognition model, and ensures the recognition effect of the original first label data.

[0070] S302: Adding a second label parameter to a first label recognition model having the first parameter, to obtain a second label recognition model, wherein the second parameter of the second label recognition model includes the first parameter and the second label parameter, and the first parameter includes the backbone network parameter and the first label parameter.

[0071] S303: Fix the backbone network parameters and the first label parameters, and train the second label recognition model using the second label data to optimize the second label parameters.

[0072] S304: Train a second label recognition model using the first label data and the second label data to optimize the second parameter.

[0073] In one embodiment, the second label recognition model may be trained based on a set model loss function until the model loss of the second label recognition model reaches a preset loss function threshold. Based on this, the second label recognition model is trained, including:

[0074] S3041: Based on the set model loss function, the second label recognition model is trained, where the set model loss function includes a combination of one or more of a classification network loss function, an old model constrained distillation loss function, and a new and old model parameter constrained loss function.

[0075] Among them, the model loss function includes one or more combinations of the classification network loss function, the old model constraint distillation loss function, and the new and old model parameter constraint loss function. This embodiment takes the combination of the classification network loss function, the old model constraint distillation loss function, and the new and old model parameter constraint loss function as an example for description. The combination of the three loss functions can be a weighted summation method according to the set weights, and the weight values ​​of different loss functions can be set according to actual conditions. For example, the weight values ​​of each loss function are set to 1. Assume that the classification network loss function, the old model constraint distillation loss function, and the new and old model parameter constraint loss function are loss respectively. 1 、loss 2 and loss 3 , then the model loss function Loss = loss 1 +loss 2 +loss 3 .

[0076] In one embodiment, the classification network loss function is determined based on the following formula:

[0077] loss 1 =CE(y 1 ,y 2 )+R(w b ,w o ,w n )

[0078] Among them, loss 1 is the classification network loss function, CE(y 1 ,y 2) is the cross entropy loss function (including the cross entropy loss function of the second label recognition model under the first label data and the second label data), y 1 is the first label category or the second label category (real label category) corresponding to the first label data or the second label data, y 2 When the first label data or the second label data is input into the second label recognition model, the second label recognition model outputs the first label category or the second label category (predicted label category). It can be understood that the closer the predicted label category is to the actual label category, the smaller the cross entropy loss function is. Further, R(w b ,w o ,w n ) is the regularized loss function, where w b is the backbone network parameter, w o is the first label parameter, w n For the second label parameter, the regularization loss function can use L1 regularization, L2 regularization, etc. L1 regularization reduces the complexity of the model by sparse parameters (feature sparsification, reducing the number of weight parameters), and L2 regularization reduces the complexity of the model by reducing the numerical value of the weight. This embodiment incorporates the classification network loss function into the consideration of the model loss function, so that the prediction result after optimizing the second label recognition model is close to the real label category (the first label category or the second label category).

[0079] In one embodiment, the old model constrained distillation loss function is determined based on the following formula:

[0080] loss 2 =CE(y 3 ,y 2 )

[0081] Among them, loss 2 is the constraint distillation loss function of the old model, CE(y 3 ,y 2 ) is the cross entropy loss function with temperature coefficient (T) adjustment, y 2 The first label category corresponding to the first label data output by the second label recognition model, y 3 It is the first label category output by the first label recognition model. It needs to be explained that the second label recognition model can output the first label category corresponding to the first label data and the second label category corresponding to the second label data. In the calculation of the old model constrained distillation loss function, among the label categories output by the second label recognition model, only the first label category corresponding to the first label data participates in the calculation.

[0082] When the second label recognition model is first trained, the temperature coefficient is set to a larger first set value, so that the probability distribution predicted by the second label recognition model will be smoother, and the corresponding loss will be larger, thereby preventing the second label recognition model from falling into a local optimal solution. As the training proceeds, the temperature coefficient is reduced (cooling process) to make the second label recognition model converge. The value of the temperature coefficient T can be set to:

[0083]

[0084] Among them, T 0 is the first set value, and N is the number of training cycles.

[0085] This embodiment incorporates the old model constraint distillation loss function into the model loss function to ensure that the prediction result of the second label recognition model on the old label category (first label category) is close to the prediction result of the first label recognition model on the old label category (first label category), thereby ensuring the recognition effect of the old label category (first label category).

[0086] In one embodiment, the new and old model parameter constraint loss functions are determined based on the following formula:

[0087] loss 3 =MSE(w new ,wo ld )

[0088] Among them, loss 3 is the constraint loss function of the new and old model parameters, MSE(w new ,wo ld ) is the mean square error loss function, w old is the first parameter, w new The second parameter includes at least a backbone network parameter and a first label parameter. It needs to be explained that the second parameter also includes a second label parameter, that is, the second parameter includes a backbone network parameter, a first label parameter and a second label parameter. In the calculation of the new and old model parameter constraint loss function, only the backbone network parameter and the first label parameter in the second parameter participate in the calculation. This embodiment incorporates the new and old model parameter constraint loss function into the consideration of the model loss function to ensure that the parameters of the second label recognition model and the first label recognition model are close, and reduce the situation where the effect of the second label recognition model in predicting the old label category (first label category) decreases due to excessive parameter changes.

[0089] Figure 6 A schematic diagram of a process for training a second tag recognition model provided in an embodiment of the present application is given, such as Figure 6As shown, the second label parameter is added to the first label model (the first label model is provided with the backbone network parameters and the first label parameters obtained through training optimization) to obtain the second label recognition model, and the parameters contained in the second label recognition model include the backbone network parameters, the first label parameters and the second label parameters. The backbone network parameters and the first label parameters are fixed, and the second label recognition model is warmed up using the second label data until the second label recognition model converges to obtain the initialized second label parameters. The second label data is marked with the corresponding new first label category.

[0090] In addition, the first label data previously used to train the first label recognition model is passed through the first label recognition model to extract the feature vector of each first label data, and determine the limited mean corresponding to the feature vector of each first label data in each first label category. In each first label category, the first label data with a set number of labels are screened according to the feature distance between the feature vector of each first label data and the vector mean. It can be understood that there is no definite order between the steps of warming up and initializing the second label recognition model and screening the first label data, and the two can be executed synchronously or asynchronously. The first label data is marked with the corresponding old first label category.

[0091] Furthermore, after completing the warm-up initialization of the second label recognition model and screening the first label data, the screened first label data and second label data are used to train the second label recognition model after warm-up initialization. At this time, the backbone network parameters and the first label parameters are no longer fixed, and the backbone network parameters, the first label parameters and the second label parameters are optimized at the same time. And in the training process, based on the model loss function (including the classification network loss function, the old model constraint distillation loss function and the new and old model parameter constraint loss function), it is judged whether the training of the second label recognition model is completed. After completing the training of the second label recognition model, the second label recognition model can be put online in the machine review process, and the data to be processed to be recognized is input into the first label recognition model. The second label recognition model analyzes and processes the data to be processed to obtain the corresponding data label category, and screens out the target data whose data label category is consistent with the set target category from the data to be processed.

[0092] S305: Obtain the data to be processed for review.

[0093] S306: Input the data to be processed into the second label recognition model, and the second label recognition model outputs the data label category corresponding to the data to be processed.

[0094] S307: Filter out target data whose data label category is consistent with the set target category from the data to be processed.

[0095] For example, audio, video, and pictures provided by short video, live broadcast, news and information platforms need to be reviewed, and the first label recognition model is trained using the first label data marked with the set target category, where the target category reflects the illegal content that needs to be further reviewed or screened out. The data to be processed that needs to be reviewed is input into the first label recognition model, and the first label category corresponding to each piece of data to be processed is output.

[0096] When it is necessary to add a new type of illegal content that needs to be reviewed, second sample data marked with a second label category is collected according to the newly added illegal content, and the second label parameter is added to the original first label recognition model to obtain a second label recognition model. After warming up and optimizing the second label recognition model, the first label data and the second label data are used to train the second label recognition model.

[0097] When receiving the pending data that needs to be reviewed later, the pending data can be input into the trained second label recognition model, and the second label recognition model outputs the corresponding data label category (the identifiable label category includes the first label category and the second label category). The pending data can be screened according to the correspondence between the data label category and the target label category, and the target data screened out from the pending data can be submitted to the manual review process for manual review. On the basis of maintaining the original label category recognition capability, the ability to discover new illegal label categories online is increased to reduce the pressure of manual review. On the basis of the original first label recognition model, the label recognition capability of the label recognition model is expanded to reduce the time consumed by model training. On the premise of ensuring the original label recognition effect, no new machine consumption is added, and the recognition of more illegal content is covered, further reducing the burden on the manual review side.

[0098] In the above, the second label recognition model is obtained by adding the second label parameters on the basis of the original first label recognition model, and the first label parameters and backbone network parameters originally obtained by training the first label recognition model are fixed, and the second label recognition model is trained using the second label data to warm up and initialize the second label parameters, and then the second label recognition model is trained using the first label data and the second label data to optimize the backbone network parameters, the first label parameters and the second label parameters, while ensuring the old label recognition capability, to achieve better coverage of the new label recognition capability, and through the warm-up fine-tuning of the second label recognition model, the newly added second label parameters have a better initialization, thereby improving the subsequent convergence ability of the second label recognition model. At the same time, by screening the first label data, the amount of training data for the first label data can be controlled, and the main features of the first label data can be retained, reducing the complexity and time of subsequent training of the second label recognition model, ensuring the recognition effect of the original first label data, covering more new labels without reducing the old label ability, and realizing real-time learning of new label category (second label category) data by the model, effectively reducing the time-consuming and inefficient retraining of the model when the number of new and old label data sets (second label data and first label data) is huge, and consuming a large amount of computing and storage resources. At the same time, through the old model constraint distillation loss function and the new and old model parameter constraint loss function to realize the constraint on the original first parameter, realize the offset constraint of the second label recognition model relative to the first label recognition model, and ensure the recognition effect of the old label category (first label category). By utilizing the joint training method of the new and old models that combines the second label recognition model with the first label recognition model, as well as the adaptive settings of different loss functions, the complexity of model training can be effectively reduced, and a balance can be achieved in the recognition effects of the new and old label categories (first label categories). The label recognition model can be expanded efficiently and quickly, and the review process can reduce machine consumption while ensuring the recognition effect of the original categories, cover more first label categories, and further reduce the burden on the manual review side.

[0099] Figure 7 A schematic diagram of the structure of a data processing device based on a class incremental learning model provided in an embodiment of the present application is given. Figure 7 The data processing device based on the class incremental learning model includes a data acquisition module 31, a label recognition module 32 and a data screening module 33.

[0100] Among them, the data acquisition module 31 is used to obtain the processed data to be reviewed; the label recognition module 32 is used to input the processed data into the second label recognition model, and the second label recognition model outputs the data label category corresponding to the processed data. The second label recognition model is obtained by adding a second label parameter to the first label recognition model and training with the first label data and the second label data. The first label recognition model is obtained by training based on the first label data; the data screening module 33 is used to screen out the target data whose data label category is consistent with the set target category from the processed data.

[0101] An embodiment of the present application also provides a data processing device based on a class incremental learning model, and the data processing device based on a class incremental learning model can integrate the data processing device based on a class incremental learning model provided in an embodiment of the present application. Figure 8 Schematic diagram of a data processing device based on a class incremental learning model provided by an embodiment of the present application. Figure 8 , the data processing device based on the class incremental learning model includes: an input device 43, an output device 44, a memory 42 and one or more processors 41; the memory 42 is used to store one or more programs; when one or more programs are executed by one or more processors 41, the one or more processors 41 implement the data processing method based on the class incremental learning model provided in the above embodiment. The above-mentioned data processing device, equipment and computer based on the class incremental learning model can be used to execute the data processing method based on the class incremental learning model provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0102] The embodiment of the present application also provides a storage medium containing computer executable instructions, which are used to execute the data processing method based on the class incremental learning model provided in the above embodiment when executed by a computer processor. Of course, the storage medium containing computer executable instructions provided in the embodiment of the present application, whose computer executable instructions are not limited to the data processing method based on the class incremental learning model as above, can also execute the relevant operations in the data processing method based on the class incremental learning model provided in any embodiment of the present application. The data processing device, equipment and storage medium based on the class incremental learning model provided in the above embodiment can execute the data processing method based on the class incremental learning model provided in any embodiment of the present application. For the technical details not described in detail in the above embodiment, please refer to the data processing method based on the class incremental learning model provided in any embodiment of the present application.

[0103] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the protection scope of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A data processing method based on a class incremental learning model, It is characterized in that include: Obtaining data to be processed for review, wherein the data to be processed includes one of audio, video and picture; Input the data to be processed into a second label recognition model, and the second label recognition model outputs the data label category corresponding to the data to be processed, the second label recognition model is obtained by adding a second label parameter to the first label recognition model and training with the first label data and the second label data, and the first label recognition model is obtained by training based on the first label data; Filtering out target data whose data label category is consistent with a set target category from the data to be processed; The training process of the second label recognition model includes: adding the second label parameter to the first label recognition model set with the first parameter to obtain the second label recognition model, the second parameter of the second label recognition model includes the first parameter and the second label parameter, and the first parameter includes the backbone network parameter and the first label parameter; fixing the backbone network parameter and the first label parameter, and using the second label data to warm up and initialize the second label recognition model to optimize the second label parameter; using the first label data and the second label data, training the second label recognition model according to the set model loss function to optimize the second parameter, the set model loss function includes the classification network loss function, the old model constrained distillation loss function and the new and old model parameter constrained loss function, wherein, among the label categories output by the second label recognition model, the first label category corresponding to the first label data participates in the calculation of the old model constrained distillation loss function, the old model constrained distillation loss function is a cross entropy loss function with temperature coefficient adjustment, and the temperature coefficient decreases as the training progresses.

2. The data processing method based on the class incremental learning model according to claim 1, It is characterized in that Different first label data correspond to different first label categories. Before training the second label recognition model, the method further includes: The first tag data corresponding to different first tag categories are filtered according to the set tag quantity.

3. The data processing method based on the class incremental learning model according to claim 2, It is characterized in that The screening of the first label data corresponding to different first label categories includes: Inputting the first label data into the first label recognition model to obtain a feature vector of each first label data; The first label data corresponding to different first label categories are screened according to the feature distance between the feature vectors.

4. The data processing method based on the class incremental learning model according to claim 3, It is characterized in that The screening of the first label data corresponding to different first label categories according to the feature distance between the feature vectors includes: Determine a vector mean between the feature vectors of each of the first label data in each first label category; For each of the first label categories, the first label data are screened according to the feature distance between the feature vector of each of the first label data and the vector mean.

5. The data processing method based on the class incremental learning model according to claim 1, It is characterized in that The classification network loss function is determined based on the following formula: loss 1 =CE(y 1 ,y 2 )+R(w b ,w o ,w n ) Among them, loss 1 is the classification network loss function, CE(y 1 ,y 2 ) is the cross entropy loss function, R(w b ,w o ,w n ) is the regularization loss function, y 1 is the first label category or the second label category corresponding to the first label data or the second label data, y 2 is the first label category or the second label category output by the second label recognition model, w b is the backbone network parameter, w o is the first label parameter, w n is the second tag parameter.

6. A data processing device based on a class incremental learning model, It is characterized in that It includes a data acquisition module, a tag identification module and a data screening module, among which: The data acquisition module is used to acquire the data to be processed for review, wherein the data to be processed includes one of audio, video and picture; The label recognition module is used to input the data to be processed into a second label recognition model, and the second label recognition model outputs the data label category corresponding to the data to be processed, the second label recognition model is obtained by adding a second label parameter to the first label recognition model, and training with the first label data and the second label data, and the first label recognition model is obtained by training based on the first label data; The data screening module is used to screen out target data whose data label category is consistent with a set target category from the data to be processed; The training process of the second label recognition model includes: adding the second label parameter to the first label recognition model set with the first parameter to obtain the second label recognition model, the second parameter of the second label recognition model includes the first parameter and the second label parameter, and the first parameter includes the backbone network parameter and the first label parameter; fixing the backbone network parameter and the first label parameter, and using the second label data to warm up and initialize the second label recognition model to optimize the second label parameter; using the first label data and the second label data, training the second label recognition model according to the set model loss function to optimize the second parameter, the set model loss function includes the classification network loss function, the old model constrained distillation loss function and the new and old model parameter constrained loss function, wherein, among the label categories output by the second label recognition model, the first label category corresponding to the first label data participates in the calculation of the old model constrained distillation loss function, the old model constrained distillation loss function is a cross entropy loss function with temperature coefficient adjustment, and the temperature coefficient decreases as the training progresses.

7. A data processing device based on a class incremental learning model, It is characterized in that include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method based on the class incremental learning model as described in any one of claims 1 to 5.

8. A storage medium containing computer executable instructions, It is characterized in that When the computer executable instructions are executed by a computer processor, they are used to execute the data processing method based on the class incremental learning model as described in any one of claims 1 to 5.

Images