A data processing method, apparatus, device, and medium

Abstract

A data processing method, device, equipment and medium are disclosed. The method comprises: in response to a data processing request instruction, determining a data segment in which target data is located; performing a deletion operation on the data segment in which the target data is located, and finding the last data segment before the data segment in which the target data is located; and starting from the last data segment, retraining a pre-created target training model to obtain a new target training model, wherein the pre-created target training model is a model obtained by training an original training model according to low-forgetting-probability data segments and high-forgetting-probability data segments divided according to forgetting probability. The embodiment solves the problem that the SISA model in the prior art cannot fully utilize forgetting probability information, improves the prediction accuracy of the target training model after aggregation, speeds up the retraining of the model, and improves the usability of the model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer technology, and in particular to a data processing method and device, equipment and medium. BACKGROUND

[0002] In the training process of a machine learning model, a large amount of data needs to be used. These data may contain a large amount of user privacy data, such as a large amount of personal information of past patients in a data set used for training a medical diagnosis model; and a large amount of user data on the Internet in a data set of a recommendation system. The use of a large amount of data improves the training effect of the machine learning model, but also brings many security and privacy problems.

[0003] A data poison attack is an attack in which an attacker inputs carefully designed data into a model training to disturb the real distribution of the data, so that the model obtains an incorrect output. Once the model receives a poison attack, the accuracy of the model will decrease, and even the output result will be skewed in the direction expected by the attacker. Domestic and foreign Internet companies have all been subjected to various data poison attacks on machine learning models.

[0004] A membership inference attack is an attack in which an attacker analyzes a published machine learning model to infer the training set of the model. For example, an attacker can obtain the private information of a target by judging whether the target is in the data set of a certain disease diagnosis model. Since this type of attack does not require a specific model structure, it only needs to call the machine learning model interface provided by a large Internet company on the network, it therefore poses a great threat to privacy, and many users hope that the model training party will delete their data from the data set and eliminate the influence of their data on the trained model.

[0005] To counter data poison attacks, many studies have proposed to improve the robustness of the model to prevent training data poisoning. However, for user requests to delete data and its resulting impact, previous mechanisms, including differential privacy, cannot perfectly implement the right to be forgotten. Therefore, researchers have begun to use machine unlearning to satisfy the right to be forgotten of users.

[0006] The most straightforward way to implement machine unlearning is to delete the samples to be destroyed from the original training dataset and retrain the model from scratch. However, when the dataset is large and the revocation requests occur frequently, retraining from scratch incurs a very high computational overhead. According to the concept proposed by Cao et al., machine unlearning learning needs to meet two requirements: 1) complete unlearning: for the data destruction request issued by the user, the existing model needs to be forgotten by the data to obtain a new model, which needs to have the same prediction results as the model obtained by retraining from scratch; 2) timely: compared with retraining, unlearning learning needs to reduce the computational overhead, and more quickly realize the system redeployment to obtain higher availability.

[0007] Cao et al. proposed to transform the learning algorithm into the summation form after the statistical query learning, and decompose the dependency between the training data. To delete a data instance, the model owner only needs to delete the transformation of the data instance from the summation that depends on this instance. However, the algorithm of Cao and Yang is not applicable to learning algorithms that cannot be transformed into the summation form, such as neural networks.

[0008] Ginart et al. studied the machine unlearning technology for the k-means clustering algorithm, the core of which is to propose a learning algorithm efficient for data deletion. Specifically, the algorithm constructs data deletion into an online problem, and gives the time efficiency analysis of the optimal deletion efficiency. The data deletion operation for learning algorithm A can be defined as R A (D,A(D),i), which takes the dataset D, the machine learning model A(D), and a certain model in the hypothesis space according to the index i∈{1,…,n}. A data deletion operation can be defined as if for all D and i, the random variables A(D i ) and R A (D,A(D),i) are equivalent in distribution, then A(D i ) = d R A (D,A(D),i). However, this technology cannot be generalized to other machine learning models.

[0009] Therefore, Bourtoule et al. proposed a more general algorithm SISA (Sharded, Isolated, Sliced and Aggregated) for deep learning. The main idea of SISA is to divide the training data into several disjoint shards, and train a sub-model for each shard. To delete a specific instance, the model owner only needs to retrain the sub-model containing this instance. In order to further speed up the forgetting process, the author proposes to divide each shard into several slices and store the intermediate model parameters when updating each slice. However, the SISA model cannot make good use of the forgetting probability information when applied in the scenario where the data forgetting probability is known. The special shard method proposed by Bourtoule et al. for the scenario where the data forgetting probability is known makes the model prediction accuracy decrease, reducing the usability of the model. SUMMARY

[0010] The present application provides a data processing method, device, equipment and medium, to solve the problem that the SISA model in the prior art cannot fully utilize the forgetting probability information, improve the prediction accuracy of the model, and improve the usability of the model.

[0011] According to an aspect of the present application, a data processing method is provided, comprising:

[0012] In response to a data processing request instruction, determining a data segment where target data is located, wherein the target data is data in an original data set that is about to perform a data deletion operation;

[0013] Performing a deletion operation on the data segment where the target data is located, and finding the last data segment before the data segment where the target data is located;

[0014] Starting from the last data segment, retraining a pre-created target training model to obtain a new target training model, wherein the pre-created target training model is a model obtained by training an original training model according to low forgetting probability data segments and high forgetting probability data segments divided according to forgetting probability, and the number of segments of the low forgetting probability data segments and the high forgetting probability data segments is equal.

[0015] According to another aspect of the present application, a data processing device is provided, comprising:

[0016] A determining module for determining a data segment where target data is located in response to a data processing request instruction, wherein the target data is data in an original data set that is about to perform a data deletion operation;

[0017] The processing module is configured to perform a deletion operation on the data segment where the target data is located, and find the last data segment before the data segment where the target data is located.

[0018] The retraining module is configured to retrain a pre-created target training model to obtain a new target training model, starting from the last data segment, wherein the pre-created target training model is a model obtained by training an original training model using low-forgetting-probability data segments and high-forgetting-probability data segments divided according to forgetting probabilities, and wherein the number of the low-forgetting-probability data segments is equal to the number of the high-forgetting-probability data segments.

[0019] According to another aspect of the present application, an electronic device is provided, the electronic device comprising:

[0020] at least one processor; and

[0021] a memory connected to the at least one processor in communication; wherein

[0022] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the data processing method according to any one of the embodiments of the present application.

[0023] According to another aspect of the present application, a computer readable storage medium is provided, the computer readable storage medium stores computer instructions for enabling a processor to implement the data processing method according to any one of the embodiments of the present application when executed by the processor.

[0024] The technical solution of the embodiments of the present application divides original data into low-forgetting-probability data segments and high-forgetting-probability data segments according to forgetting probabilities, and trains an original training model using low-forgetting-probability data segments and high-forgetting-probability data segments with approximately the same total amount of data to obtain a target training model with a higher prediction accuracy, so that when a data processing request instruction is received, the data segment where the target data is located can be directly deleted, and the pre-created target training model can be retrained starting from the last data segment before the data segment where the target data is located, thereby solving the problem that the SISA model in the prior art cannot fully utilize forgetting probability information, improving the prediction accuracy of the aggregated target training model, speeding up the retraining of the model, and improving the usability of the model.

[0025] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0027] Figure 1 A flow chart of a data processing method provided for the first embodiment of the present application;

[0028] Figure 2 A flow chart of a data processing method provided for the second embodiment of the present application;

[0029] Figure 3 A flow chart of a data processing method provided for the third embodiment of the present application;

[0030] Figure 4 A research flow chart of fine-grained data destruction using a machine learning model provided by the embodiments of the present application;

[0031] Figure 5 A flow chart of a fine-grained data processing algorithm provided by the embodiments of the present application;

[0032] Figure 6 A prediction accuracy of two models corresponding to a Purchase data set in different sharding cases provided by the embodiments of the present application;

[0033] Figure 7 A prediction accuracy of two models corresponding to an SVHN data set in different sharding cases provided by the embodiments of the present application;

[0034] Figure 8 An illustration of the number of data points affected in the data deletion stage provided by the embodiments of the present application;

[0035] Figure 9 A structural schematic diagram of a data processing device provided for the fourth embodiment of the present application;

[0036] Figure 10 A structural schematic diagram of an electronic device provided by the fifth embodiment of the present application. DETAILED DESCRIPTION

[0037] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0038] It should be noted that the terms "original," "intermediate," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0039] It should be noted that the relevant parameters involved in the embodiments of the present invention will be explained in detail below:

[0040] Randomness in Machine Learning: The goal of machine learning algorithms is to learn a randomness from the sample space. to mark space The mapping. According to the Probably Approximately Correct (PAC) learning theory, if for a mapping c, c(x) = y holds for any sample (x, y), then c is called the target concept. Learning Algorithm Even if you know nothing about the target concept, you can access a known dataset. Characterize the data distribution. For learning algorithms. The set of all possible concepts considered is called the "hypothesis space". This indicates that any possible value in the hypothesis space... This is called a hypothesis. For a given dataset... Learning Algorithms By solving the objective function, we obtain a hypothesis h that is as close as possible to the target concept c.

[0041] In the process of solving the learning algorithm, the randomness of the result mainly comes from two aspects: the randomness of the training process and the randomness of the learning algorithm.

[0042] Randomness in the training process: given a dataset In this process, it is usually necessary to first randomly sample small batches of data from the dataset, and the order in which the data is sampled varies in different training sessions. Furthermore, training is typically parallel and without explicit synchronization, meaning that the random data acquisition order during parallel training can make the training process unpredictable.

[0043] The randomness of learning algorithms: Intuitively, the goal of learning algorithms is to operate within a broad hypothesis space. The goal is to find the optimal hypothesis h. This hypothesis is typically defined by setting fixed parameter weights for the learning model. PAC learning theory posits that the hypothesis h, which is as close as possible to the target concept c, is one of many hypotheses that minimize empirical risk. However, commonly used optimization functions, such as stochastic gradient descent, converge to only one of several local minima for any convex loss function. Furthermore, the randomness involved in training, even with the same dataset... Obtaining the same final hypothesis h using the same learning algorithm is very challenging.

[0044] Due to the randomness in machine learning, it is difficult to quantify the impact of certain data points on the model, making it difficult to remove them from the final model.

[0045] The goal of forgetting learning: Although the goal of forgetting learning is very clear—to extract the influence of certain data in the dataset within an existing model—the task of forgetting is not simple to implement due to the randomness of machine learning algorithms. Existing forgetting learning methods all have various problems, and an ideal forgetting learning method should meet the following requirements:

[0046] Understandability: Since basic retraining methods are easy to understand and implement, any forgetting learning algorithm should be easy to understand and simple to apply and correct for non-experts.

[0047] Availability: If a large amount of data needs to be deleted, or if representative data points need to be removed, a decrease in model accuracy is understandable. Even retraining the model from scratch will cause an accuracy drop when data is destroyed. A good forgetting learning method should be able to control this drop to a level similar to that of retraining the model.

[0048] Destruction should be provable: Like retraining, forgetting learning should be able to prove that deleted data points no longer affect model parameters. Furthermore, this proof should be concise and require no expert assistance.

[0049] Applicability: any machine learning model should be applicable to a good forgetting learning method, regardless of the complexity or other properties of the model.

[0050] Reducing forgetting time: the forgetting learning method should be faster than retraining in any case.

[0051] No additional overhead: any available forgetting learning method should not introduce additional computational overhead in the original complex model training process.

[0052] The present application proposes a new machine forgetting learning method for data processing to improve the prediction accuracy of the model, and realizes the compromise between the forgetting speed and the model accuracy in the forgetting process. By analyzing the forgetting probability of known data, the original data set is divided into low forgetting probability data segments and high forgetting probability data segments. Learning models are established for the two types of data. Since existing research shows that transfer learning can reduce the accuracy decline caused by using machine learning forgetting algorithm, the present application can first use the SISA model to train low forgetting probability data and then use a method similar to transfer learning to adjust the high forgetting probability data based on the existing model. Finally, the target training model is obtained through the aggregation method.

[0053] Embodiment one

[0054] Figure 1 A flowchart of a data processing method provided by the first embodiment of the present application. The present embodiment can be applied to the case of data deletion under the condition that the data deletion request probability is known. The method can be executed by a data processing device, which can be realized in the form of hardware and / or software. The data processing device can be configured in a terminal device. Exemplarily, the terminal device can be a computer, iPad, or other terminal with data processing function. As shown in the figure, the method comprises S110-S130. Figure 1

[0055] S110, in response to the data processing request instruction, determining the data segment where the target data is located.

[0056] ​The target data is data in the original data set that is about to perform a data deletion operation. The data processing request instruction refers to an instruction to delete part of the data in the original data set. The instruction to delete part of the data in the original data set can be understood as an instruction to forget or destroy part of the data in the original data set. In an embodiment, upon receiving the instruction of the data deletion operation, the data segment in which the data about to perform the data deletion operation is located is determined. The data segment refers to a segment that is divided in advance according to the forgetting probability of the data. In an embodiment, the data segment includes a low-forgetting-probability data segment and a high-forgetting-probability data segment. It can be understood that the data segment in which the target data is located can be a low-forgetting-probability data segment or a high-forgetting-probability data segment. Of course, in order to facilitate the deletion of the target data and the subsequent retraining of the model, the target data can only belong to one data segment.

[0057] S120, performing a deletion operation on the data segment in which the target data is located, and finding the last data segment before the data segment in which the target data is located.

[0058] In an embodiment, after determining the data segment in which the target data is located, the data segment in which the target data is located is directly deleted from the original data set, and the last data segment before the data segment in which the target data is located is found. Illustratively, assuming that the original data set is divided into 10 data segments, data segment 1, data segment 2, data segment 3, …, data segment 9, and data segment 10, and the data segment in which the target data is located is data segment 3, then the last data segment before the data segment in which the target data is located is data segment 2.

[0059] S130, retraining the pre-created target training model from the last data segment to obtain a new target training model.

[0060] The pre-created target training model is a model obtained by training the original training model according to the low-forgetting-probability data segment and the high-forgetting-probability data segment divided according to the forgetting probability, wherein the number of the low-forgetting-probability data segment and the high-forgetting-probability data segment is equal.

[0061] In the embodiment, the process of retraining the pre-created target training model can refer to the process of training the original training model to obtain the target training model. It should be noted that the process of training the original training model to obtain the target training model includes: dividing the original data set according to the forgetting probability to obtain a plurality of low-forgetting-probability data segments and a plurality of high-forgetting-probability data segments, then training the original training model by using a low-forgetting-probability data segment and a high-forgetting-probability data segment in sequence to obtain a corresponding intermediate isolated training model, and then performing aggregation processing on all intermediate isolated training models to obtain a corresponding target training model.

[0062] In the embodiment, after the data segment where the target data is located is deleted from the original data set, the pre-created target training model is retrained from the last data segment to obtain a new target training model. For example, in the example in S120, the process of retraining is explained, that is, the pre-created target training model is retrained from the data segment 2, specifically, the model is trained by using the data segment 1, the data segment 2 and the data segment 4 to obtain a corresponding intermediate isolated training model (denoted as model 1); then the model 1 is trained by using the data segment 1, the data segment 2, the data segment 4 and the data segment 5 to obtain a corresponding intermediate isolated training model (denoted as model 2), and the like is sequentially performed until the incremental training of the model is completed to obtain a plurality of corresponding intermediate isolated training models, and then the plurality of intermediate isolated training models are aggregated to obtain a new target training model.

[0063] The technical scheme of the embodiment divides the original data into low-forgetting-probability data segments and high-forgetting-probability data segments according to the forgetting probability, and trains the original training model by using low-forgetting-probability data segments and high-forgetting-probability data segments with approximately the same total amount of data to obtain a target training model with high prediction accuracy, so that when a data processing request instruction is received, the pre-created target training model can be retrained by directly deleting the data segment where the target data is located and starting from the last data segment before the data segment where the target data is located, thereby solving the problem that the SISA model in the prior art cannot fully utilize the forgetting probability information, improving the prediction accuracy of the aggregated target training model, speeding up the retraining of the model, and improving the usability of the model.

[0064] Embodiment Two

[0065] Figure 2 A flowchart of a data processing method provided for the embodiment two of the application, the embodiment is further refined on the basis of the division process of the original data set, the retraining process of the original training model, and the aggregation process. For example, Figure 2As shown, the method comprises:

[0066] S210, dividing the original data set to obtain corresponding low-forgetting-probability data segments and high-forgetting-probability data segments.

[0067] The number of segments of the low-forgetting-probability data segments and the high-forgetting-probability data segments is equal. It can be understood that the number of low-forgetting-probability data segments and the number of high-forgetting-probability data segments in the original data set are the same, the intersection of the low-forgetting-probability data segments and the high-forgetting-probability data segments is empty, and the union of the low-forgetting-probability data segments and the high-forgetting-probability data segments is the original data set. It should be noted that the amount of data contained in each low-forgetting-probability data segment is the same, and the amount of data contained in each high-forgetting-probability data segment is the same, but the amount of data contained in the low-forgetting-probability data segments and the high-forgetting-probability data segments is not equal, and in general, the amount of data contained in each low-forgetting-probability data segment is greater than the amount of data contained in each high-forgetting-probability data segment.

[0068] In an embodiment, the saved data information is used to locate the data segment where the data to be forgotten is located, and the last data segment before the data segment is found, and the original training model is divided from the data segment.

[0069] S220, sequentially using the low-forgetting-probability data segments and the high-forgetting-probability data segments to train the original training model to obtain corresponding intermediate isolated training models.

[0070] The total number of intermediate isolated training models is equal to the number of low-forgetting-probability data segments and high-forgetting-probability data segments, and the intermediate isolated training models, the low-forgetting-probability data segments, and the high-forgetting-probability data segments are one-to-one corresponding.

[0071] In an embodiment, in order to ensure the isolation between the respective models, a single model is trained on only one data segment. It can be understood that the original training model is retrained using a low-forgetting-probability data segment and a high-forgetting-probability data segment respectively to obtain a corresponding intermediate isolated training model. Exemplarily, first, the original training model is pre-trained using the first low-forgetting-probability data segment to obtain a corresponding pre-processed isolated training model (denoted as the first pre-processed isolated training model), and then the first pre-processed isolated training model is fine-tuned based on the first high-forgetting-probability data segment to obtain a corresponding intermediate isolated training model (denoted as the first intermediate isolated training model). Similarly, on the basis of the first pre-processed isolated training model, the first pre-processed isolated training model is trained using the union of the first low-forgetting-probability data segment and the second low-forgetting-probability data segment to obtain a corresponding pre-processed isolated training model (denoted as the second pre-processed isolated training model), and then the second pre-processed isolated training model is fine-tuned based on the second high-forgetting-probability data segment to obtain a corresponding intermediate isolated training model (denoted as the second intermediate isolated training model). By analogy, until the training of the original training model using all low-forgetting-probability data segments and high-forgetting-probability data segments is completed.

[0072] S230, performing aggregation processing on all intermediate isolated training models using a preset aggregation algorithm to obtain a corresponding target training model, so as to predict the target data using the target training model.

[0073] The preset aggregation algorithm refers to an algorithm for aggregating all intermediate isolated training models to obtain a final target training model. Exemplarily, the preset aggregation algorithm can include: majority voting method; simple average method. In an embodiment, a plurality of intermediate isolated training models are aggregated using the preset aggregation algorithm to obtain a corresponding target training model.

[0074] S240, in response to a data processing request instruction, determining a data segment where the target data is located.

[0075] The target data is data that is about to perform a data deletion operation.

[0076] S250, performing a deletion operation on the data segment where the target data is located, and finding the last data segment before the data segment where the target data is located.

[0077] S260, retraining the pre-created target training model from the last data segment to obtain a new target training model.

[0078] The target training model is a model obtained by training an original training model according to the low-forgetting-probability data segments and the high-forgetting-probability data segments divided according to the forgetting probability, wherein the number of the low-forgetting-probability data segments is equal to the number of the high-forgetting-probability data segments.

[0079] The technical scheme of the embodiment is based on the above-mentioned embodiment, and the forgetting probability corresponding to the original data is obtained by analyzing the data source of the original data set, the original data is divided into low-forgetting-probability data segments and high-forgetting-probability data segments according to the forgetting probability, the original model is trained by using the low-forgetting-probability data segments and the high-forgetting-probability data segments, a plurality of intermediate isolated training models with similar learning abilities are obtained, and the plurality of intermediate isolated training models are aggregated by using a preset aggregation algorithm to obtain a corresponding target training model, thereby solving the problem that the SISA model in the prior art cannot fully utilize the forgetting probability information, improving the prediction accuracy of the aggregated target training model, and improving the usability of the model.

[0080] Embodiment Three

[0081] Figure 3 A flowchart of a data processing method provided by Embodiment Three of the present application is further refined in the division process of the original data set, the retraining process of the original training model, and the aggregation process based on the above-mentioned embodiment. As shown in Figure 3 , the method comprises:

[0082] S310, dividing the original data set into a low-forgetting-probability data set and a high-forgetting-probability data set according to the forgetting probability of the original data in the original data set.

[0083] The intersection of the low-forgetting-probability data set and the high-forgetting-probability data set is empty, and the union of the low-forgetting-probability data set and the high-forgetting-probability data set is the original data set. For example, it is assumed that the original data set is denoted as The original data set can be divided into a low-forgetting-probability data set and a high-forgetting-probability data set , and Meanwhile,

[0084] In an embodiment, S310 comprises S3101-S3102:

[0085] S3101, sorting all original data in the original data set in ascending order according to the forgetting probability of the original data.

[0086] In an embodiment, assuming that the forgetting probabilities of all the original data in the original data set are known, all the original data are sorted according to the forgetting probabilities from small to large to obtain all the data according to the forgetting probabilities from small to large.

[0087] S3102, dividing the original data according to the proportion of data or the proportion of forgetting probability sums to obtain a low-forgetting-probability data set and a high-forgetting-probability data set.

[0088] The data proportion refers to the proportion of the data amount of the low-forgetting-probability data and the data amount of the high-forgetting-probability data in the original data set. For example, assuming that the data proportion is 4:1, the total data amount of the low-forgetting-probability data accounts for 80% of all the original data in the original data set, and the total data amount of the high-forgetting-probability data accounts for 20% of all the original data in the original data set. In an embodiment, the proportion of the forgetting probability sums refers to the proportion between the sum of the forgetting probabilities of all the data in the low-forgetting-probability data set and the sum of the forgetting probabilities of all the data in the high-forgetting-probability data set. For example, assuming that the sum of the forgetting probabilities of all the data in the low-forgetting-probability data set is 0.8, the sum of the forgetting probabilities of all the data in the high-forgetting-probability data set is 0.6, and the data amount contained in the high-forgetting-probability data set is half of the data amount contained in the low-forgetting-probability data set, the proportion between them is 8:3.

[0089] In an embodiment, the original data set is divided according to the proportion of data or the proportion of forgetting probability sums to obtain a low-forgetting-probability data set and a high-forgetting-probability data set.

[0090] S320, performing equal-piece division on the low-forgetting-probability data set and the high-forgetting-probability data set respectively to obtain corresponding low-forgetting-probability data pieces and high-forgetting-probability data pieces.

[0091] Each low-forgetting-probability data piece contains an equal amount of data, and each high-forgetting-probability data piece contains an equal amount of data. It can be understood that the data amount contained in all the low-forgetting-probability data pieces is the same, and the data amount contained in all the high-forgetting-probability data pieces is the same. In an embodiment, in order to limit the impact of data within a smaller range, the divided data pieces are further divided. For example, the sorted low-forgetting-probability data and high-forgetting-probability data are respectively divided into S pieces of data of the same size. For the low-forgetting-probability data set The data set will be divided into At the same time, for any piece of data And And Similarly, for the high-forgetting-probability data set The same division is also performed. By dividing the original data into multiple disjoint data segments, the training process can train multiple groups of data segments in parallel, which can reduce the training time.

[0092] S330, using the incremental training method, and based on the low forgetting probability data segment, the original training model is pre-trained to obtain a pre-processed isolated training model corresponding to each low forgetting probability data segment.

[0093] It should be noted that for the data set can be divided into R disjoint data segments by uniform division, and satisfy and The incremental training process on includes the following steps: first, import the target model, randomly initialize the parameters, and use the first data segment training model, get model and save the parameters of model ; then, based on , use training model get model and save the parameters of model ; in turn, in the Rth step, use training model get model and save model as the output model of this stage .

[0094] In this embodiment, for any one low forgetting probability data segment corresponding to the can be obtained by incremental training, and the parameters of model are saved and used for fine-tuning of the model in the next step.

[0095] S340, according to the size relationship between the amount of data contained in the low forgetting probability data set and the high forgetting probability data set, the parameters of part of the layers in the pre-processed isolated training model are fixed.

[0096] In the embodiment, according to the size relationship between the amount of data contained in the low forgetting probability data set and the high forgetting probability data set, the parameters of part of the layers in the pre-processed isolated training model are fixed, so as to reduce the retraining time of the model, that is, to improve the speed of retraining.

[0097] S350, using the incremental training method, and based on the high forgetting probability data segment, the pre-processed isolated training model is fine-tuned to obtain an intermediate isolated training model corresponding to each high forgetting probability data segment.

[0098] The specific implementation process of the incremental training manner can be referred to the description in S330, which is not described here again. In an embodiment, the model is adjusted on the data set using the incremental training manner, and the adjusted model is denoted as , and the parameters after each modification of the model are saved.

[0099] In an embodiment, the original data set is processed in layers, and then each layer is processed in slices, so that the total amount of data contained in each data slice is approximately the same, so that each data slice can obtain a learner (i.e., an intermediate isolated training model) with similar learning ability.

[0100] S360, the majority voting method or the simple averaging method is used to aggregate all the intermediate isolated training models to obtain the corresponding target training model.

[0101] In an embodiment, by fine-tuning the pre-processed isolated training model using the high-forgetting-probability data slice, R intermediate isolated training models can be obtained. Since the data sets for training R models contain the same amount of data, these models have similar performance, and combining multiple learners can reduce the insufficient generalization performance of a single learner due to misselection.

[0102] Among them, common combination strategies include averaging method, voting method and learning method. In machine forgetting learning, the combination strategy should have the following characteristics: the combination strategy should not involve training data, otherwise in some cases, the combination mechanism itself will have to be forgotten. Therefore, in this embodiment, the majority voting method or the simple averaging method can be used to combine the plurality of intermediate isolated training models to obtain the corresponding target training model.

[0103] S370, in response to the data processing request instruction, determining the data slice where the target data is located.

[0104] Among them, the target data is the data in the original data set that is about to perform a data deletion operation.

[0105] S380, performing a deletion operation on the data slice where the target data is located, and finding the last data slice before the data slice where the target data is located.

[0106] S390, retraining the pre-created target training model from the last data slice to obtain a new target training model.

[0107] ​The pre-created target training model is a model obtained by training an original training model according to low-forgetting-probability data segments and high-forgetting-probability data segments divided according to forgetting probabilities, wherein the number of low-forgetting-probability data segments and the number of high-forgetting-probability data segments are equal.

[0108] The technical solution of the embodiment, on the basis of the above embodiment, modifies low-forgetting-probability data with high-forgetting-probability data in each data segment, and does not cause a large decrease in prediction accuracy due to the mutual isolation of data segments. Moreover, by adjusting the learning rate of the two layers of data, forgetting of low-forgetting-probability data features by the model can be prevented, and the retraining speed can be accelerated by freezing some parameters in the first layer of the model, that is, the freezing technology is used to balance the retraining time and the prediction accuracy of the model, and in the case where the data forgetting probability is known, fine-grained and efficient deletion operations are performed on the data.

[0109] In an embodiment, the analysis process of the retraining time is as follows: the number of samples affected in the retraining process is directly proportional to the retraining time, and therefore, the influence of each parameter on the retraining time can be analyzed by the number of samples affected in the retraining.

[0110] Since in the training process, the original data set can be divided into a low-forgetting-probability data set and a high-forgetting-probability data set , the retraining time overhead also includes two parts: the overhead of forgetting data appearing in and the overhead of forgetting data appearing in .

[0111] Suppose there are K data processing requests (such as forgetting requests) in total, and the i-th forgetting request occurs in with a probability denoted as P L , and occurs in with a probability denoted as P H . According to the data set division rule (division according to forgetting probability), P L << P H .

[0112] If the i-th forgetting request occurs in , the expected number of data point samples that need to be retrained is:

[0113] If the i-th forgetting request occurs in , the upper limit of the expected number of data point samples that need to be retrained is:

[0114]

[0115] Combining the above two formulas, the upper bound of the expected number of samples in each forgetting request of the overall model is:

[0116] The upper bound of the expected number of samples in each forgetting request of the existing SISA model is Since Moreover, research shows Therefore, compared with the SISA model, the hierarchical machine learning forgettable algorithm has a greater improvement in forgetting data time.

[0117] In an implementation, Figure 4 is a research flowchart for fine-grained data destruction using a machine learning model provided by an embodiment of the present application. As Figure 4 shown, the hierarchical machine learning forgettable algorithm includes three parts: data grouping training and optimization, distribution differentiated data destruction improvement, and model isolation training and aggregation. Among them, the data grouping training and optimization includes: a sensitivity degree measurement model and a data access mode model, and two different models are used for data grouping; the model isolation training and aggregation includes: a model training process and a model aggregation process (including: a screen projection mechanism design and a model weight setting).

[0118] Figure 5 is a flowchart of a fine-grained data processing algorithm provided by an embodiment of the present application. As Figure 5 shown, assuming that the original data set is The original data set can be divided into a low-forgetting probability data set and a high-forgetting probability data set For the low-forgetting probability data set The data set will be divided into At the same time, for any shard and And Similarly, the same division is also performed on the high-forgetting probability data set For any low-forgetting probability data segment The corresponding can be obtained through incremental training, and the parameters of the model are saved; then, for any low-forgetting probability data segment The corresponding can be obtained through incremental training, and the parameters of the model are saved, obtaining S intermediate isolation training models, and the S intermediate isolation training models are aggregated to obtain and output the corresponding target training model.

[0119] In an embodiment, assuming that the information of the original data set used is as shown in Table 1, in the Purchase data set, 600 items with the largest purchase amount are selected as the category attribute.

[0120] Table 1 Data set information

[0121]

[0122] In the experiment, the same model structure as the SISA model is selected, and specific model information is shown in Table 2, including various deep neural networks, and the deep neural networks have different numbers of hidden layers and different layer sizes.

[0123] Table 2 DNN model structure used

[0124]

[0125] For the three different data sets, two data forgetting probability distribution situations are assumed respectively, one is an exponential distribution, and the other is a Pareto distribution. The implementation effect of the method is tested through experiments.

[0126] Figure 6 is the prediction accuracy of the two kinds of models in different slices provided by an embodiment of the present application using the Purchase data set; Figure 7 is the prediction accuracy of the two kinds of models in different slices provided by an embodiment of the present application using the SVHN data set. As shown in Figure 6 and 7 , respectively, the hierarchical machine forgettable learning method (i.e., the HMU training method) and the SISA training method proposed by an embodiment of the present application are used to compare and analyze the prediction accuracy of the two training methods in different data segments using the Purchase data and the SVHN data.

[0127] As shown in Figure 6 and 7As shown in the comparison results of the prediction accuracy, when the number of data segments is 1, that is, no data segmentation is used, the prediction accuracy of the SISA training method is higher, because in the hierarchical machine forget learning method (that is, the HMU training method), even if the number of data segments is 1, data is layered according to the forgetting probability, which reduces the fitting ability of the model; when the number of data segments is greater than 1, in the above two groups of experiments, the prediction accuracy of the SISA training method decreases obviously, because the SISA training method uses a small amount of data with high forgetting probability to train part of the isolated model, and obtains part of the weak learner, which affects the prediction accuracy of the overall model through model aggregation. This decrease will become more and more obvious with the increase of the number of segments. In the model of the embodiment, because the data sets of each isolated model are of the same size and have similar learning ability, the accuracy decrease caused by unbalanced learning ability is avoided.

[0128] Figure 8 is a schematic diagram of the number of data points affected in the data deletion stage provided by the embodiment of the application. As Figure 8 shown, because the experiment simulates the case that the amount of data with high forgetting probability is small in reality, under the condition that the number of data points affected in the data deletion stage is less in the hierarchical machine forget learning method HMU training method proposed in the embodiment of the application than in the SISA training method.

[0129] Embodiment Four

[0130] Figure 9 is a structural schematic diagram of a data processing device provided by the fourth embodiment of the application. As Figure 9 shown, the device includes a determination module 910, a processing module 920, and a retraining module 930.

[0131] The determination module 910 is configured to determine a data segment where target data is located in response to a data processing request instruction, wherein the target data is data in an original data set that is about to perform a data deletion operation.

[0132] The processing module 920 is configured to perform a deletion operation on the data segment where the target data is located, and find the last data segment before the data segment where the target data is located.

[0133] The retraining module 930 is configured to retrain a pre-created target training model from the last data segment to obtain a new target training model, wherein the pre-created target training model is a model obtained by training an original training model according to low-forgetting-probability data segments and high-forgetting-probability data segments divided according to forgetting probabilities, and the number of the low-forgetting-probability data segments and the high-forgetting-probability data segments is equal.

[0134] In an embodiment, before determining the data segment where the target data is located in response to the data processing request instruction, the data processing apparatus further comprises:

[0135] a dividing module, configured to divide the original data set to obtain the low-forgetting-probability data segment and the high-forgetting-probability data segment in response to the data processing request instruction;

[0136] a training module, configured to train the original training model by using the low-forgetting-probability data segment and the high-forgetting-probability data segment in sequence to obtain the intermediate isolated training model; wherein the total number of the intermediate isolated training model is equal to the number of the low-forgetting-probability data segment and the high-forgetting-probability data segment, and the intermediate isolated training model, the low-forgetting-probability data segment and the high-forgetting-probability data segment are one-to-one corresponding;

[0137] an aggregation processing module, configured to aggregate all the intermediate isolated training models by using a preset aggregation algorithm to obtain the target training model, so as to predict the target data by using the target training model.

[0138] In an embodiment, the dividing module comprises:

[0139] a first dividing unit, configured to divide the original data set into the low-forgetting-probability data set and the high-forgetting-probability data set according to the forgetting probability of the original data in the original data set;

[0140] a second dividing unit, configured to divide the low-forgetting-probability data set and the high-forgetting-probability data set into the low-forgetting-probability data segment and the high-forgetting-probability data segment respectively; wherein the data amount contained in each low-forgetting-probability data segment is equal, and the data amount contained in each high-forgetting-probability data segment is equal.

[0141] In an embodiment, the retraining module comprises:

[0142] a pre-training unit, configured to pre-train the original training model based on the low-forgetting-probability data segment by using an incremental training manner to obtain the pre-processed isolated training model corresponding to each low-forgetting-probability data segment;

[0143] a fine-tuning unit, configured to fine-tune the pre-processed isolated training model based on the high-forgetting-probability data segment by using an incremental training manner to obtain the intermediate isolated training model corresponding to each high-forgetting-probability data segment.

[0144] In an embodiment, the aggregation processing module is specifically configured to aggregate all the intermediate isolated training models by using a majority voting method or a simple average method to obtain the target training model.

[0145] In an embodiment, the first dividing unit comprises:

[0146] The sorting subunit is configured to sort all original data in the original data set in ascending order according to the forgetting probability of the original data.

[0147] The dividing subunit is configured to divide the original data according to the proportion of the data or the proportion of the forgetting probability sum, to obtain a low-forgetting-probability data set and a high-forgetting-probability data set.

[0148] In an embodiment, before the intermediate isolated training model corresponding to each high-forgetting-probability data segment is obtained by fine-tuning the pre-processed isolated training model based on the high-forgetting-probability data segment in the incremental training manner, the retraining module further comprises:

[0149] The fixing unit is configured to fix the parameters of part of layers in the pre-processed isolated training model according to the size relationship between the amount of data contained in the low-forgetting-probability data set and the high-forgetting-probability data set.

[0150] The data processing apparatus provided by the embodiments of the present application can execute the data processing method provided by any of the embodiments of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0151] Embodiment five

[0152] Figure Five A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0153] As Figure 10As shown, the electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0154] Various components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, a speaker, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0155] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as a data processing method: in response to a data processing request instruction, determining a data segment where target data is located, wherein the target data is data in an original data set that is about to perform a data deletion operation; performing a deletion operation on the data segment where the target data is located, and finding the last data segment before the data segment where the target data is located; and retraining a pre-created target training model starting from the last data segment to obtain a new target training model, wherein the target training model is a model obtained by training an original training model according to low-forgetting-probability data segments and high-forgetting-probability data segments divided according to a forgetting probability, wherein the number of low-forgetting-probability data segments and high-forgetting-probability data segments is equal.

[0156] In some embodiments, the data processing method can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, parts or all of the computer program can be loaded and / or installed onto electronic device 10 via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of the data processing method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the data processing method by way of other means, e.g., with the aid of firmware.

[0157] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0158] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor of the machine, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a standalone software package, or entirely on a remote machine or server.

[0159] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0160] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0161] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0162] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0163] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, each step described in the present application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.

[0164] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that, include: The original dataset is divided into corresponding low-forgetting-probability data segments and high-forgetting-probability data segments; wherein, the original dataset includes: Purchase dataset, SVHN dataset and CIFAR-100 dataset; the high-forgetting-probability data segments are obtained by equally dividing the high-forgetting-probability dataset, and the low-forgetting-probability data segments are obtained by equally dividing the low-forgetting-probability dataset. The original training model is trained sequentially using the low-forgetting-probability data segments and the high-forgetting-probability data segments to obtain the corresponding intermediate isolation training model; wherein, the total number of intermediate isolation training models is equal to the number of low-forgetting-probability data segments and the high-forgetting-probability data segments, and there is a one-to-one correspondence between the intermediate isolation training models, the low-forgetting-probability data segments, and the high-forgetting-probability data segments; A preset aggregation algorithm is used to aggregate all the intermediate isolated training models to obtain the corresponding target training model. In response to a data processing request instruction, the data segment containing the target data is determined, wherein the target data is the data segment in the original dataset from which the data deletion operation is about to be performed, and the data segment refers to a segment that has been pre-divided according to the data forgetting probability; Perform a deletion operation on the data segment containing the target data, and find the last data segment preceding the data segment containing the target data; Starting from the last data segment, the pre-created target training model is retrained to obtain a new target training model. The pre-created target training model is a model obtained by training the original training model with low-forgetting-probability data segments and high-forgetting-probability data segments obtained according to the forgetting probability. The number of low-forgetting-probability data segments and high-forgetting-probability data segments is equal. The step of sequentially training the original training model using the low-forgetting-probability data segment and the high-forgetting-probability data segment to obtain the corresponding intermediate isolation training model includes: An incremental training method is adopted, and the original training model is pre-trained based on the low-forgetting-probability data segments to obtain a pre-processed isolated training model corresponding to each low-forgetting-probability data segment. Based on the relationship between the amount of data contained in the low forgetting probability dataset and the high forgetting probability dataset, the parameters of some layers in the preprocessed isolated training model are fixed. An incremental training method is adopted, and the preprocessed isolated training model is fine-tuned based on the high-forgetting-probability data segments to obtain an intermediate isolated training model corresponding to each high-forgetting-probability data segment.

2. The method according to claim 1, characterized in that, The process of dividing the original dataset to obtain corresponding low-forgetting-probability data segments and high-forgetting-probability data segments includes: The original dataset is divided into a low-forgetting-probability dataset and a high-forgetting-probability dataset based on the forgetting probability of the original data in the original dataset. The low-forgetting-probability dataset and the high-forgetting-probability dataset are divided into equal segments to obtain corresponding low-forgetting-probability data segments and high-forgetting-probability data segments; wherein each low-forgetting-probability data segment contains the same amount of data, and each high-forgetting-probability data segment contains the same amount of data.

3. The method according to claim 1, characterized in that, The step of aggregating all the intermediate isolated training models using a preset aggregation algorithm to obtain the corresponding target training model includes: The majority voting method or simple averaging method is used to aggregate all intermediate isolated training models to obtain the corresponding target training model.

4. The method according to claim 2, characterized in that, The step of dividing the original dataset into a low-forgetting-probability dataset and a high-forgetting-probability dataset based on the forgetting probability of the original data in the original dataset includes: Sort all the original data in the original dataset in ascending order according to the forgetting probability of the original data. The original data is divided into low-forgetting-probability datasets and high-forgetting-probability datasets according to the proportion of data or the proportion of forgetting probabilities.

5. A data processing apparatus, characterized in that, include: The partitioning module is used to partition the original dataset in response to a data processing request instruction, thereby obtaining corresponding low-forgetting-probability data segments and high-forgetting-probability data segments. The original dataset includes the Purchase dataset, the SVHN dataset, and the CIFAR-100 dataset. The high-forgetting-probability data segments are obtained by partitioning the high-forgetting-probability dataset into equal segments, and the low-forgetting-probability data segments are obtained by partitioning the low-forgetting-probability dataset into equal segments. The training module is used to train the original training model sequentially using low-forgetting-probability data segments and high-forgetting-probability data segments to obtain the corresponding intermediate isolation training models. The total number of intermediate isolation training models is equal to the number of low-forgetting-probability data segments and high-forgetting-probability data segments, and there is a one-to-one correspondence between the intermediate isolation training models, low-forgetting-probability data segments, and high-forgetting-probability data segments. The aggregation processing module is used to aggregate all intermediate isolated training models using a preset aggregation algorithm to obtain the corresponding target training model. The determination module is used to determine the data segment containing the target data in response to a data processing request instruction. The target data is the data in the original dataset that is about to undergo a data deletion operation. The data segment refers to a segment that is pre-divided according to the data forgetting probability. The processing module is used to perform a deletion operation on the data segment containing the target data, and to find the last data segment preceding the data segment containing the target data; A retraining module is used to retrain the pre-created target training model starting from the last data segment to obtain a new target training model. The pre-created target training model is a model obtained by training the original training model with low-forgetting-probability data segments and high-forgetting-probability data segments obtained according to the forgetting probability. The number of low-forgetting-probability data segments and high-forgetting-probability data segments is equal. The training module includes: The pre-training unit is used to pre-train the original training model using incremental training and based on low-forgetting-probability data segments to obtain a pre-processed isolated training model corresponding to each low-forgetting-probability data segment. Fixed units are used to fix the parameters of some layers in the preprocessed isolated training model based on the relationship between the amount of data contained in the low forgetting probability dataset and the high forgetting probability dataset. The fine-tuning unit is used to fine-tune the preprocessed isolated training model using incremental training and based on high-forgetting-probability data segments to obtain an intermediate isolated training model corresponding to each high-forgetting-probability data segment.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data processing method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the data processing method according to any one of claims 1-4.

Images