Method, apparatus, storage medium and processor for determining model training data

Abstract

The embodiment of the present application provides a method for determining model training data. The method comprises the following steps: inputting sample data without desensitization processing into a neural network model to train the neural network model to obtain a first model; performing desensitization processing on the sample data by using multiple desensitization methods to obtain desensitization data corresponding to each desensitization method; inputting the desensitization data corresponding to each desensitization method into the neural network model to train the neural network model to obtain multiple second models; determining model parameters of the first model and each second model; comparing the model parameters of the first model with the model parameters of each second model to determine a model difference value between the first model and each second model; determining a second model with the smallest model difference value as a target model; determining a desensitization method corresponding to the target model as a target desensitization method to perform desensitization processing on the sample data by using the target desensitization method, and reducing model training difference.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a method and device for determining model training data, a storage medium and a processor. BACKGROUND

[0002] In order to reliably protect sensitive data, desensitization of sensitive data is an effective means to prevent sensitive data leakage. In the prior art, common data desensitization methods include substitution, shuffle, numerical transformation and encryption, etc. However, desensitizing sensitive data by different desensitization methods and training a model using desensitized data will bring great differences to the result of model training. For example, desensitizing sensitive data by substitution may cause information loss in desensitized data, and training a model using desensitized data with information loss will affect the training effect of model training to some extent. SUMMARY

[0003] The embodiments of the present application provide a method and device for determining model training data, a storage medium and a processor.

[0004] To achieve the above object, the first aspect of the present application provides a method for determining model training data, comprising:

[0005] inputting sample data without desensitization processing into a neural network model to train the neural network model and obtain a first model;

[0006] desensitizing the sample data by multiple desensitization methods to obtain desensitized data corresponding to each desensitization method;

[0007] inputting the desensitized data corresponding to each desensitization method into the neural network model to train the neural network model and obtain multiple second models;

[0008] determining model parameters of the first model and each second model;

[0009] comparing the model parameters of the first model with the model parameters of each second model to determine a model difference value between the first model and each second model;

[0010] determining the second model with the smallest model difference value as a target model;

[0011] determining the desensitization method corresponding to the target model as a target desensitization method to desensitize the sample data using the target desensitization method to obtain data for training the neural network model.

[0012] In the embodiments of the present application, the model difference value includes an AUC value between the models, and the model difference value Y of each second model and the first model is determined according to formula (1):

[0013]

[0014] wherein Y i represents the model difference value between the second model trained by the sample data desensitized by the i th desensitization method and the first model, V s is the AUC value of the first model, V ei is the AUC value of the second model trained by the sample data desensitized by the i th desensitization method, is the overfitting degree value between the second model trained by the sample data desensitized by the i th desensitization method and the first model.

[0015] In the embodiments of the present application, the sample data without desensitization processing is input into the neural network model to train the neural network model to obtain the first model, including: submitting a machine learning pipeline to a first running environment, and inputting the sample data without desensitization processing into the neural network model of the machine learning pipeline in the first running environment to train the neural network model and obtain the first model; and inputting the desensitized data corresponding to each desensitization method into the neural network model to train the neural network model to obtain a plurality of second models, including: submitting a machine learning pipeline to a second running environment, and inputting the desensitized data corresponding to each desensitization method into the neural network model of the machine learning pipeline in the second running environment to train the neural network model and obtain a plurality of second models.

[0016] In the embodiments of the present application, the method further includes: obtaining target training parameters of a target model, the target training parameters including environment running parameters of a second running environment where the target model is located, model parameters of the target model, and a desensitization method used for desensitized data for training the target model; and determining the target training parameters as training parameters for subsequent model training.

[0017] In the embodiments of the present application, the first running environment is a trusted environment, the data in the trusted environment includes sensitive data, and the second running environment is a debugging environment, and the data in the debugging environment are all desensitized data.

[0018] In the embodiments of the present application, the model parameters of the neural network model trained by the desensitized sample data are consistent with the model parameters of the neural network model trained by the sample data without desensitization.

[0019] In the embodiments of the present application, the model difference value includes a lift and / or a ks statistic.

[0020] The second aspect of the present application provides a processor configured to perform the method for determining model training data described above.

[0021] The third aspect of the present application provides an apparatus for determining model training data, the apparatus comprising:

[0022] a first training module configured to input the sample data without desensitization processing into the neural network model to train the neural network model to obtain a first model;

[0023] a data desensitization module configured to desensitize the sample data by a plurality of desensitization methods to obtain desensitized data corresponding to each desensitization method;

[0024] a second training module configured to input the desensitized data corresponding to each desensitization method into the neural network model to train the neural network model to obtain a plurality of second models;

[0025] a model comparison module configured to determine model parameters of the first model and each second model, compare the model parameters of the first model with the model parameters of each second model respectively to determine a model difference value between the first model and each second model;

[0026] a model selection module configured to determine a second model with the smallest model difference value as a target model, determine a desensitization method corresponding to the target model as a target desensitization method, and desensitize the sample data using the target desensitization method to obtain data for training the neural network model.

[0027] In the embodiments of the present application, the model difference value comprises an AUC value between the models, and the model comparison module is further configured to determine the model difference value Y between each second model and the first model according to formula (1):

[0028]

[0029] wherein Y i represents the model difference value between the second model trained by the sample data desensitized using the i-th desensitization method and the first model, V s is the AUC value of the first model, V ei is the AUC value of the second model trained by the sample data desensitized using the i-th desensitization method, is the overfitting degree value between the second model trained by the sample data desensitized using the i-th desensitization method and the first model.

[0030] In the embodiments of the present application, the first training module is further configured to submit the machine learning pipeline to the first running environment, input the sample data without desensitization processing into the neural network model of the machine learning pipeline under the first running environment to train the neural network model, and obtain the first model; and the second training module is further configured to submit the machine learning pipeline to the second running environment, input the desensitized data corresponding to each desensitization method into the neural network model of the machine learning pipeline under the second running environment to train the neural network model, and obtain the plurality of second models.

[0031] In the embodiments of the present application, the model selection module is further configured to obtain a target training parameter of a target model, the target training parameter including an environment running parameter of a second running environment where the target model is located, a model parameter of the target model, and a desensitization method used for desensitized data for training the target model; and determine the target training parameter as a training parameter for subsequent model training.

[0032] In the embodiments of the present application, the first running environment is a trusted environment, and the data in the trusted environment includes sensitive data; and the second running environment is a debugging environment, and the data in the debugging environment are all desensitized data.

[0033] In the embodiments of the present application, the model parameters of the neural network model trained by the desensitized sample data are consistent with the model parameters of the neural network model trained by the sample data without desensitization.

[0034] In the embodiments of the present application, the model difference value includes a lift and / or a ks statistic between the models.

[0035] The fourth aspect of the present application provides a machine readable storage medium, the machine readable storage medium stores instructions, the instructions, when executed by a processor, cause the processor to be configured to perform the above-mentioned method for determining model training data.

[0036] The fifth aspect of the present application provides a computer program product, including a computer program, the computer program, when executed by a processor, implements the above-mentioned method for determining model training data.

[0037] Through the above technical solution, the target desensitization method can be determined, and the data without desensitization processing is desensitized by the desensitization method to obtain the model training data, so as to reduce the difference of model training effect and further improve the training effect of the model.

[0038] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation part. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which are included to provide a further understanding of the embodiments of the application and constitute a part of this specification, illustrate embodiments of the application and together with the specific description given above and the specific description given below, serve to explain embodiments of the application. In the drawings:

[0040] Figure 1 Fig. 1 schematically shows a flow diagram of a method for determining model training data according to an embodiment of the application;

[0041] Figure 2 Fig. 2 schematically shows another flow diagram of a method for determining model training data according to an embodiment of the application;

[0042] Figure 3 Fig. 3 schematically shows an application environment diagram of a method for determining model training data according to an embodiment of the application;

[0043] Figure 4 Fig. 4 schematically shows an application environment diagram of a method for determining model training data according to another embodiment of the application;

[0044] Figure 5 Fig. 5 schematically shows an application environment diagram of a method for determining model training data according to yet another embodiment of the application;

[0045] Figure 6 Fig. 6 schematically shows a structural block diagram of an apparatus for determining model training data according to an embodiment of the application;

[0046] Figure 7 Fig. 7 schematically shows an internal structure diagram of a computer device according to an embodiment of the application. DETAILED DESCRIPTION

[0047] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It should be understood that the specific embodiments described herein are merely used to explain and illustrate the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0048] Figure 1 Fig. 1 schematically shows a flow diagram of a method for determining model training data according to an embodiment of the application. As shown in Figure 1 In an embodiment of the present application, a method for determining model training data is provided, comprising the following steps:

[0049] In step 101, the sample data without desensitization processing is input into the neural network model to train the neural network model, and a first model is obtained.

[0050] In step 102, the sample data is desensitized by multiple desensitization methods to obtain desensitized data corresponding to each desensitization method.

[0051] In step 103, the desensitized data corresponding to each desensitization method is input into the neural network model to train the neural network model, and multiple second models are obtained.

[0052] In step 104, the model parameters of the first model and each second model are determined.

[0053] In step 105, the model parameters of the first model and the model parameters of each second model are compared respectively to determine the model difference value between the first model and each second model.

[0054] In step 106, the second model with the smallest model difference value is determined as a target model.

[0055] In step 107, the desensitization method corresponding to the target model is determined as a target desensitization method to desensitize the sample data using the target desensitization method to obtain data for training the neural network model.

[0056] When training the model, the training data can include two kinds. One is the data after desensitization processing by the desensitization method, and the other can be data without desensitization processing, which can include sensitive data. In this way, the model can be trained by two kinds of training data respectively. The processor can input the sample data without desensitization processing into the neural network model to train the neural network model, and obtain a first model. The sample data without desensitization processing can include sensitive data. The sensitive data can refer to data that may cause harm to society or individuals after leakage. Specifically, the sensitive data can include personal privacy data such as name, ID number, address, phone number, bank card account number, etc., and can refer to data that enterprises or social institutions are not suitable to publish, such as enterprise operation data, etc. The first model can refer to a model obtained by training the neural network model with sample data without desensitization.

[0057] The processor can desensitize the sample data not subjected to desensitization processing through various desensitization methods to obtain desensitized data corresponding to each desensitization method. The desensitization method can refer to substitution, shuffling, numerical transformation, encryption, etc. The processor can input the desensitized data corresponding to each desensitization method into the neural network model to train the neural network model to obtain a plurality of second models. That is, one second model can be obtained by training the neural network model through the desensitized data corresponding to each desensitization method. The second model can refer to a model obtained by training the neural network model through the desensitized data.

[0058] The processor can determine the model parameters of the first model and each second model. The model parameters can refer to weights in the neural network, etc. The processor can compare the model parameters of the first model with the model parameters of each second model to determine the model difference value between the first model and each second model. To ensure the comparability of subsequent model parameters, the model parameters of the first model and each second model can be consistent. The model difference can refer to the AUC value, lift, and ks statistics between models, etc. The processor can determine the second model with the smallest model difference value as the target model. The processor can determine the desensitization method corresponding to the target model as the target desensitization method to desensitize the sample data using the target desensitization method to obtain data for training the neural network model.

[0059] Through the above technical solution, a more appropriate desensitization method can be obtained, and the data not subjected to desensitization is desensitized through this desensitization method to obtain model training data, which can reduce the difference in model training effect and further improve the model training effect.

[0060] In one embodiment, the model difference value includes the AUC value between the models, and the model difference value Y between each second model and the first model is determined according to formula (1):

[0061]

[0062] Y = - (V i V s is the AUC value of the first model, V ei is the AUC value of the second model trained by the sample data desensitized using the i-th desensitization method, is the overfitting degree value between the second model trained by the sample data desensitized using the i-th desensitization method and the first model.

[0063] The model difference value of the first model and each second model can include an AUC value between the models. The AUC value can range from 0≤AUC≤1. The processor can determine the model difference value Y of each second model and the first model according to formula (1). Wherein, formula (1) can be expressed as Wherein, Y i represents the model difference value between the second model trained by the sample data desensitized by the i-th desensitization method and the first model, V s is the AUC value of the first model, V ei is the AUC value of the second model trained by the sample data desensitized by the i-th desensitization method, is the overfitting degree value between the second model trained by the sample data desensitized by the i-th desensitization method and the first model.

[0064] In one embodiment, the sample data without desensitization processing is input into the neural network model to train the neural network model to obtain the first model, including submitting a machine learning pipeline to a first running environment, and inputting the sample data without desensitization processing into the neural network model of the machine learning pipeline in the first running environment to train the neural network model to obtain the first model.

[0065] The processor can submit a machine learning pipeline to a first running environment, and in the first running environment, the sample data without desensitization processing can be input into the neural network model of the machine learning pipeline to train the neural network model to obtain the first model. Wherein, the machine learning pipeline can refer to a work flow composed of various steps and executable. For example, the machine pipeline can include steps of data extraction, data verification, data preparation, model training, model evaluation and model verification. The first running environment can refer to a trusted environment, specifically, it can refer to a running environment that can directly access data without desensitization processing, and the feedback data information to the developer is controllable. Taking the identity card number as an example, the modeling personnel can see the desensitized data in the front end, while in the first running environment, the machine learning pipeline can be allowed to read the data without desensitization processing. Thus, in the first running environment, the modeling personnel can extract information helpful to model training such as gender and birthplace according to the coding rules of the identity card number in the machine learning pipeline. The first model can refer to the model obtained by training the neural network model by the sample data without desensitization.

[0066] In an embodiment, the desensitization data corresponding to each desensitization method is respectively input into the neural network model to train the neural network model, and the plurality of second models are obtained.

[0067] The processor can submit the machine learning pipeline to the second running environment, and in the second running environment, the desensitization data corresponding to each desensitization method can be respectively input into the neural network model of the machine learning pipeline to train the neural network model, and the plurality of second models are obtained. The machine learning pipeline can refer to a work flow composed of various steps and executable. For example, the machine pipeline can include steps of data extraction, data verification, data preparation, model training, model evaluation, and model verification. The second running environment can refer to a debugging environment, specifically, a machine learning pipeline debugging environment, and can refer to a running environment that can access the desensitization data. The second model can refer to a model obtained by training the neural network model with the desensitization data.

[0068] In an embodiment, the method further includes: obtaining target training parameters of the target model, the training parameters including environment running parameters of the second running environment where the target model is located, model parameters of the target model, and a desensitization method used for desensitization data for training the target model; and determining the target training parameters as training parameters for subsequent model training.

[0069] In the case where the processor determines the second model with the smallest model difference value as the target model, the processor can obtain target training parameters of the target model. The target training parameters can include environment running parameters of the second running environment where the target model is located, model parameters of the target model, and a desensitization method used for desensitization data for training the target model. Specifically, the environment running parameters of the second running environment can include machine learning pipeline code and the like in the second running environment. The model parameters of the target model can refer to weights of the neural network and the like. The desensitization method used for the desensitization data for training the target model can include replacement, shuffling, numerical transformation, encryption, and the like. In the case where the target training parameters of the target model are obtained, the processor can determine the target model parameters as training parameters for subsequent model training.

[0070] In an embodiment, the first running environment is a trusted environment, and the data in the trusted environment includes sensitive data. The second running environment is a debugging environment, and the data in the debugging environment are all desensitization data.

[0071] The first running environment can refer to a trusted environment, and data in the trusted environment can include sensitive data, i.e., data that has not been desensitized. The second running environment can refer to a debugging environment, specifically, a machine learning pipeline debugging environment. Data in the debugging environment can be desensitized data.

[0072] In one embodiment, the model parameters of the neural network model trained by the desensitized sample data are consistent with the model parameters of the neural network model trained by the non-desensitized sample data.

[0073] The neural network model trained by the desensitized sample data can obtain a plurality of second models. The neural network model trained by the non-desensitized sample data can obtain a first model. The model parameters of the neural network model trained by the desensitized sample data can be consistent with the model parameters of the neural network model trained by the non-desensitized sample data. That is, the model parameters of the second model can be consistent with the model parameters of the first model. The model parameters can refer to weights of the neural network, etc.

[0074] In one embodiment, the model difference value includes a lift and / or a ks statistic between the models.

[0075] The lift and the ks statistic can refer to evaluation indicators of the models. The lift can evaluate the prediction ability of the model. The ks statistic can evaluate the accuracy of the model prediction. The processor can determine the model difference value of each second model and the first model according to the lift and / or the ks statistic between the models.

[0076] In one embodiment, as shown in Figure 2 Another flowchart of a method for determining model training data is provided. As shown in Figure 2 First, the processor can analyze and explore the data. After analyzing and exploring the data, the processor can develop a machine learning pipeline and test the developed machine learning pipeline. In the case that the machine learning pipeline test fails, the data is analyzed and explored again. In the case that the machine learning pipeline test passes, the tested machine learning pipeline can be run to train the model of the machine learning pipeline. After training the model of the machine learning pipeline, the trained model can be compared and analyzed to determine the target model. Then, it is determined whether the target model meets the business requirements. In the case that the target model does not meet the business requirements, the data is analyzed and explored again.

[0077] In one embodiment, as shown in Figure 3 An application environment diagram of a method for determining model training data is provided.

[0078] The processor can analyze and explore the data distribution statistics and the sampling detailed data. The data distribution statistics can be obtained from the model development data area. The data distribution statistics can be information that does not involve sensitive data. The sampling detailed data can be obtained from the detailed data of the model development data area. The detailed data can include the sampling detailed data and the full-amount detailed data. The sampling detailed data and the full-amount detailed data can be information that involves sensitive data. When reading the sampling detailed data, the processor can first perform simple replacement on the sampling detailed data through the model development data access controller and the pre-set sensitive data identification and replacement strategy, and then return the replaced sampling detailed data.

[0079] In one embodiment, as shown in FIG. 1, another application environment diagram for determining a method of model training data is provided. Figure 4

[0080] When developing a machine learning pipeline, the processor can submit the machine learning pipeline to a machine learning pipeline trial running environment. The machine learning pipeline can refer to an executable workflow composed of various steps. For example, the machine learning pipeline can include data extraction, data verification, data preparation, model training, model evaluation, and model validation steps. The machine learning pipeline trial running environment can include a machine learning pipeline debugging environment and a machine learning pipeline trusted environment.

[0081] If the processor submits the machine learning pipeline to the machine learning pipeline debugging environment, the processor can read the sampling desensitized data from the model development data area and input the sampling desensitized data into the machine learning pipeline to train the model in the machine learning pipeline by using the sampling desensitized data. The sampling desensitized data can be determined by obtaining the sampling detailed data from the detailed data in the model development data area and performing desensitization processing on the sampling detailed data using different desensitization methods. After training the model, the processor can evaluate and validate the training effect of the model in the machine learning pipeline.

[0082] If the processor submits the machine learning pipeline to the machine learning pipeline trusted environment, the processor can read the sampling sensitive data from the detailed data of the model development data area and input the sampling sensitive data into the machine learning pipeline to train the model in the machine learning pipeline by using the sampling sensitive data. After training the model, the processor can evaluate and validate the training effect of the model in the machine learning pipeline.

[0083] ​The processor can compare the models trained in the machine learning pipeline debugging environment and the machine learning pipeline trusted environment to determine model differences between the models. Specifically, the differences between the models can be determined by model indicators. Then, the processor can determine the model with the smallest difference as the target model, and determine the desensitization method corresponding to the target model. The processor can return the running result information of the model with the smallest difference for the model development personnel to view the machine learning pipeline. The running result information can include model indicator information, machine learning pipeline code, model training parameters, and the desensitization method corresponding to the model with the smallest model indicator difference.

[0084] In one embodiment, as Figure 5 indicated, another application environment diagram for the method for determining model training data is also provided.

[0085] The processor can submit the machine learning pipeline to a machine learning pipeline running environment. The machine learning pipeline can refer to an executable workflow composed of various steps. For example, the machine learning pipeline can include steps such as data extraction, data verification, data preparation, model training, model evaluation, and model verification. The processor can read the desensitized full amount of detailed data from the detailed data in the model development data area. The desensitized full amount of detailed data can be desensitized using the desensitization method corresponding to the target model. The processor can input the desensitized full amount of detailed data into the machine learning pipeline to train the model in the machine learning pipeline using the desensitized full amount of detailed data. After training the model, the processor can evaluate and verify the training effect of the model in the machine learning pipeline. After the model is verified, the processor can return the running result information for the model development personnel to view the machine learning pipeline running result.

[0086] Through the above technical solutions, a more appropriate desensitization method can be obtained, and the data that has not been desensitized is desensitized using this desensitization method to obtain model training data, which can reduce the difference in model training effect and further improve the model training effect. At the same time, the data that has not been desensitized is desensitized using this desensitization method, which can improve the security of data desensitization and greatly reduce the cost of sensitive data security management and control.

[0087] Figure 1 、 Figure 2 For one embodiment, a flowchart of the method for determining model training data is provided. It should be understood that although Figure 1 、 Figure 2The steps in the flowchart are shown in sequence according to the arrows, but the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the steps are not strictly limited in sequence, and the steps can be executed in other sequences. Moreover, Figure 1 , Figure 2 At least part of the steps in the flowchart can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of the sub-steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or sub-steps or stages of other steps.

[0088] The embodiment of the present application provides a processor, which is used for running a program, wherein the program is used for executing the method for determining model training data.

[0089] In one embodiment, as shown in Figure 6 , a device for determining model training data is provided, comprising a first training module, a data desensitization module, a second training module, a model comparison module, and a model selection module, wherein:

[0090] The first training module 601 is configured to input sample data without desensitization processing into a neural network model to train the neural network model, and obtain a first model.

[0091] The data desensitization module 602 is configured to perform desensitization processing on the sample data by using multiple desensitization methods, to obtain desensitization data corresponding to each desensitization method.

[0092] The second training module 603 is configured to input the desensitization data corresponding to each desensitization method into the neural network model respectively to train the neural network model, and obtain multiple second models.

[0093] The model comparison module 604 is configured to determine model parameters of the first model and each second model, compare the model parameters of the first model with the model parameters of each second model respectively, and determine a model difference value between the first model and each second model.

[0094] The model selection module 605 is configured to determine a second model with the smallest model difference value as a target model, determine a desensitization method corresponding to the target model as a target desensitization method, and use the target desensitization method to perform desensitization processing on the sample data to obtain data for training the neural network model.

[0095] In an embodiment, the model difference value comprises an AUC value between the models, and the model comparison module is further configured to determine the model difference value Y of each second model with respect to the first model according to formula (1):

[0096]

[0097] wherein Y i represents the model difference value between the second model trained by the sample data desensitized by the i-th desensitization method and the first model, V s is the AUC value of the first model, V ei is the AUC value of the second model trained by the sample data desensitized by the i-th desensitization method, is the overfitting degree value between the second model trained by the sample data desensitized by the i-th desensitization method and the first model.

[0098] In an embodiment, the first training module is further configured to submit the machine learning pipeline to the first running environment, and input the sample data without desensitization into the neural network model of the machine learning pipeline in the first running environment to train the neural network model and obtain the first model; and the second training module is further configured to submit the machine learning pipeline to the second running environment, and input the desensitized data corresponding to each desensitization method into the neural network model of the machine learning pipeline in the second running environment to train the neural network model and obtain the plurality of second models.

[0099] In an embodiment, the model selection module is further configured to obtain a target training parameter of a target model, the target training parameter comprising an environment running parameter of a second running environment where the target model is located, a model parameter of the target model, and a desensitization method used for desensitized data for training the target model; and determine the target training parameter as a training parameter for subsequent model training.

[0100] In an embodiment, the first running environment is a trusted environment, and the data in the trusted environment comprises sensitive data; and the second running environment is a debugging environment, and the data in the debugging environment are all desensitized data.

[0101] In an embodiment, the model parameter of the neural network model trained by the desensitized sample data is consistent with the model parameter of the neural network model trained by the sample data without desensitization.

[0102] In an embodiment, the model difference value comprises a lift and / or a ks statistic between the models.

[0103] Embodiments of the present application provide a storage medium having a program stored thereon, the program being executed by a processor to implement the above method for determining model training data.

[0104] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram thereof can be as shown in FIG. 1. Figure 7 The computer device includes a processor A01, a network interface A02, a memory (not shown in the figure) and a database (not shown in the figure) connected through a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02 and a database (not shown in the figure). The internal memory A03 provides an environment for the operating system B01 and the computer program B02 in the non-volatile storage medium A04 to run. The database of the computer device is used to store sample data and model parameters and other data. The network interface A02 of the computer device is used to communicate with external terminals through network connection. The computer program B02 is executed by the processor A01 to implement a method for determining model training data.

[0105] Those skilled in the art can understand that Figure 7 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0106] The embodiment of the present application provides a device, which includes a processor, a memory and a program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: inputting sample data without desensitization processing into a neural network model to train the neural network model and obtain a first model; performing desensitization processing on the sample data by using multiple desensitization methods to obtain desensitized data corresponding to each desensitization method; inputting the desensitized data corresponding to each desensitization method into the neural network model to train the neural network model and obtain multiple second models; determining model parameters of the first model and each second model; comparing the model parameters of the first model with the model parameters of each second model to determine a model difference value between the first model and each second model; determining a second model with the smallest model difference value as a target model; determining a desensitization method corresponding to the target model as a target desensitization method to perform desensitization processing on the sample data using the target desensitization method to obtain data for training the neural network model.

[0107] In one embodiment, the model difference value includes an AUC value between the models, and the model difference value Y of each second model and the first model is determined according to formula (1):

[0108]

[0109] Y i is a model difference value between the second model trained by the sample data desensitized by the i-th desensitization method and the first model, V s is an AUC value of the first model, V ei is an AUC value of the second model trained by the sample data desensitized by the i-th desensitization method, is an overfitting degree value between the second model trained by the sample data desensitized by the i-th desensitization method and the first model.

[0110] In an embodiment, the sample data without desensitization processing is input into the neural network model to train the neural network model to obtain the first model, including: submitting the machine learning pipeline to the first running environment, and inputting the sample data without desensitization processing into the neural network model of the machine learning pipeline in the first running environment to train the neural network model to obtain the first model; and inputting the desensitized data corresponding to each desensitization method into the neural network model to train the neural network model to obtain a plurality of second models, including: submitting the machine learning pipeline to the second running environment, and inputting the desensitized data corresponding to each desensitization method into the neural network model of the machine learning pipeline in the second running environment to train the neural network model to obtain a plurality of second models.

[0111] In an embodiment, the method further includes: obtaining target training parameters of a target model, the target training parameters including environment running parameters of a second running environment where the target model is located, model parameters of the target model, and a desensitization method used for desensitized data for training the target model; and determining the target training parameters as training parameters for subsequent model training.

[0112] In an embodiment, the first running environment is a trusted environment, the data in the trusted environment includes sensitive data, and the second running environment is a debugging environment, and the data in the debugging environment are all desensitized data.

[0113] In an embodiment, the model parameters of the neural network model trained by the desensitized sample data are consistent with the model parameters of the neural network model trained by the sample data without desensitization.

[0114] In an embodiment, the model difference value includes a lift and / or a ks statistic between the models.

[0115] The application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program of the following method steps: inputting sample data without desensitization processing into a neural network model to train the neural network model to obtain a first model; performing desensitization processing on the sample data by using multiple desensitization methods to obtain desensitized data corresponding to each desensitization method; inputting the desensitized data corresponding to each desensitization method into the neural network model respectively to train the neural network model to obtain multiple second models; determining model parameters of the first model and each second model; comparing the model parameters of the first model with the model parameters of each second model respectively to determine a model difference value between the first model and each second model; determining the second model with the smallest model difference value as a target model; determining the desensitization method corresponding to the target model as a target desensitization method to perform desensitization processing on the sample data using the target desensitization method to obtain data for training the neural network model.

[0116] In one embodiment, the model difference value includes an AUC value between the models, and the model difference value Y between each second model and the first model is determined according to formula (1):

[0117]

[0118] wherein Y i represents the model difference value between the second model trained by the sample data desensitized using the i-th desensitization method and the first model, V s is the AUC value of the first model, V ei is the AUC value of the second model trained by the sample data desensitized using the i-th desensitization method, is the overfitting degree value between the second model trained by the sample data desensitized using the i-th desensitization method and the first model.

[0119] In one embodiment, inputting sample data without desensitization processing into a neural network model to train the neural network model to obtain a first model includes: submitting a machine learning pipeline to a first running environment, and inputting sample data without desensitization processing into the neural network model of the machine learning pipeline in the first running environment to train the neural network model to obtain the first model; and inputting the desensitized data corresponding to each desensitization method into the neural network model respectively to train the neural network model to obtain multiple second models includes: submitting the machine learning pipeline to a second running environment, and inputting the desensitized data corresponding to each desensitization method into the neural network model of the machine learning pipeline in the second running environment to train the neural network model to obtain the multiple second models.

[0120] In an embodiment, the method further comprises: obtaining target training parameters of the target model, the target training parameters comprising environment running parameters of a second running environment where the target model is located, model parameters of the target model, and a desensitization method used for desensitizing data for training the target model; and determining the target training parameters as training parameters for subsequent model training.

[0121] In an embodiment, the first running environment is a trusted environment, the data in the trusted environment comprises sensitive data, and the second running environment is a debugging environment, and the data in the debugging environment are all desensitized data.

[0122] In an embodiment, the model parameters of the neural network model trained by the desensitized sample data are consistent with model parameters of the neural network model trained by the sample data without desensitization.

[0123] In an embodiment, the model difference value comprises a lift and / or a ks statistic between the models.

[0124] Those skilled in the art will understand that embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) having computer usable program code embodied therein.

[0125] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce an apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0126] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0127] These computer program instructions can also be loaded into computer or other programmable data processing devices to cause a series of operational steps to be performed on the computer or other programmable devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable devices provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 Figure 1 One or more flowchart or block diagrams can be used to describe embodiments of the present application as can be set forth in the accompanying drawings. However, these flowcharts and block diagrams should not be construed as limiting the present application in any way. It will be readily appreciated that the order of the operations in the methodologies can be executed in any order. Moreover, individual operations can optionally be combined or the operations can be separated into additional operations.

[0128] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0129] The memory can include non-persistent memory and / or persistent memory, embodied as random access memory (RAM) such as static random access memory (SRAM), dynamic random access memory (DRAM), or other random access memories: read only memory (ROM); flash memory, or other non-volatile memory. The memory is an example of computer readable media.

[0130] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media such as modulated data signals and carrier waves.

[0131] It should also be noted that the terms "comprising", "comprises", "including", "includes" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article or apparatus that comprises the element.

[0132] ​The above merely provides an example of the present application, and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall fall into the scope of claims of the present application.

Claims

1. A method for determining model training data, characterized in that, The method comprises: inputting sample data without desensitization processing into a neural network model to train the neural network model, to obtain a first model; performing desensitization processing on the sample data by using multiple desensitization methods to obtain desensitized data corresponding to each desensitization method; inputting the desensitized data corresponding to each desensitization method into the neural network model respectively to train the neural network model, to obtain multiple second models; determining model parameters of the first model and each second model; comparing the model parameters of the first model with the model parameters of each second model respectively to determine a model difference value between the first model and each second model; determining a second model with the smallest model difference value as a target model; determining a desensitization method corresponding to the target model as a target desensitization method, to perform desensitization processing on sample data by using the target desensitization method to obtain data for training the neural network model.

2. The method for determining model training data according to claim 1, characterized in that, The inputting sample data without desensitization processing into a neural network model to train the neural network model, to obtain a first model comprises: submitting a machine learning pipeline to a first running environment, and inputting sample data without desensitization processing into a neural network model of the machine learning pipeline in the first running environment to train the neural network model, to obtain a first model. The inputting the desensitized data corresponding to each desensitization method into the neural network model respectively to train the neural network model, to obtain multiple second models comprises: submitting a machine learning pipeline to a second running environment, and inputting the desensitized data corresponding to each desensitization method into a neural network model of the machine learning pipeline in the second running environment to train the neural network model, to obtain multiple second models.

3. The method for determining model training data according to claim 2, c h a r a c t e r i z e d b y, The method further comprises: obtaining target training parameters of the target model, the target training parameters comprising environment running parameters of a second running environment where the target model is located, model parameters of the target model, and a desensitization method used for desensitized data for training the target model; determining the target training parameters as training parameters for subsequent model training.

4. The method for determining model training data according to claim 2, wherein, The first running environment is a trusted environment, data in the trusted environment comprises sensitive data, and the second running environment is a debugging environment, data in the debugging environment are all desensitized data.

5. The method for determining model training data of claim 1, wherein, Model parameters of the neural network model trained by using desensitized sample data are consistent with model parameters of the neural network model trained by using sample data without desensitization.

6. The method for determining model training data of claim 1, wherein, The model difference value comprises a lift and / or a ks statistic between models.

7. A processor, comprising: An apparatus configured to perform the method for determining model training data according to any one of claims 1 to 6.

8. An apparatus for determining model training data, the apparatus comprising: The apparatus comprises: a first training module configured to input sample data without desensitization processing into a neural network model to train the neural network model, to obtain a first model; a data desensitization module configured to perform desensitization processing on the sample data by using multiple desensitization methods to obtain desensitized data corresponding to each desensitization method; The second training module is configured to input the desensitization data corresponding to each desensitization method into the neural network model respectively to train the neural network model, and obtain a plurality of second models; The model comparison module is configured to determine model parameters of the first model and each second model, compare the model parameters of the first model with the model parameters of each second model respectively to determine a model difference value between the first model and each second model; The model selection module is configured to determine a second model with the smallest model difference value as a target model, determine a desensitization method corresponding to the target model as a target desensitization method, and use the target desensitization method to desensitize sample data to obtain data for training the neural network model.

9. The apparatus for determining model training data of claim 8, wherein, The first training module is further configured to submit the machine learning pipeline to a first running environment, and input sample data without desensitization into the neural network model of the machine learning pipeline in the first running environment to train the neural network model and obtain a first model. The second training module is further configured to submit the machine learning pipeline to a second running environment, and input desensitization data corresponding to each desensitization method into the neural network model of the machine learning pipeline in the second running environment to train the neural network model and obtain a plurality of second models.

10. The apparatus for determining model training data of claim 9, wherein, The model selection module is further configured to: obtain target training parameters of the target model, the target training parameters including environment running parameters of the second running environment where the target model is located, model parameters of the target model, and a desensitization method used for desensitization data for training the target model; determine the target training parameters as training parameters for subsequent model training.

11. The apparatus for determining model training data of claim 9, wherein, The first running environment is a trusted environment, and data in the trusted environment includes sensitive data. The second running environment is a debugging environment, and data in the debugging environment are all desensitization data.

12. The apparatus for determining model training data of claim 8, wherein, The model parameters of the neural network model trained by the desensitized sample data are consistent with the model parameters of the neural network model trained by the sample data without desensitization.

13. The apparatus for determining model training data of claim 8, wherein, The model difference value includes a lift and / or a ks statistic between models.

14. A machine-readable storage medium having stored thereon instructions, the instructions being executable by a machine to cause the machine to: The instructions, when executed by a processor, cause the processor to be configured to perform the method for determining model training data according to any one of claims 1 to 6.

15. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method for determining model training data according to any one of claims 1 to 6.

Images