Method and device for testing privacy calculation part of model reasoning in privacy calculation product

Abstract

The embodiment of the invention provides a method and device for testing a privacy computing part of model reasoning in a privacy computing product. And the privacy calculation product performs privacy calculation on the input data by using the network model contained in the privacy calculation product. During a test, the test device obtains a first test set of a plaintext, obtains a first test model which is used for testing and is not subjected to encryption processing, inputs the first test set of the plaintext into the local first test model, and obtains a first prediction result through plaintext calculation of the first test model. And meanwhile, the privacy calculation product also uses the included first test model to perform privacy calculation by taking the first test set as input data to obtain a second prediction result. And the test equipment obtains a second prediction result output by the privacy calculation product, compares the first prediction result with the second prediction result to obtain a first comparison result, and determines a first test result for the privacy calculation part based on the first comparison result.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of computer technology, and in particular, to a method and apparatus for testing the privacy computing portion of model reasoning in a privacy computing product. Background Art

[0002] Privacy computing products are a technical solution that combines technologies from multiple fields such as cryptography, data science, artificial intelligence, and security hardware. The purpose is to achieve secure computing and analysis of data while protecting data privacy. This type of product can ensure that the privacy and security of data are protected during the data processing process, while allowing the value of data to be shared and utilized. Privacy computing products have a wide range of application scenarios. They can be used in industries such as finance, communications, and healthcare to help these industries integrate and analyze data while complying with data protection regulations.

[0003] The model service provided by privacy computing products is the key support for the intelligence level of the product. The model reasoning part has become an indispensable component. For example, enterprises use the model reasoning function in privacy computing products to process massive amounts of user data and their complex behavior patterns, so as to more accurately predict customer behavior and market trends and improve marketing effectiveness. Model reasoning in privacy computing products can usually be implemented based on technologies such as multi-party secure computing, federated learning, and trusted execution environment. How much impact the privacy computing part of privacy computing products will have on the performance of model reasoning is an important aspect of product testing.

[0004] At present, we hope to have relevant solutions that can more accurately test the impact of the privacy computing part on the model reasoning performance. Summary of the invention

[0005] One or more embodiments of this specification describe a method and device for testing the privacy computing part of model reasoning in a privacy computing product, so as to more accurately test the impact of the privacy computing part on the model reasoning performance. The specific technical solution is as follows.

[0006] In a first aspect, an embodiment provides a method for testing a privacy computing part of a model reasoning in a privacy computing product, wherein the privacy computing product performs privacy computing on input data using a network model contained therein; the method is executed by a testing device, and includes:

[0007] Obtain a first test set in plain text, and obtain a first test model used for testing and not encrypted;

[0008] Inputting the first test set in plain text into the first test model locally, and obtaining a first prediction result through plain text calculation of the first test model;

[0009] Obtain a second prediction result output by the privacy computing product, where the second prediction result is obtained by performing privacy computing by the privacy computing product using the first test model contained therein and the first test set as input data;

[0010] Comparing the first prediction result and the second prediction result to obtain a first comparison result;

[0011] A first test result for the privacy computing part is determined based on the first comparison result.

[0012] In one embodiment, before obtaining the second prediction result output by the privacy computing product, the method further includes:

[0013] Upload the first test model to the privacy computing product, and input the first test set into the privacy computing product, so that the privacy computing product performs the following steps: encrypt the first test model, and / or encrypt the first test set.

[0014] In one embodiment, the privacy computing product includes a first product deployed in a first user and a second product deployed in a second user; the first test model is split into a first model and a second model, and the first test set is split into a first subset and a second subset; the first product is used to perform privacy computing using the first model and the first subset as input data; the second product is used to perform privacy computing using the second model and the second subset as input data; the second prediction result is obtained through the interaction of privacy data between the first product and the second product.

[0015] In one implementation, the step of inputting the first test set in plain text into the first test model locally includes:

[0016] Inputting the first subset of plaintext into a local first model, and performing a first plaintext calculation on the first subset through the first model;

[0017] Inputting the second subset of plaintext into a local second model, and performing a second plaintext calculation on the second subset through the second model;

[0018] The first prediction result is determined through plaintext data interaction between the first plaintext calculation and the second plaintext calculation.

[0019] In one implementation, the step of uploading the first test model to the privacy computing product includes:

[0020] Upload the first model file exported from the first test model under the first deep learning framework to the privacy computing product, so that the privacy computing product loads the first model file under its second deep learning framework to obtain the first test model; the first model file has a general format.

[0021] In one embodiment, the method further comprises:

[0022] Based on a preset first modification strategy, modify the data in the first model file to obtain a second model file that simulates an abnormal scenario;

[0023] Uploading the second model file to the privacy computing product, so that the privacy computing product executes: loading the second model file to obtain a second test model containing anomalies, and performing privacy computing using the second test model and the first test set as input data to obtain a third prediction result;

[0024] Obtaining the third prediction result output by the privacy computing product;

[0025] Comparing the third prediction result with the second prediction result to obtain a second comparison result;

[0026] A second test result for the privacy computing part is determined based on the second comparison result.

[0027] In one embodiment, the method further comprises:

[0028] Based on a preset second modification strategy, performing exception processing on the data in the first test set to obtain a second test set simulating an abnormal scenario;

[0029] Obtain a fourth prediction result output by the privacy computing product, where the fourth prediction result is obtained by: the privacy computing product performs privacy computing using the first test model contained therein and the second test set as input data;

[0030] Comparing the fourth prediction result with the second prediction result to obtain a third comparison result;

[0031] A third test result for the privacy computing part is determined based on the third comparison result.

[0032] In one embodiment, the first test model is any one of a plurality of test models; the plurality of test models include network models constructed under the following model architectures: a feedforward neural network, a convolutional neural network, a recurrent neural network, and a natural language processing model.

[0033] In one embodiment, the first test set is any one of a plurality of test sets; the data distribution of the samples in the plurality of test sets includes: uniform distribution, normal distribution or skewed distribution.

[0034] In a second aspect, an embodiment provides a device for testing a privacy computing part of a model reasoning in a privacy computing product, wherein the privacy computing product performs privacy computing on input data using a network model contained therein; the device is deployed in a test device and includes:

[0035] A digital model acquisition module is configured to acquire a first test set in plain text and acquire a first test model for testing that has not been encrypted;

[0036] A local test module is configured to input the first test set in plain text into the first test model locally, and obtain a first prediction result through plain text calculation of the first test model;

[0037] A product-side testing module is configured to obtain a second prediction result output by the privacy computing product, where the second prediction result is obtained by performing privacy computing by the privacy computing product using the first test model contained therein and the first test set as input data;

[0038] A prediction comparison module, configured to compare the first prediction result and the second prediction result to obtain a first comparison result;

[0039] A result determination module is configured to determine a first test result for the privacy calculation part based on the first comparison result.

[0040] In a third aspect, an embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute any one of the methods described in the first aspect.

[0041] In a fourth aspect, an embodiment provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described in any one of the first aspects is implemented.

[0042] The method and device provided in the embodiments of this specification use the same test set and the same test model to perform model prediction on the local side and the product side respectively, and the local side uses plaintext data for plaintext calculation, and the product side uses the same test set and the same test model for privacy calculation, and the test results of the privacy calculation part in the product are determined by comparing the local prediction results with the product prediction results. This comparative test method can more accurately test the degree of influence of the privacy calculation part in the privacy computing product on the model reasoning performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A schematic diagram of an implementation scenario of an embodiment disclosed in this specification;

[0045] Figure 2-1 and Figure 2-2 These are two process diagrams of multiple parties using privacy computing products to perform joint model reasoning;

[0046] Figure 2-3 It is a processing flow chart of how privacy computing products split and encrypt models before performing model reasoning;

[0047] Figure 3 A flowchart of a method for testing the privacy computing part of model reasoning in a privacy computing product provided in an embodiment;

[0048] Figure 4 It is a structural diagram of the ONNX model file;

[0049] Figure 5 A schematic block diagram of an apparatus for testing the privacy computing part of model reasoning in a privacy computing product provided in an embodiment. DETAILED DESCRIPTION

[0050] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0051] Figure 1 This is a schematic diagram of an implementation scenario of an embodiment disclosed in this specification. It includes test equipment and users of privacy computing products. For example, the users include user A and user B, etc. The first product is deployed in user A, and the second product is deployed in user B. Here, the first product and the second product both belong to the privacy computing product. User A and user B can interact with privacy data. Each user can process data through their own computing device. Figure 1 It is just a schematic diagram of a scenario. In actual application, there may be one or more users.

[0052] Privacy computing products include privacy computing. Privacy computing is a technology that processes and analyzes sensitive data through encryption and privacy protection technology. Its goal is to protect individual privacy while allowing data owners to perform meaningful calculations and analyses on data without directly accessing the original data. The privacy computing product here (hereinafter referred to as the product) can be understood as an application or a combination of software and hardware. It can provide an application interface for users to operate. The privacy computing product includes a model reasoning part, which includes a network model. The product performs model reasoning through the network model. This model reasoning is privacy computing based on privacy protection. Model reasoning refers to the process of using a trained machine learning model to predict or infer new input data.

[0053] The above network model refers to a neural network. Neural networks are an algorithmic framework inspired by the structure and function of the human brain, used to simulate the human learning process, and widely used in the fields of machine learning and artificial intelligence. They consist of a large number of nodes (or "neurons"), which are usually organized in multiple layers, and the nodes of each layer can be connected to the nodes of the previous and / or next layers. The core function of a neural network is to perform various tasks such as classification, regression, clustering, generation, and reinforcement learning by learning patterns from data.

[0054] Privacy computing products can be provided by service providers to users. In product applications, the network model in the privacy computing product can be provided by the service provider or uploaded to the product by the user. The user can use the model reasoning function in the product to perform reasoning calculations on the business data of the party and obtain the reasoning analysis results of the business data. Among them, the business data of the user contains privacy data. When reasoning and calculating it, the product can ensure that the privacy data is not leaked. At the same time, when the user uploads the model to the product, the product will also perform privacy processing on the model to protect the privacy data in the model from being leaked.

[0055] When there are multiple users, multiple users can perform joint model reasoning based on the models in their respective products. The following describes the privacy computing process in the product by taking user A and user B performing joint model reasoning as an example in a specific scenario. Figure 2-1 and Figure 2-2 These are two process diagrams of multiple parties using privacy computing products to perform joint model reasoning.

[0056] exist Figure 2-1In the scenario shown, user A initiates a model inference task and sends a notification message of remote model calculation to user B. User A and user B each perform model calculation based on the loaded data and model to obtain their own calculation results (such as classification results). User B returns the calculation result to user A. User A decrypts the remote calculation result returned by user B and merges the double-end calculation results to obtain the final calculation result. In this scenario, each party uses different data for prediction, and after each obtains the prediction result, the prediction results are merged to obtain the final prediction result of this model inference.

[0057] exist Figure 2-2 In the scenario shown, user A loads data and models, performs the first layer of model calculations, obtains calculation results, and initiates remote model calculations to user B. User B loads data and models, performs the first layer of model calculations, obtains calculation results, and returns the calculation results to user A. User A decrypts the remote calculation results returned by user B, and merges the double-end calculation results. User A continues to perform subsequent model calculations to obtain the final calculation results. In this scenario, each party uses different data to perform model reasoning of part of the calculation layer, and merges the intermediate results of the output of part of the calculation layer obtained by the reasoning of each party. Then one of the participants continues to perform subsequent model reasoning based on the intermediate results to obtain the final prediction result of this model reasoning. Figure 2-2 In the scenario shown, the calculation results of the first layer of both parties are merged. The first layer here is just an example, and the calculation results of multiple layers of both parties can also be merged.

[0058] The above content explains the different modes of the model reasoning process in privacy computing products. These modes or scenarios are just examples, and there may be other modes or scenarios in actual applications. In the above reasoning process, due to the different data characteristics processed by the models of each party, the privacy computing product also provides splitting and encryption of the complete model before performing model reasoning. Figure 2-2 The scenario shown is used as an example to illustrate the model segmentation processing flow of the privacy computing product.

[0059] Figure 2-3It is a processing flow chart for the privacy computing product to split and encrypt the model before executing model reasoning. Take the first product in user A to split the model as an example. After obtaining the complete network model, the first product can split the complete network model according to the model information, for example, split the first layer of the model to obtain the first layer model and the remaining model, and split the first layer model according to the characteristic fields of the data owned by each user to obtain the model parameters of the first layer of party A and the model parameters of the first layer of party B. The remaining model and the model parameters of the first layer of party A form the model information of party A. The first product can directly use the model parameters of the first layer of party B as the model information of party B without encrypting them; or, it can also encrypt the model parameters of the first layer of party B, obtain the encrypted model parameters as the model information of party B, and send them to party B. In this scenario, the complete network model is obtained by training by party A, and party A can choose to encrypt or not encrypt the model parameters split to party B according to privacy requirements. In actual applications, the above splitting operation can also be performed by a third party. The algorithm for encrypting the model parameters can be a symmetric encryption algorithm, a homomorphic encryption algorithm, or a secret sharing algorithm.

[0060] Summarizing the model reasoning process listed above, we can see that the privacy processing that may be involved in model reasoning in privacy computing products includes: encryption of the model, encryption of the data, or encryption of both. In addition, the process of joint model reasoning involves the interaction of private data between multiple parties. The above content provides an enumerated description of privacy computing products. Understanding the model reasoning process of privacy computing products will help explain the subsequent test plan.

[0061] When a privacy computing product is first created or put into use after a version update, it is necessary to test the privacy computing product. At present, the requirements for privacy computing products are mainly reflected in the following aspects: one is that the level of privacy protection must be high, and the other is that the model prediction must be accurate. Privacy computing will inevitably affect the model reasoning process, resulting in certain errors. If the error is controlled within a certain range, it is an important aspect to improve the accuracy of the product. Therefore, it is necessary to test the privacy computing part of the model reasoning in the privacy computing product to test the degree of influence of the privacy computing part on the model reasoning performance.

[0062] In order to more accurately test the degree of influence of the privacy computing part on the model reasoning performance, an embodiment of this specification provides a testing method. In this testing method, at the local end, the first test set of plain text is input into the local first test model, and the first prediction result is obtained by the plain text calculation of the first test model; at the product end, the privacy computing product uses the first test model contained therein to perform privacy calculation with the first test set as input data to obtain a second prediction result; the first prediction result and the second prediction result are compared to obtain a first comparison result, and the first test result for the privacy computing part is determined based on the first comparison result. Both the local end and the product end perform model reasoning based on the same test set and test model. The difference is that the local end performs plain text reasoning and the product end performs privacy reasoning. Therefore, the comparison result can reflect the influence of the privacy computing part on the model reasoning result, thereby obtaining a more accurate reasoning result.

[0063] Combine the following Figure 3 This embodiment will be described in detail.

[0064] Figure 3 A flowchart of a method for testing the privacy computing part of model reasoning in a privacy computing product provided in an embodiment. The privacy computing product is used to provide model reasoning services. When providing model reasoning services, the network model contained therein is used to perform privacy calculations on the input data, and this privacy calculation is used to protect the privacy data in the model and input data from being leaked. The privacy computing product is provided by the service provider to the user, and the user can be one or more. When the product is actually used (not during testing), the user can upload his own network model and / or data to the product and use the model reasoning service provided by the product. The service can be online or offline. During testing, multiple computing devices can be used to simulate the user, and the product can be tested using a test model and a test set. The test method is executed by the test device and includes the following steps.

[0065] Step S310: The testing device obtains a first test set T1 in plain text, and obtains a first test model N1 that is used for testing and has not been encrypted.

[0066] The first test set T1 contains multiple samples for testing the privacy computing product P1. The features of these samples are all plaintext data. The distribution of sample features in the first test set T1 can be uniform distribution, normal distribution, or skewed distribution to be as close as possible to the features of data in the real world. The data types of sample features in the first test set T1 can include different numerical types (such as integers, floating point types) and non-numerical data (such as strings or enumeration types).

[0067] Before testing, data and model preparation can be performed in advance. When preparing data, it can include collecting data, cleaning data, handling missing values, normalizing or standardizing data, and may also include more complex steps such as feature engineering and data enhancement. After the data is properly prepared and processed, it needs to be split into training sets and test sets so that the model can learn on the training set and perform product evaluation on the test set to ensure the generalization ability of the model. When designing the data scenario of the first test set T1 for testing, pay attention to the shape, type, and arrangement order of the data, which can improve the accuracy of model reasoning in privacy computing products and make the testing process of privacy computing more rigorous.

[0068] The first test model N1 can be implemented using a neural network model. For example, the network model can be constructed under the following model architecture: feedforward neural network, convolutional neural network, recurrent neural network or natural language processing model, etc. The first test model N1 is a trained model used to make business predictions on input data and can output prediction results relatively accurately. The prediction result can be a classification value or a regression value, etc. The first test model N1 is a model used to test the privacy computing product P1, and is not a model of the privacy computing product P1 in actual use. The model parameters contained in the first test model N1 are in plain text and have not been encrypted. The first test model N1 can be designed to perform specific tasks, such as identifying risks for users, or determining the answer text for the input question text, etc.

[0069] When designing the first test model N1, when choosing different model architectures, it is necessary to determine the type and order of various layers (such as convolutional layers, pooling layers, fully connected layers, etc.), as well as the parameters of each layer (such as the number of neurons, activation functions, regularization methods, etc.). This design process may need to depend on the nature of the problem, the characteristics of the data, and previous research on similar problems. In the field of privacy computing, in order to prevent model information leakage, the product side usually encrypts the first layer of the model (see Figure 2-3 As shown), more attention can be paid to the first layer of the model when designing the first test model N1.

[0070] In privacy computing, ensuring the accuracy of model reasoning can ensure that the testing process of the privacy computing part is more rigorous and accurate. This requires a detailed investigation of the structure of the neural network in the first test model N1. Figure 2-3 Taking the products involved as an example, we can focus on the different types of the first layer of the network model. The first layer is the layer that the input data first passes through, and it is crucial to the accuracy and effectiveness of all subsequent operations. The following are several common types of the first layer of neural network models, and how to ensure the accuracy of model reasoning in privacy computing by focusing on these types.

[0071] For feedforward neural network models, such as in multilayer perceptrons or dense networks, the first layer is usually a fully connected layer. It is necessary to ensure that the dimension of the input data matches the weight matrix of the fully connected layer to avoid errors caused by inconsistent dimensions.

[0072] For convolutional neural network models, the first layer is usually the convolutional layer. It is necessary to pay attention to parameters such as the convolution kernel size, step size, and padding method to ensure that they are correctly implemented in privacy computing to avoid compromising the reasoning accuracy.

[0073] In recurrent neural networks, which include variants such as LSTM and GRU, the first layer is the recurrent layer. In a privacy computing environment, special attention should be paid to whether the state information is accurately maintained and updated, because incorrect state processing will affect the reasoning results of the entire sequence.

[0074] For natural language processing models, the embedding layer can map discrete words to a continuous vector space. The dimension of the word embedding must be consistent with the input requirements of the subsequent layers, and the embedding vector must remain unchanged during the privacy computing reasoning process.

[0075] In privacy computing, the calculation of the normalization layer must take into account the challenges brought by data encryption to ensure that unpredictable errors are not introduced. For example, the batch normalization layer in a convolutional neural network is usually used to stabilize the training process of the convolutional neural network.

[0076] Focusing on the different types of the first layer of the first test model N1 can ensure the accurate transmission and processing of input data in the privacy computing environment. This helps the model maintain accuracy while protecting data privacy, and also helps to discover possible problems and further optimize the reasoning performance of the model in privacy computing.

[0077] Model training for the first test model N1 is an iterative process that requires setting appropriate loss functions, optimizers, and evaluation metrics. At this stage, the model will gradually improve its performance through a cycle of forward propagation, loss calculation, backpropagation, and weight update. Techniques such as early stopping, model checkpoints, and learning rate decay can be used to improve the efficiency of the training process and the performance of the final model.

[0078] When the training is complete, the first test model N1 will be evaluated to verify its performance on independent test data. The evaluation may use various indicators such as accuracy, precision, recall, F1 score, ROC-AUC, etc., depending on the nature of the task. The evaluation can give a more objective understanding of the actual performance of the model.

[0079] Data preparation, model preparation, model training and other processes can be performed in the test device or in other devices. When the product needs to be tested, the test model can obtain the first test set T1 and the first test model N1 from other devices.

[0080] In step S320, the test device inputs the plaintext first test set T1 into the local first test model N1, and obtains the first prediction result Y1 through the plaintext calculation of the first test model N1. When the first test set T1 contains n samples, n prediction results can be calculated by the first test model N1, and the first prediction result Y1 contains the prediction results of n samples, that is, n prediction results. When the plaintext and complete first test set T1 is input into the local first test model N1, multiple samples can be input one by one. The model prediction process performed in this step is completely unencrypted and is a plaintext calculation process. The specific model reasoning process will not be repeated here, and reference can be made to the existing technology.

[0081] Step S330: The test device obtains a second prediction result Y2 output by the privacy computing product P1. The second prediction result Y2 is obtained by the privacy computing product P1 performing privacy computing using the first test model N1 contained therein and the first test set T1 as input data.

[0082] When the first test set T1 includes n samples, the second prediction result Y2 includes the prediction results of n samples, that is, n prediction results. The test device can obtain the second prediction result Y2 from the device running the privacy computing product P1.

[0083] Before the test device obtains the second prediction result Y2 output by the privacy computing product P1, the test device can upload the first test model N1 to the privacy computing product P1 and input the first test set T1 into the privacy computing product P1. Of course, the first test model N1 and the first test set T1 can also be uploaded to the privacy computing product P1 through other devices.

[0084] Among them, the privacy computing product P1 can be deployed in one user or in multiple users.

[0085] When the privacy computing product P1 is deployed in multiple users, for example, the privacy computing product P1 includes a first product deployed in a first user (for example, user A) and a second product deployed in a second user (for example, user B), the first test model N1 is split into a first model and a second model, and the first test set T1 is split into a first subset and a second subset to simulate the actual product usage scenario.

[0086] The first product is used to perform privacy calculation using the first model and the first subset as input data, and the second product is used to perform privacy calculation using the second model and the second subset as input data. The second prediction result Y2 is obtained through the interaction of privacy data between the first product and the second product. The process of the first product and the second product performing model reasoning through privacy data interaction can be seen in Figure 2-1 and Figure 2-1 , I will not go into details here.

[0087] The execution of the above steps S320 and S330 can be in no particular order. Corresponding to the situation in which the privacy computing product P1 is deployed in multiple users in step S330, when executing step S320, that is, inputting the first test set T1 of plaintext into the local first test model N1, the test device can also adopt the following steps: inputting the first subset of plaintext into the local first model, and performing a first plaintext calculation on the first subset through the first model; inputting the second subset of plaintext into the local second model, and performing a second plaintext calculation on the second subset through the second model; determining the first prediction result Y1 through the interaction of plaintext data between the first plaintext calculation and the second plaintext calculation. This embodiment can be used as a feasible option.

[0088] Now let's go back to step S330. When there are multiple users, the second prediction result Y2 is the merged result of the samples in the first test set T1. The test device can directly obtain the second prediction result Y2 from the device of a certain user, or obtain unmerged prediction results from the devices of multiple users, merge these prediction results, and obtain the second prediction result Y2.

[0089] The splitting process of the first test model N1 and the splitting process of the first test set T1 may be performed by a test device, or by a device of a user, or by other devices.

[0090] In summary, during testing, the privacy computing product P1 (including the first product and the second product) mainly performs the following steps: encrypt the first test model N1, and / or encrypt the first test set T1. The encryption here includes encryption algorithms used in multi-party secure computing such as homomorphic encryption and secret sharing, as well as symmetric encryption or asymmetric encryption based on trusted computing.

[0091] For example, the privacy computing product P1 encrypts the first test model N1 to obtain an encrypted model, and uses the encrypted model to perform privacy computing with the first test set T1 as input data. Alternatively, the privacy computing product P1 uses the first test set T1 as input data, encrypts the input data to obtain encrypted data, and uses the first test model N1 to perform privacy computing on the encrypted data. Alternatively, the privacy computing product P1 encrypts the first test model N1 to obtain an encrypted model, uses the first test set T1 as input data, encrypts the input data to obtain encrypted data, and then uses the encrypted model to perform privacy computing on the encrypted data. These include the following types: one is model encryption but data is not encrypted, one is model encryption but data is encrypted, and one is both encrypted.

[0092] Step S340, compare the first prediction result Y1 and the second prediction result Y2 to obtain a first comparison result. Both sets of results include n data, and the consistency between the two sets of n data is compared accordingly to obtain a first comparison result. The comparison can be done automatically by writing a script, mainly to check whether there are differences between the two sets of results. The first comparison result can show the consistency or difference between the two sets of results, such as the ratio of the number of consistent results, or the ratio of the number of inconsistent results. If a difference is found, the cause can be further analyzed, such as data transmission errors, model service configuration problems, or other technical problems.

[0093] Step S350: Determine a first test result for the privacy computing part based on the first comparison result. The first prediction result may include the accuracy and credibility analysis of the privacy computing part in this test.

[0094] Since some errors may be introduced during the encryption and decryption process, it is necessary to evaluate the error tolerance of the results. The error can be controlled within a preset range. When the inconsistency between the two sets of results does not exceed the preset range, it means that the accuracy of the ciphertext calculation will not have a negative impact on business decisions.

[0095] In this embodiment, the process shown in the above steps S310 to S350 is a testing process. In order to improve the accuracy of the test, the privacy computing product can also be tested multiple times. For example, multiple tests can be performed by replacing the first test set T1. The first test set T1 can be selected from multiple test sets, and the data distribution of samples in the multiple test sets includes: uniform distribution, normal distribution or skewed distribution. Each time a test set is selected, a test result can be obtained. In this way, corresponding test results can be obtained for different data distributions in the test set, thereby simulating the diversity of the real world and improving the accuracy of model reasoning. When the accuracy of model reasoning is improved, the privacy computing part can be accurately tested.

[0096] When designing the test set, if the data scenarios cover various abnormal situations that the model may encounter in the real world, the test of the privacy computing part will be more accurate and the test results will be closer to the actual situation.

[0097] In another embodiment, after step S310, the testing device may further perform the following steps 1a to 4a.

[0098] Step 1a: Based on a preset second modification strategy, perform exception processing on the data in the first test set T1 to obtain a second test set T2 that simulates an abnormal scenario.

[0099] Step 2a, obtaining the fourth prediction result Y4 output by the privacy computing product P1. The fourth prediction result Y4 is obtained by the privacy computing product P1 performing privacy computing using the first test model N1 contained therein and the second test set T2 as input data.

[0100] Step 3a, comparing the fourth prediction result with the second prediction result to obtain a third comparison result.

[0101] Step 4a: Determine a third test result for the privacy calculation part based on the third comparison result.

[0102] In step 1a, the first test set T1 can be understood as original data, and the second test set T2 can be understood as a data set containing abnormal situations. The second modification strategy can include any one of a plurality of modification strategies.

[0103] In this embodiment, the first test set T1 can be any one of the multiple test sets. The sample features in the multiple test sets can have different data distributions. Now, taking the first test set T1 as an example, the process of performing exception processing on the test set in step 1a is explained, which includes the following modification strategies.

[0104] Change the order of data features. The test data should include different feature arrangements and orders, and attention should be paid to the performance of the model when the order of the test set features is inconsistent with the training set. That is, adjust the order of samples in the first test set T1 to obtain the second test set T2.

[0105] Add edge cases like extreme values, outliers, missing values, or illegal values ​​in feature values ​​to ensure the model can handle these cases robustly.

[0106] Considering the characteristics of encrypted data, such as noise accumulation in homomorphic encryption and changes in numerical precision that may be introduced by the encryption algorithm, modifications to eigenvalues ​​can be introduced.

[0107] In addition, during testing, the first test model N1 may also be tested for batch data inference to ensure the accuracy and performance of the model when processing large amounts of data.

[0108] In step 2a, the fourth prediction result Y4 includes the prediction results of n samples. The process of the test device obtaining the fourth prediction result Y4 and the process of the privacy computing product P1 determining the fourth prediction result Y4 can refer to the process of the test device obtaining the second prediction result Y2 in step S330 and the relevant description of the privacy computing product P1 determining the second prediction result Y2, which will not be repeated here.

[0109] For step 3a, reference may be made to the relevant description in step S340, and for step 4a, reference may be made to the relevant description in step S350, which will not be repeated here.

[0110] The above steps 1a to 4a may also be executed multiple times, and the second modification strategy selects a different modification strategy each time, thereby obtaining test results under different modification strategies.

[0111] In this embodiment, the first test set T1 is modified abnormally to obtain the second test set T2, and the privacy computing product P1 is tested using the second test set T2 to obtain the fourth prediction result Y4. Then the fourth prediction result Y4 can be compared with the second prediction result Y2 obtained when the privacy computing product P1 is tested using the first test set T1. In other words, the prediction result obtained by the product end using the original data is compared with the test result obtained using the test set containing abnormal data. The test result can reflect the degree of influence of the privacy computing part on the reasoning result when the input data is abnormal.

[0112] In another embodiment, in order to improve the accuracy and comprehensiveness of the test, the product can also be tested using test models of different model architectures. For example, multiple test models can be pre-trained, and the multiple test models include network models constructed under the following model architectures: feedforward neural network, convolutional neural network, recurrent neural network and natural language processing model, etc. The first test model N1 mentioned above is replaced with any one of the multiple test models, and steps S310 to S350 can be used to obtain test results under different model architectures, thereby obtaining richer and more comprehensive test results.

[0113] In another embodiment, since the privacy computing product can allow the user to upload their own model in actual application, there is a difference between the deep learning architecture used by the privacy computing product and the deep learning architecture of the user. For example, the product uses the TensorFlow architecture, and the user's network model is trained using the PyTorch architecture.

[0114] When the test device uploads the first test model N1 to the privacy computing product P1 in step S330, the first model file derived from the first test model N1 under the first deep learning framework can be uploaded to the privacy computing product P1, so that the privacy computing product P1 loads the first model file under its second deep learning framework to obtain the first test model N1. The step of the privacy computing product P1 loading the first model file under the second deep learning framework it contains can refer to the existing technology and will not be repeated here.

[0115] The first deep learning framework and the second deep learning framework may be the same or different. For example, the two deep learning frameworks may be in the framework of PyTorch, TensorFlow, Caffe2, etc. The deep learning framework provides tools and libraries for implementing various neural network models. For example, PyTorch or TensorFlow may be used to build and train convolutional neural networks, recurrent neural networks, or natural language processing models.

[0116] The first model file has a universal format and can be used in the first deep learning framework and the second deep learning framework. The universal format here means that it can be used in the first deep learning framework and the second deep learning framework, and the format enables the first model file to be transferred across different deep learning frameworks. The universal format may be, for example, the Open Neural Network Exchange (ONNX) format. Model files in the ONNX format provide an open standard format process for machine learning models. ONNX is also an open deep learning model exchange format. Its goal is to provide a universal framework that allows models to be seamlessly shared, converted, and deployed between different deep learning frameworks. ONNX enables models to be transferred across different deep learning frameworks, thereby enhancing the portability and interoperability of the model.

[0117] Figure 4It is a structural diagram of an ONNX model file. An ONNX model file can be represented by the ModelProto class. ModelProto contains log information such as version (version), creator (provider), and graph information that stores the structure of the computational graph. The ModelProto class is usually the highest-order structure, defining the structure of the entire network model. The GraphProto class includes input tensor information (input), output tensor information (output), node information (Node), and model weights (initializer). The GraphProto class defines the model's calculation logic and the Node nodes with parameters, forming a directed graph structure. The tensor information ValueInfoProto class includes tensor name, basic data type, and shape. The NodeProto class containing node information contains relevant information (type and shape, etc.) such as the operator name (name), operator input tensor, operator output tensor, etc. The NodeProto class is used to store the shape and tensor dimension information of the network's input and output. The NodeProto class contains the name (op_name), input, output, and attribute list of each node. The property list contains the hyperparameter settings of the function, such as the convolution kernel size and step size. The NodeProto class defines the structure of each node in the network directed graph, and each node corresponds to a layer in the model.

[0118] The ONNX model file treats each layer or operator of each network as a node, and then uses these nodes to build a computational graph, which is equivalent to a network. Finally, the graph is combined with other information of the ONNX model file to generate a ModelProto class, which is the final ONNX model file.

[0119] During testing, it is sometimes necessary to modify the content of the ONNX model file to adapt to a specific reasoning environment or meet certain testing requirements. In order to improve the robustness of the model and ensure the accuracy of reasoning in the face of various abnormal scenarios, a series of modifications can be made to simulate various possible abnormal situations. By comparing the prediction results of the product under the original ONNX model file and the ONNX model file after adding abnormalities, the differences shown by privacy computing can be tested.

[0120] In another embodiment, in order to simulate the anomalies contained in the model of the product in the world, and to test the differences shown by privacy computing based on the prediction results of the comparison product under the original ONNX model file and the ONNX model file after adding the anomalies, thereby improving the accuracy and comprehensiveness of the test, after step S310, the test device can also perform the following steps 1b to 4b.

[0121] Step 1b: based on a preset first modification strategy, modify the data in the first model file N1 to obtain a second model file N2 that simulates an abnormal scenario.

[0122] Step 2b, upload the second model file N2 to the privacy computing product P1, so that the privacy computing product P1 performs the following steps: load the second model file to obtain a second test model N2 containing anomalies, and use the second test model N2 and the first test set T1 as input data to perform privacy calculations to obtain a third prediction result Y3.

[0123] Step 3b, obtain the third prediction result Y3 output by the privacy computing product P1.

[0124] Step 4b, comparing the third prediction result Y3 and the second prediction result Y2 to obtain a second comparison result.

[0125] Step 5b: Determine a second test result for the privacy calculation part based on the second comparison result.

[0126] In step 1b of this embodiment, the first model file N1 can be understood as the original model, and the second model file N2 can be understood as the test model containing the abnormal situation. The first modification strategy can include any one of a plurality of modification strategies. The second test model N2 is substantially similar to the first test model N1, except that the second test model N2 contains a small amount of abnormalities. Such abnormalities are usually not too large.

[0127] The first model file N1 can be any one of multiple test models. Now, taking any first model file N1 as an example, the process of performing exception handling on the first test model N1 in step 1b is described. The following are some possible modification strategies for the first model file, and how these modification strategies can help verify and improve the accuracy of model reasoning in privacy computing.

[0128] Changing node types. For example, changing the first layer from a Gemm (general matrix multiplication plus bias) node to a Relu (non-linear activation function) node. This type of change can break the functionality of the model because the Gemm and Relu operations are mathematically completely different and they require different inputs and parameters. Such changes may cause the model to fail when loading, or even if it loads, it will produce errors at runtime.

[0129] Adding nodes. Adding new nodes to the model, such as additional activation functions, regularization layers, or other transformations, can simulate changes or disturbances that the data may encounter during transmission. The added nodes must be compatible with the existing graph structure. Adding unconnected nodes may cause problems when the model is executed because the inputs and outputs of the new nodes may not be correctly connected to the existing computational graph.

[0130] Modify layer inputs / outputs. Changing the input-output connections of a layer allows you to test how the model responds to unexpected changes in the input data path, which in privacy computing can occur due to data encryption and decryption operations. Modifications to node inputs and outputs should be done with caution to avoid compromising the integrity of the model. Incorrect input or output modifications may cause errors in the flow of data through the model, which can affect the operation or accuracy of the model.

[0131] Modify attribute values. Adjust node attributes such as alpha, beta, transB (whether to transpose the B matrix in matrix multiplication), pads, strides, etc. When modifying node attributes, be careful to keep the attribute values ​​within an acceptable range. Incorrect attribute values ​​can lead to calculation errors in model execution, especially when these attribute values ​​are used to define operational behaviors, such as padding size, stride, etc.

[0132] Delete or replace binary weight data. Deleting the model's weights or artificially replacing certain weight values ​​can simulate situations where the weight data is corrupted or transmitted incorrectly, thereby testing the model's fault tolerance. Deleting the model's binary bytes may cause the model file to become corrupted and unable to load or parse correctly. ONNX model files use a specific binary format that contains various information and parameters of the model. If you delete the bytes, the model file may become invalid.

[0133] The above modification strategy can make targeted modifications to the ONNX model file to test and verify the robustness and accuracy of the model when dealing with various abnormal scenarios. This is especially important for privacy computing products, because encryption and privacy protection measures may introduce additional complexity and challenges. These measures can ensure that the model can still provide reliable reasoning results in complex real-world environments.

[0134] Next, we will explain the actual operation process when executing test tasks on the product side. The test task execution phase involves deploying the trained model to the product environment and running the model on actual data for prediction or reasoning. The following are the detailed steps of this phase: the product side initiates the ONNX model reasoning task, deploys the model service task, the task is approved by the user, the model is scored, and the scoring results are obtained.

[0135] Initiate an ONNX model inference task on the product side and upload the first model file previously trained and exported in ONNX format to the product environment. The upload can be done through the product side interface, or through automated scripts and application programming interfaces (APIs).

[0136] When deploying a model service task, the configuration includes information such as the model service name, source data, and calculation logic. After configuration, the model service can be deployed online.

[0137] Tasks are approved by users. In privacy computing scenarios, tasks are completed by multiple users. Especially in scenarios involving sensitive data or key business operations, reasoning tasks may need to be approved by partners or relevant regulatory agencies. The approval process may include approval of models and data to ensure compliance of deployment.

[0138] Perform model scoring. Once approval is complete, the model service is ready to start accepting actual input data and making predictions or inferences. The data will be passed to the model service and the model will generate output based on its training, which is often referred to as model scoring.

[0139] Get the scoring results. After the reasoning is completed, the prediction results, such as the first prediction result, the second prediction result, the third prediction result, and the fourth prediction result, will be obtained. These results can be category labels, probability scores, regression values, etc. The results will be used by subsequent business processes, such as personalized recommendations, risk assessment, and automated decision-making. Depending on the needs of the application, the scoring results may need further processing or presentation, such as displaying them to end users in the form of visualization or reports.

[0140] Next, we will explain the actual operation process when the test task is executed locally on the test device. The local task execution stage mainly involves scoring the ONNX model file generated before the local call, obtaining the first prediction result and saving it. The specific steps include: loading the ONNX model file, creating an inference session, preparing input data, performing model inference, processing output results, and using inference results.

[0141] Load ONNX model file. Load a deep learning model file that was previously trained and exported to ONNX format. You can use onnx.load to load a model from a model file and convert it to an ONNX model object.

[0142] Create an inference session. Use the ONNX library to create an inference session for performing model inference. The inference session takes the loaded ONNX model as input and provides inference capabilities.

[0143] Prepare input data. Prepare input data according to the input requirements of the model. The shape, type, and order of the input data need to match the input of the model. The specific scenario data has been given in the data preparation stage. You can use NumPy arrays or other supported data types to store input data.

[0144] Perform model inference. Use the inference session to infer the input data and get the output results. You can use the run method to execute inference and pass in the input data.

[0145] Process the output results. Process the inference output results according to the output requirements of the model. The shape, type, and order of the output results need to match the output of the model. You can use NumPy arrays or other supported data types to process the output data.

[0146] Use the inference results. Perform subsequent operations and applications based on the inference results. You can perform tasks such as classification and regression based on the design of the model.

[0147] After the deployment and reasoning process of the test model is executed, verifying the test task results is a key step to ensure the accuracy and reliability of the model. The following is the specific process of verifying the test task results, including: downloading the product results, saving the local scoring results, comparing the local results with the product result data, obtaining the test results, error evaluation, and the steps after the verification is completed.

[0148] Download the results from the product side. Once the model inference task is completed in the product environment, the inference results need to be downloaded from the product system. This is usually done through the API or user interface provided by the product to ensure accurate download and integrity of the results.

[0149] Save the local scoring results. At the same time, the same model inference task should be run locally, using the same data set as the product side, in order to generate a benchmark result. These local scoring results are saved for subsequent comparative analysis.

[0150] Compare local results with product-side result data. When both sets of prediction results are ready, compare the locally generated scoring results with the results downloaded from the product side. The comparison can be done automatically by writing a script or manually, mainly to check whether there are any differences between the two sets of results.

[0151] Get the test results. After the comparative analysis is completed, you will get a report showing the consistency or difference between the two sets of results. If there is a difference, you need to further analyze the cause, which may be a data transmission error, a model service configuration problem, or other technical problems.

[0152] Error assessment. In principle, for plaintext data scoring, the inference results on the product side and locally should be exactly the same, without any errors. For ciphertext data scoring, since some errors may be introduced during the encryption and decryption process, the results need to be evaluated for error tolerance. The error needs to be controlled within 6 decimal places, which is usually enough to ensure that the accuracy of ciphertext calculations does not have a negative impact on business decisions.

[0153] The steps after the verification is completed. If the test results meet the accuracy requirements, the model reasoning task can be considered successful, and the user's model can be used in the product environment, indicating that the entire process test has passed. If it does not meet the requirements, it means that the entire process test has failed, and it is necessary to trace back and analyze the model's reasoning process to find the root cause of the difference, which may involve retraining the model, adjusting the reasoning service configuration, and correcting the data processing process.

[0154] This series of result verification steps can ensure that the model performs as expected in the actual product environment. For scenarios with special requirements in terms of data privacy and security, when using encrypted data for scoring, the verification process also ensures that the model's reasoning accuracy is properly controlled while protecting user privacy.

[0155] In this specification, the word "first" in terms such as the first test set, the first test model, the first prediction result, the first comparison result, etc., and the corresponding "second" (if any) in the text, etc., are merely for the convenience of distinction and description and do not have any limiting meaning.

[0156] In this specification, the test equipment and the user's equipment can be implemented by any device, equipment, platform, equipment cluster, etc. with computing and processing capabilities.

[0157] The foregoing describes certain embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in an order different from that in the embodiments, and the desired results may still be achieved. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0158] Figure 5 A schematic block diagram of a device for testing the privacy computing part of model reasoning in a privacy computing product provided in an embodiment. The privacy computing product uses the network model it contains to perform privacy computing on input data. Figure 3 The device 500 is deployed in a test device and includes:

[0159] A digital model acquisition module 510 is configured to acquire a first test set in plain text, and acquire a first test model for testing that has not been encrypted;

[0160] The local test module 520 is configured to input the first test set in plain text into the first test model locally, and obtain a first prediction result through plain text calculation of the first test model;

[0161] The product-side testing module 530 is configured to obtain a second prediction result output by the privacy computing product, where the second prediction result is obtained by performing privacy computing by the privacy computing product using the first test model contained therein and the first test set as input data;

[0162] A prediction comparison module 540 is configured to compare the first prediction result and the second prediction result to obtain a first comparison result;

[0163] The result determination module 550 is configured to determine a first test result for the privacy calculation part based on the first comparison result.

[0164] In one embodiment, the device 500 further includes: a digital model uploading module (not shown in the figure). The digital model uploading module is configured to: before obtaining the second prediction result output by the privacy computing product, upload the first test model to the privacy computing product, input the first test set into the privacy computing product, so that the privacy computing product performs the following steps: encrypt the first test model, and / or encrypt the first test set.

[0165] In one embodiment, the privacy computing product includes a first product deployed in a first user and a second product deployed in a second user; the first test model is split into a first model and a second model, and the first test set is split into a first subset and a second subset; the first product is used to perform privacy computing using the first model and the first subset as input data; the second product is used to perform privacy computing using the second model and the second subset as input data; the second prediction result is obtained through the interaction of privacy data between the first product and the second product.

[0166] In one embodiment, the local end test module 520 includes: a first calculation submodule, a second calculation submodule and a prediction determination submodule (not shown in the figure). Among them, the first calculation submodule is configured to input the first subset of plaintext into a local first model, and perform a first plaintext calculation on the first subset through the first model; the second calculation submodule is configured to input the second subset of plaintext into a local second model, and perform a second plaintext calculation on the second subset through the second model; the prediction determination submodule is configured to determine the first prediction result through the plaintext data interaction between the first plaintext calculation and the second plaintext calculation.

[0167] In one embodiment, when the digital model uploading module uploads the first test model to the privacy computing product, it includes: uploading the first model file exported from the first test model under a first deep learning framework to the privacy computing product, so that the privacy computing product loads the first model file under its second deep learning framework to obtain the first test model; the first model file has a general format.

[0168] In one embodiment, the apparatus 500 further includes: a first modification module 610, a first upload module 620, a first acquisition module 630, a first comparison module 640, and a first determination module 650. The first modification module 610 is configured to modify the data in the first model file based on a preset first modification strategy to obtain a second model file that simulates an abnormal scenario; the first upload module 620 is configured to upload the second model file to the privacy computing product so that the privacy computing product executes: loading the second model file to obtain a second test model containing anomalies, and performing privacy computing using the second test model and the first test set as input data to obtain a third prediction result; the first acquisition module 630 is configured to obtain the third prediction result output by the privacy computing product; the first comparison module 640 is configured to compare the third prediction result with the second prediction result to obtain a second comparison result; and the first determination module 650 is configured to determine the second test result for the privacy computing part based on the second comparison result.

[0169] In one embodiment, the apparatus 500 further includes: a second modification module 560, a second acquisition module 570, a second comparison module 580, and a second determination module 590. The second modification module 560 is configured to perform exception processing on the data in the first test set based on a preset second modification strategy to obtain a second test set simulating an abnormal scenario; the second acquisition module 570 is configured to obtain a fourth prediction result output by the privacy computing product, the fourth prediction result being obtained by the privacy computing product using the first test model contained therein and the second test set as input data to perform privacy computing; the second comparison module 580 is configured to compare the fourth prediction result with the second prediction result to obtain a third comparison result; and the second determination module 590 is configured to determine a third test result for the privacy computing part based on the third comparison result.

[0170] In one embodiment, the first test model is any one of a plurality of test models; the plurality of test models include network models constructed under the following model architectures: a feedforward neural network, a convolutional neural network, a recurrent neural network, and a natural language processing model.

[0171] In one embodiment, the first test set is any one of a plurality of test sets; the data distribution of the samples in the plurality of test sets includes: uniform distribution, normal distribution or skewed distribution.

[0172] The above device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.

[0173] The present specification also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute Figure 1 , Figure 2-1 , Figure 2-2 , Figure 2-3 , Figure 3 and Figure 4 The method of any one of the above.

[0174] The embodiment of the present specification also provides a computing device, including a memory and a processor, wherein the memory stores an executable code, and when the processor executes the executable code, Figure 1 , Figure 2-1 , Figure 2-2 , Figure 2-3 , Figure 3 and Figure 4 The method of any one of the above.

[0175] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the storage medium and computing device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0176] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present invention may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0177] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the embodiments of the present invention in detail. It should be understood that the above description is only a specific implementation method of the embodiments of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A method for testing the privacy computing part of model reasoning in a privacy computing product, wherein the privacy computing product uses the network model it contains to perform privacy computing on input data; The method is performed by a test device and includes: Obtain a first test set in plain text, and obtain a first test model used for testing and not encrypted; Inputting the first test set in plain text into the first test model locally, and obtaining a first prediction result through plain text calculation of the first test model; Obtain a second prediction result output by the privacy computing product, where the second prediction result is obtained by performing privacy computing by the privacy computing product using the first test model contained therein and the first test set as input data; Comparing the first prediction result and the second prediction result to obtain a first comparison result; A first test result for the privacy computing part is determined based on the first comparison result.

2. The method according to claim 1, before obtaining the second prediction result output by the privacy computing product, further comprising: Upload the first test model to the privacy computing product, and input the first test set into the privacy computing product, so that the privacy computing product performs the following steps: encrypt the first test model, and / or encrypt the first test set.

3. According to the method described in claim 2, the privacy computing product includes a first product deployed in a first user and a second product deployed in a second user; the first test model is split into a first model and a second model, and the first test set is split into a first subset and a second subset; the first product is used to perform privacy computing using the first model and the first subset as input data; the second product is used to perform privacy computing using the second model and the second subset as input data; the second prediction result is obtained through the interaction of privacy data between the first product and the second product.

4. The method according to claim 3, wherein the step of inputting the first test set in plain text into the first local test model comprises: Inputting the first subset of plaintext into a local first model, and performing a first plaintext calculation on the first subset through the first model; Inputting the second subset of plaintext into a local second model, and performing a second plaintext calculation on the second subset through the second model; The first prediction result is determined through plaintext data interaction between the first plaintext calculation and the second plaintext calculation.

5. According to the method of claim 2, the step of uploading the first test model to the privacy computing product comprises: Upload the first model file exported from the first test model under the first deep learning framework to the privacy computing product, so that the privacy computing product loads the first model file under its second deep learning framework to obtain the first test model; the first model file has a general format.

6. The method according to claim 5, further comprising: Based on a preset first modification strategy, modify the data in the first model file to obtain a second model file that simulates an abnormal scenario; Uploading the second model file to the privacy computing product, so that the privacy computing product executes: loading the second model file to obtain a second test model containing anomalies, and performing privacy computing using the second test model and the first test set as input data to obtain a third prediction result; Obtaining the third prediction result output by the privacy computing product; Comparing the third prediction result with the second prediction result to obtain a second comparison result; A second test result for the privacy computing part is determined based on the second comparison result.

7. The method according to claim 1, further comprising: Based on a preset second modification strategy, perform exception processing on the data in the first test set to obtain a second test set that simulates an abnormal scenario; Obtain a fourth prediction result output by the privacy computing product, where the fourth prediction result is obtained by: the privacy computing product performs privacy computing using the first test model contained therein and the second test set as input data; Comparing the fourth prediction result with the second prediction result to obtain a third comparison result; A third test result for the privacy computing part is determined based on the third comparison result.

8. According to the method of claim 1, the first test model is any one of a plurality of test models; the plurality of test models include network models constructed under the following model architectures: a feedforward neural network, a convolutional neural network, a recurrent neural network, and a natural language processing model.

9. The method according to claim 1, wherein the first test set is any one of a plurality of test sets; and the data distribution of samples in the plurality of test sets comprises: Uniform, normal, or skewed distribution.

10. A device for testing the privacy computing part of model reasoning in a privacy computing product, wherein the privacy computing product uses the network model it contains to perform privacy computing on input data; The device is deployed in a test device and includes: A digital model acquisition module is configured to acquire a first test set in plain text and acquire a first test model for testing that has not been encrypted; A local test module is configured to input the first test set in plain text into the first test model locally, and obtain a first prediction result through plain text calculation of the first test model; A product-side testing module is configured to obtain a second prediction result output by the privacy computing product, where the second prediction result is obtained by performing privacy computing by the privacy computing product using the first test model contained therein and the first test set as input data; A prediction comparison module, configured to compare the first prediction result and the second prediction result to obtain a first comparison result; A result determination module is configured to determine a first test result for the privacy calculation part based on the first comparison result.

11. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 9.

12. A computing device comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 9 is implemented.

Images