Cross-industry risk control model verification method, apparatus and device, and storage medium

Abstract

The invention discloses a cross-industry risk control model verification method and device, equipment and a storage medium, and the method comprises the steps: carrying out the intermediate weight iterative calculation of an obtained updated model, generating an updated intermediate weight and a hyper-parameter, and obtaining a random sampling iteration round number combination; and finally, through updating the intermediate weight and hyper-parameter, generating proof data corresponding to iteration and training round number combination, so that the technical problems that the performance is reduced and the attack is easily damaged by the adaptive backdoor attack when the prior knowledge of the existing attack is used for detecting the opponent at present are solved.

Description

Technical Field

[0001] The present application relates to the field of risk control model technology, and in particular to a cross-industry risk control model verification method, device, equipment and storage medium. Background Art

[0002] Machine Learning as a Service (MLaaS) is an innovative service that has emerged in the context of cloud computing and has rapidly changed the way data is processed and decision-made in multiple industries in recent years. MLaaS provides enterprises and developers with a convenient way to take advantage of advanced machine learning algorithms and models without in-depth expertise or expensive hardware resources. In this service model, industry-leading cloud service providers, etc., greatly simplify the entire life cycle of machine learning by providing comprehensive capabilities, including data cleaning, model training, reasoning, and other machine learning-related tasks. The rise of MLaaS has injected new impetus into innovation in all walks of life, making it more convenient and feasible to apply machine learning technology in healthcare, finance, autonomous driving, and other key areas.

[0003] Currently, many cloud service providers offer MLaaS that covers multiple links such as data cleaning, training, and inference. This provides customers with the convenience of outsourcing computationally intensive model training to professional MLaaS third parties, avoiding the tedious task of managing local computing hardware and resources. However, as technology becomes more popular, the training integrity of MLaaS faces some challenges that cannot be ignored. Recent research has revealed the potential threat of backdoor attacks. Such attackers can manipulate training data or tamper with weights during model training, which has a negative impact on the performance of the model. Although the resulting model performs well in most cases, it may produce incorrect predictions under certain input conditions. Of particular concern is that if the MLaaS provider is attacked by a backdoor, it may have serious consequences for customers in key areas such as healthcare, autonomous driving, and robotics. Therefore, when choosing MLaaS services, customers must take necessary measures to ensure the integrity and reliability of model training in their own interests.

[0004] Therefore, in order to meet the common interests of MLaaS providers and customers, it is urgent to develop an MLaaS scheme that can provide both privacy protection for providers and verifiability for customers. In existing technologies, prior knowledge of existing attacks is currently used to detect performance degradation caused by adversaries, but this approach has no theoretical guarantee and is vulnerable to adaptive backdoor attacks.

[0005] Therefore, the current technical problems of using prior knowledge of existing attacks to detect performance degradation caused by adversaries and being vulnerable to adaptive backdoor attacks need to be solved urgently. Summary of the invention

[0006] The present application provides a cross-industry risk control model verification method, device, equipment and storage medium, which first generates updated intermediate weights and hyperparameters by iteratively calculating the intermediate weights of the acquired updated model, then obtains a randomly sampled combination of iteration rounds, and finally generates proof data corresponding to the combination of iteration and training rounds by updating the intermediate weights and hyperparameters. The method solves the technical problem that the current use of prior knowledge of existing attacks to detect opponents causes performance degradation and is vulnerable to damage from adaptive backdoor attacks.

[0007] In view of this, the first aspect of the present application provides a cross-industry risk control model verification method, the method comprising:

[0008] Perform iterative calculation on the intermediate weights of the acquired update model to generate updated intermediate weights and hyperparameters;

[0009] Get the combination of random sampling iteration rounds;

[0010] Proof data corresponding to the combination of iterations and training rounds is generated by updating intermediate weights and hyperparameters.

[0011] Optionally, performing iterative calculation of intermediate weights on the acquired update model to generate updated intermediate weights and hyperparameters specifically includes:

[0012] Get the updated model and the intermediate weights calculated by the previous iterative weights;

[0013] Generate updates to intermediate weights and hyperparameters via the submission function.

[0014] Optionally, the update of intermediate weights and hyperparameters generated by submitting a function specifically includes:

[0015] Generate hyperparameters by submitting a function;

[0016] Update the intermediate weights by calculating the number of layers corresponding to the knowledge distillation algorithm and the weights corresponding to the number of layers;

[0017] The final model is determined by updating the intermediate weights.

[0018] Optionally, the combination of the number of iteration rounds of random sampling obtained specifically includes:

[0019] Obtaining the combination of iteration round numbers generated by random sampling;

[0020] The number of iterations in the iteration round number combination and the corresponding specific data sample are determined.

[0021] Optionally, generating proof data corresponding to a combination of iterations and training rounds by updating intermediate weights and hyperparameters specifically includes:

[0022] The updated intermediate weights and hyperparameters are calculated through the called proof function, the obtained public key and the zero-knowledge proof algorithm;

[0023] Generate proof data corresponding to the combination of iteration rounds.

[0024] Optionally, after generating the proof data corresponding to the combination of the number of iteration rounds, the following steps are further included:

[0025] Sending the certification data to the client, so that the client verifies the certification data to determine whether it passes the verification;

[0026] A verification result corresponding to the certification data is obtained.

[0027] Optionally, before performing iterative calculation of intermediate weights on the acquired update model to generate updated intermediate weights and hyperparameters, the following steps are also included:

[0028] Establishing a verification loop corresponding to the client including a training proof and a commitment proof;

[0029] A public key corresponding to the update model is obtained.

[0030] A second aspect of the present application provides a cross-industry risk control model verification device, the device comprising:

[0031] A first acquisition unit is used to perform iterative calculation of intermediate weights on the acquired update model to generate updated intermediate weights and hyperparameters;

[0032] A second acquisition unit is used to acquire a combination of iterative rounds of random sampling;

[0033] The proof data generation unit is used to generate proof data corresponding to the combination of iteration and training round number by updating intermediate weights and hyperparameters.

[0034] Optionally, the first acquiring unit specifically includes:

[0035] A first acquisition subunit is used to acquire the updated model and the intermediate weights obtained by the previous iterative weight calculation;

[0036] The first generation subunit is used to generate and update intermediate weights and hyperparameters by submitting a function.

[0037] Optionally, the first generating subunit is specifically used for:

[0038] Generate hyperparameters by submitting a function;

[0039] Update the intermediate weights by calculating the number of layers corresponding to the knowledge distillation algorithm and the weights corresponding to the number of layers;

[0040] The final model is determined by updating the intermediate weights.

[0041] Optionally, the second acquisition unit is specifically configured to acquire an iteration round number combination including the number of iterations and the corresponding specific data sample generated by random sampling.

[0042] Optionally, the proof data generating unit specifically includes:

[0043] The proof data calculation subunit is used to calculate the updated intermediate weights and hyperparameters through the called proof function, the obtained public key and the zero-knowledge proof algorithm;

[0044] The second generating subunit is used to generate proof data corresponding to the combination of the number of iteration rounds.

[0045] Optionally, it also includes:

[0046] The certification data returning unit is used to send the certification data to the client so that the client can verify the certification data to determine whether it passes the verification.

[0047] Optionally, it also includes:

[0048] The public key acquisition unit is used to acquire the public key corresponding to the update model.

[0049] A third aspect of the present application provides a cross-industry risk control model verification device, the device comprising a processor and a memory:

[0050] The memory is used to store the program code and transmit the program code to the processor;

[0051] The processor is used to execute the steps of the method for cross-industry risk control model verification as described in the first aspect above according to the instructions in the program code.

[0052] A fourth aspect of the present application provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the method of the first aspect above.

[0053] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0054] In the present application, a cross-industry risk control model verification method, device, equipment and storage medium are provided. The intermediate weights and hyperparameters are updated by iteratively calculating the intermediate weights of the acquired update model, and then a randomly sampled combination of iteration rounds is obtained. Finally, the intermediate weights and hyperparameters are updated to generate proof data corresponding to the combination of iteration and training rounds. The technical problem that the current use of prior knowledge of existing attacks to detect opponents causes performance degradation and is vulnerable to damage from adaptive backdoor attacks is solved.

[0055] At the same time, the present application obtains an iteration round number combination including the number of iterations and the corresponding specific data sample generated by random sampling, and calculates the updated intermediate weights and hyperparameters by calling the proof function, obtaining the public key and the zero-knowledge proof algorithm, and generates proof data corresponding to the iteration round number combination, so that the client verifies the proof data to determine whether it passes the verification, which solves the technical problem that if the MLaaS provider generates a proof to prove its integrity in the calculation (i.e., model training), the MLaaS provider also wants to prevent the hyperparameters of their training scheme from being leaked to competitors, and meets the beneficial effect of the provider's need to protect private hyperparameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a method flow chart of the cross-industry risk control model verification method in the embodiment of this application;

[0057] Figure 2 This is a schematic diagram of the structure of a cross-industry risk control model verification device in an embodiment of the present application;

[0058] Figure 3 This is a schematic diagram of the structure of a cross-industry risk control model verification device in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] This application designs a cross-industry risk control model verification method, which first generates updated intermediate weights and hyperparameters by iteratively calculating the intermediate weights of the acquired updated model, then obtains a randomly sampled combination of iteration rounds, and finally generates proof data corresponding to the combination of iteration and training rounds by updating the intermediate weights and hyperparameters. It solves the current technical problem of using prior knowledge of existing attacks to detect performance degradation caused by opponents and being vulnerable to damage from adaptive backdoor attacks.

[0061] For easier understanding, see Figure 1 , Figure 1 Flow chart of the method for verifying the cross-industry risk control model in the embodiment of the present application. Figure 1 As shown, specifically:

[0062] 101. Perform iterative calculation of intermediate weights on the acquired update model to generate updated intermediate weights and hyperparameters;

[0063] The original scheme for generating training proofs in the current MLaaS is as follows: First, the client submits data D to the server. Then, the server uses data D to train model M and generates a zero-knowledge proof π to prove the integrity of model training, that is, M = Train (D, H). Next, the server sends M and π back to the client. Finally, the client verifies the integrity of model training based on proof π.

[0064] However, in the MLaaS scenario, this design is impractical because the generation of the training proof π incurs a huge overhead. According to the assessment of those skilled in the art, it may take days or even months to generate a proof for training a convolutional neural network (CNN).

[0065] In this embodiment, a two-round challenge-response protocol is used, in which a random sampling technique is used. The first round requires the server to submit the intermediate weights of all models to the client before proving the challenge. The second round requires the server to generate relevant proofs for the iterations and cycle parts randomly selected by the client. Before this protocol, the public key corresponding to the updated model needs to be obtained.

[0066] Furthermore, the client or a third party generates a key (pk, vk)←KeyGen(1λ, CM) and distributes pk to the server and vk to the client. If the network architecture of the model does not change, this pair of keys can be used multiple times.

[0067] It is understandable that there are many types of keys generated by the client or a third party. Type 1: There is no trusted third party or the key is generated by a specified client. In this case, the client generates the key;

[0068] Type 2: The key is generated by the server. Here the server can be understood as a trusted third party in the algorithm.

[0069] Type 3: The key is generated by other trusted third parties, such as the key management center provided by the entire system, which is responsible for generating the key as a third party.

[0070] Whether the network architecture has changed or not, the corresponding technical means can be:

[0071] 1. The model network architecture does not change: For common joint modeling tasks, the network structure remains unchanged, but the model parameters are continuously updated through multiple rounds of iterations;

[0072] 2. The model network architecture changes: For example, knowledge distillation or expert mixed MoE is performed during the joint modeling process, which will involve changes to the model structure in the joint model scenario. At this time, it is necessary to update the secret key when the model structure changes to ensure security.

[0073] 3. Knowledge distillation or expert mixture models are often used for domain specific and cross-industry models

[0074] Therefore, the intermediate weights of the obtained update model are iteratively calculated to generate updated intermediate weights and hyperparameters, specifically including:

[0075] Get the updated model and the intermediate weights calculated by the previous iterative weights;

[0076] Generate and update intermediate weights and hyperparameters by submitting functions;

[0077] Among them, the generation and update of intermediate weights and hyperparameters by submitting functions specifically include:

[0078] Generate hyperparameters by submitting a function;

[0079] Update the intermediate weights by calculating the number of layers corresponding to the knowledge distillation algorithm and the weights corresponding to the number of layers;

[0080] The final model is determined by updating the intermediate weights, where represents the final model M obtained by matrix multiplication, where WL is the weight matrix of the last layer of the model, and E is the input feature matrix or embedding matrix.

[0081] The Chinese expression is as follows:

[0082] M: The output of the final model.

[0083] WL: The weight matrix of the last layer of the model.

[0084] E: The feature vector or embedding matrix input to the model. In machine learning or deep learning, this formula is often used to describe how the final output of the model is obtained by multiplying the weight matrix and the input features. This can be the last step of the neural network to generate prediction results or perform tasks such as classification and regression.

[0085] Specifically, it can be understood that the server obtains a good model and intermediate weights W = {Wk|Wk←f(D,Wk-1,H)}, which means that the set W contains a series of weights Wk, each weight Wk is calculated by the function f according to the weight Wk-1 of the previous iteration, the data set D and the output H of the hidden layer. This process is usually used when training neural networks or other machine learning models to update weights to improve the performance of the model.

[0086] The server then commits the hyperparameters H^=Commit(H) and all intermediate weights W^={W^k|W^k=Commit(Wk), k=l·E+e} updated in iteration I.

[0087] The set W^ contains the weights W^k after some kind of "commit" operation. Each weight W^k is obtained by processing the original weight Wk through the commit function. The index k of the weight Wk is calculated by multiplying l (which may represent the number of layers or cycles) by E (which may represent the number of weights per layer or cycle) plus e (which may represent an additional offset or index).

[0088] In machine learning or deep learning, the "Commit" function may refer to an operation that updates weights to model parameters, or a specific weight update strategy.

[0089] 102. Obtain a combination of random sampling iteration rounds;

[0090] When the intermediate weights of the obtained updated model are iteratively calculated to generate updated intermediate weights and hyperparameters at the same time, or before or after, it is necessary to obtain an iteration round number combination including the number of iterations and the corresponding specific data samples generated by random sampling.

[0091] It is understandable that the client randomly selects some combinations of iterations and training rounds S = (l, e) for verification. Here, SS represents a combination, where l represents the number of iterations or training rounds, and e represents a specific parameter or evidence associated with the number of iterations l. In the training process of machine learning or deep learning, l can be the round in the training process, and e can be a specific data sample, error value or other related parameter used in the round. This combination may be used to describe a specific state or stage in the training process.

[0092] 103. Generate proof data corresponding to the combination of iterations and training rounds by updating intermediate weights and hyperparameters.

[0093] After obtaining the randomly sampled combination of iteration rounds, it is necessary to calculate the updated intermediate weights and hyperparameters through the called proof function, the obtained public key and the zero-knowledge proof algorithm to generate proof data corresponding to the combination of iteration rounds.

[0094] The client requests verification of S. The server then generates a proof U = {πl,e←Prove^(pk,z,a)|(l,e)∈S}, where z = (Dl,W^k,Wk-1,s^k) and a = (Wk,Wk-1,H,sk,r). It can be understood that the set UU contains a series of proof results πl and evidence e, which are calculated by calling the Prove^ function, using the public key pkpk, zero-knowledge proof z and additional information a. At the same time, only when the (l,e) pair exists in the set S, the corresponding πl and e will be included in U.

[0095] "Prove" may refer to a zero-knowledge proof process, where Prove^ represents some form of zero-knowledge proof function used to verify a statement or proof without revealing additional information. (l,e) represents an element pair in the set SS, associated with a specific proof or evidence.

[0096] In this embodiment, after the intermediate weights and hyperparameters are updated to generate the proof data corresponding to the combination of iterations and training rounds, the following steps are also included:

[0097] The certification data is sent to the client, so that the client verifies the certification data to determine whether it passes the verification.

[0098] The server sends U back to the client. The client sets z = (D^l, Wk, W^k-1, sk) and obtains the verification result b←Verify(vk, z, πl, e). If b is true for all π∈U, the server is verified.

[0099] The server sends the set U back to the client. Based on the received information, the client sets a data structure z, which contains the following elements: Dl (which may represent some data or state), Wk (the weight of the current round), W^k-1 (the weight of the previous round), and sk (which may be a signature or security token). Next, the client uses the verification function Verify to obtain the verification result b. This verification process involves the verification key vk, the data structure z, and the proof πl,eπl,e received from the server.

[0100] The output b of the verification function Verify is a Boolean value indicating whether the verification is successful. If the verification result b is true (i.e., the verification is successful) for all proofs π in the set U, the server has passed the verification.

[0101] This process ensures that the client can verify the correctness of the information provided by the server without revealing the specific proof details. This is an application example of zero-knowledge proof technology, which is often used to enhance system security and privacy protection.

[0102] In the verification function Verify(vk,z,πl,e), ‘π’ usually represents a proof. In the context of zero-knowledge proof, it is a special kind of evidence used to prove that a statement is true without revealing any information to the verifier except that the statement is true.

[0103] Specifically, the meanings of 'π' here may include:

[0104] 1. Proof: It is a piece of evidence generated by a prover to prove a fact or knowledge to a verifier without revealing any additional information.

[0105] 2. Zero knowledge: The prover can generate such a proof that the verifier can only know that the prover knows certain information, but cannot know the specific content of the information.

[0106] 3. Interactivity: In some cases, the proof process may be interactive, that is, multiple rounds of communication may be required between the prover and the verifier.

[0107] 4. Non-interaction: In other cases, the proof may be non-interactive, that is, the prover generates the proof once and the verifier can verify it independently after receiving it without further communication.

[0108] 5. Commitment: In some protocols, 'π' can be a commitment, which represents the prover's commitment to a certain value that can be revealed and verified in the future.

[0109] 6. Verification: 'π' needs to pass the verification function Verify to verify its validity. If the verification passes, it proves that the prover does know the information they claim to know.

[0110] 7. The parameter 'πl,e' may represent a proof specific to a certain iteration l and evidence e, which indicates that the proof may depend on specific states or data during the iteration.

[0111] In cryptography and security protocols, the specific implementation and meaning of 'π' may vary depending on the specific protocol and application scenario.

[0112] It should be noted that in this embodiment, in order to generate proof π during the training process, we need to establish a loop CM. This includes two sub-proofs of two main processes, namely, the training proof and the commitment proof. The training proof is about the process of updating the weights of an epoch, while the commitment proof is about the process of committing to intermediate weights and hyperparameters. Figure 2 As shown, the detailed structure of CM is expressed in the form of NP statement 1.

[0113] The specific NP statement 1 is as follows.

[0114] (1) Given the number of training rounds e, iteration l, commitments H^, W^k, W^k-1 and sk, and data set Dl, the correct random number r, secret key sk, and intermediate weights Wk and Wk-1 for training should be as follows:

[0115] (2) Training proof: Wk = f(Dl, Wk-1, H), Wk: represents the weight in the kkth iteration.

[0116] Dl: represents the lth data point or data batch.

[0117] Wk-1: represents the weight in the k-1th iteration, that is, the weight of the previous round.

[0118] H: usually represents the activation value or feature representation of the hidden layer.

[0119] f: represents a function used to update the weight according to the current data point Dl, the weight Wk-1 of the previous round, and the activation value H of the hidden layer. The meaning of the whole formula is that during the training process, each iteration will use a specific function f to calculate and update the weight Wkk according to the current data point Dl, the weight Wk-1 of the previous round, and the activation value H of the hidden layer. This process is carried out automatically, and the purpose is to minimize the loss function of the model by continuously adjusting the weights, thereby improving the prediction or classification performance of the model.

[0120] In deep learning, f is usually a composite function that includes forward propagation and back propagation, which involves the calculation of the loss function and the update of the gradient. This process is the core part of training neural networks.

[0121] (3) Proof of commitment: H^=Commit(H,sk), W^k=Commit(Wk,sk) and W^k-1=Commit(Wk-1,sk) and sk=COMM(sk,r).

[0122] It involves a commitment scheme in cryptography, which is a cryptographic primitive that allows one participant (the committer) to commit a value to another participant (the verifier) ​​without revealing it immediately. Afterwards, the committer can reveal the value at any time, and the verifier can verify that the value is consistent with the original commitment.

[0123] The Chinese expression is as follows:

[0124] 1. H^=Commit(H,sk)

[0125] οH^: Commitment to the hidden layer activation value H.

[0126] οH: Activation value of the hidden layer.

[0127] οsk: A random number or secret value used to ensure the security of the commitment.

[0128] 2.W^k=Commit(Wk,sk)

[0129] οW^k: Commitment to weight Wk at the kth iteration.

[0130] οWk: weight of the kth iteration.

[0131] 3.W^k-1=Commit(Wk-1,sk)

[0132] οW^k-1: Commitment to weight Wk-1 at the k-1th iteration.

[0133] 4.sk=COMM(sk,r)

[0134] οsk: A generated security value used for commitment.

[0135] οr: Another random number or secret value that may be used to enhance the security of sksk or for other purposes.

[0136] ο COMM: A possible function or procedure that generates or processes the security value sk.

[0137] These formulas describe a commitment process commonly seen in cryptography or secure multi-party computation, where the Commit function is used to create a commitment to a value, and the COMM function may be used to generate or process a secure value associated with the commitment. The key to the commitment scheme is that once a commitment is created, it should be immutable, even the committer cannot change the content of the commitment until they choose to reveal it. At the same time, the verifier can verify the consistency of the commitment, ensuring that the revealed value matches the original commitment.

[0138] In this embodiment, a cross-industry risk control model verification method is provided. First, the intermediate weights and hyperparameters are updated by iteratively calculating the intermediate weights of the obtained updated model, and then a randomly sampled combination of iteration rounds is obtained. Finally, the intermediate weights and hyperparameters are updated to generate proof data corresponding to the combination of iteration and training rounds. This solves the technical problem that the current use of prior knowledge of existing attacks to detect adversaries causes performance degradation and is vulnerable to damage from adaptive backdoor attacks.

[0139] At the same time, a random sampling strategy is adopted to improve the efficiency of proof generation and cope with the challenge of backdoor attacks. In order to successfully implement a backdoor attack, the damaged data samples and the number of training iterations need to reach a certain ratio. Drawing on the low-cost sampling QAP equation, after receiving the server model, the client randomly samples (l, e) pairs and challenges the server for proof to reduce the traversal check time cost of the entire model training. To prevent the leakage of intermediate weights and hyperparameters, the joint modeling trusted technology requires the server to submit all intermediate weights and hyperparameters H to the client before conducting a proof challenge.

[0140] At the same time, in this embodiment, an iteration round number combination including the number of iterations and the corresponding specific data sample is obtained by random sampling, and the updated intermediate weights and hyperparameters are calculated by calling the proof function, the obtained public key and the zero-knowledge proof algorithm, and the proof data corresponding to the iteration round number combination is generated, so that the client verifies the proof data to determine whether it passes the verification, which solves the technical problem that if the MLaaS provider generates a proof to prove its integrity in the calculation (i.e., model training), the MLaaS provider also wants to prevent the hyperparameters of their training scheme from being leaked to competitors, and meets the beneficial effect of the provider's need to protect private hyperparameters.

[0141] In this embodiment, zero-knowledge proof, zero-trust modeling, two-round challenge-response protocol and random sampling technology are combined to take into account the common interests of MLaaS providers and customers. This design realizes the verification of the model training process. Customers can randomly select some iterations and batches for verification and can ask the server to provide corresponding proofs to verify the integrity of the training process. In addition, this solution also meets the provider's need to protect private hyperparameters, while greatly reducing the time cost required for training verification generation.

[0142] See also Figure 2 , Figure 2 Schematic diagram of the structure of the cross-industry risk control model verification device in the embodiment of the present application. Figure 2 As shown, specifically:

[0143] The first acquisition unit 201 is used to perform iterative calculation of intermediate weights on the acquired update model to generate updated intermediate weights and hyperparameters;

[0144] A second acquisition unit 202 is used to acquire a combination of iterative rounds of random sampling;

[0145] The proof data generating unit 203 is used to generate proof data corresponding to the combination of iteration and training round number by updating intermediate weights and hyper parameters.

[0146] Optionally, the first acquiring unit specifically includes:

[0147] A first acquisition subunit is used to acquire the updated model and the intermediate weights obtained by the previous iterative weight calculation;

[0148] The first generation subunit is used to generate and update intermediate weights and hyperparameters by submitting a function.

[0149] Optionally, the first generating subunit is specifically used for:

[0150] Generate hyperparameters by submitting a function;

[0151] Update the intermediate weights by calculating the number of layers corresponding to the knowledge distillation algorithm and the weights corresponding to the number of layers;

[0152] The final model is determined by updating the intermediate weights.

[0153] Optionally, the second acquisition unit is specifically configured to acquire an iteration round number combination including the number of iterations and the corresponding specific data sample generated by random sampling.

[0154] Optionally, the proof data generating unit specifically includes:

[0155] The proof data calculation subunit is used to calculate the updated intermediate weights and hyperparameters through the called proof function, the obtained public key and the zero-knowledge proof algorithm;

[0156] The second generating subunit is used to generate proof data corresponding to the combination of the number of iteration rounds.

[0157] Optionally, it also includes:

[0158] The certification data returning unit is used to send the certification data to the client so that the client can verify the certification data to determine whether it passes the verification.

[0159] Optionally, it also includes:

[0160] The public key acquisition unit is used to acquire the public key corresponding to the update model.

[0161] In this embodiment, a cross-industry risk control model verification device is provided. The intermediate weights and hyperparameters are updated by iteratively calculating the intermediate weights of the acquired updated model, and then a randomly sampled combination of iteration rounds is obtained. Finally, the intermediate weights and hyperparameters are updated to generate proof data corresponding to the combination of iteration and training rounds. This solves the technical problem that the current use of prior knowledge of existing attacks to detect opponents causes performance degradation and is vulnerable to damage from adaptive backdoor attacks.

[0162] At the same time, a random sampling strategy is adopted to improve the efficiency of proof generation and cope with the challenge of backdoor attacks. In order to successfully implement a backdoor attack, the damaged data samples and the number of training iterations need to reach a certain ratio. Drawing on the low-cost sampling QAP equation, after receiving the server model, the client randomly samples (l, e) pairs and challenges the server for proof to reduce the traversal check time cost of the entire model training. To prevent the leakage of intermediate weights and hyperparameters, the joint modeling trusted technology requires the server to submit all intermediate weights and hyperparameters H to the client before conducting a proof challenge.

[0163] At the same time, in this embodiment, an iteration round number combination including the number of iterations and the corresponding specific data sample is obtained by random sampling, and the updated intermediate weights and hyperparameters are calculated by calling the proof function, the obtained public key and the zero-knowledge proof algorithm, and the proof data corresponding to the iteration round number combination is generated, so that the client verifies the proof data to determine whether it passes the verification, which solves the technical problem that if the MLaaS provider generates a proof to prove its integrity in the calculation (i.e., model training), the MLaaS provider also wants to prevent the hyperparameters of their training scheme from being leaked to competitors, and meets the beneficial effect of the provider's need to protect private hyperparameters.

[0164] In this embodiment, zero-knowledge proof, zero-trust modeling, two-round challenge-response protocol and random sampling technology are combined to take into account the common interests of MLaaS providers and customers. This design realizes the verification of the model training process. Customers can randomly select some iterations and batches for verification and can ask the server to provide corresponding proofs to verify the integrity of the training process. In addition, this solution also meets the provider's need to protect private hyperparameters, while greatly reducing the time cost required for training verification generation.

[0165] The present application embodiment also provides another cross-industry risk control model verification device, such as Figure 3 For the sake of convenience, only the parts related to the embodiments of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The terminal can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (English full name: Personal Digital Assistant, English abbreviation: PDA), a sales terminal (English full name: Point of Sales, English abbreviation: POS), a car computer, etc., taking the terminal as a mobile phone as an example:

[0166] Figure 3 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal provided in an embodiment of the present application. Figure 3 The mobile phone includes: a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090. Those skilled in the art can understand that Figure 3 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0167] Combine the following Figure 3 A detailed introduction to the various components of the mobile phone:

[0168] The RF circuit 1010 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 1080 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 1010 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (full name: Low Noise Amplifier, English abbreviation: LNA), a duplexer, etc. In addition, the RF circuit 1010 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communications can use any communication standard or protocol, including but not limited to Global System of Mobile communications (Global System of Mobile communication, English abbreviation: GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, English abbreviation: CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (Long Term Evolution, English abbreviation: LTE), e-mail, Short Messaging Service (SMS), etc.

[0169] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1020. The memory 1020 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 1020 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0170] The input unit 1030 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1031 or near the touch panel 1031 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 1080, and can receive and execute commands sent by the processor 1080. In addition, the touch panel 1031 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1031, the input unit 1030 may further include other input devices 1032. Specifically, the other input devices 1032 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0171] The display unit 1040 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1040 may include a display panel 1041. Optionally, the display panel 1041 may be configured in the form of a liquid crystal display (full name in English: Liquid Crystal Display, English abbreviation: LCD), an organic light-emitting diode (full name in English: Organic Light-Emitting Diode, English abbreviation: OLED), etc. Further, the touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it is transmitted to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides a corresponding visual output on the display panel 1041 according to the type of touch event. Although in Figure 3 In the embodiment, the touch panel 1031 and the display panel 1041 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.

[0172] The mobile phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1041 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1041 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0173] The audio circuit 1060, the speaker 1061, and the microphone 1062 can provide an audio interface between the user and the mobile phone. The audio circuit 1060 can transmit the received audio data to the speaker 1061 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 1062 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1060 and converted into audio data, and then the audio data is output to the processor 1080 for processing, and then sent to another mobile phone through the RF circuit 1010, or the audio data is output to the memory 1020 for further processing.

[0174] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 1070. It provides users with wireless broadband Internet access. Figure 3 A WiFi module 1070 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0175] The processor 1080 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 1020 and calling data stored in the memory 1020, it executes various functions of the mobile phone and processes data, thereby monitoring the mobile phone as a whole. Optionally, the processor 1080 may include one or more processing units; preferably, the processor 1080 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1080.

[0176] The mobile phone also includes a power supply 1090 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 1080 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0177] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0178] In the embodiment of the present application, the processor 1080 included in the terminal also has the following functions:

[0179] Perform iterative calculation on the intermediate weights of the acquired update model to generate updated intermediate weights and hyperparameters;

[0180] Get the combination of random sampling iteration rounds;

[0181] Proof data corresponding to the combination of iterations and training rounds is generated by updating intermediate weights and hyperparameters.

[0182] An embodiment of the present application also provides a computer-readable storage medium for storing program code, which is used to execute any one of the implementation methods of a cross-industry risk control model verification method described in the aforementioned embodiments.

[0183] In the embodiments of the present application, a cross-industry risk control model verification method, apparatus, device and storage medium are provided. The intermediate weights and hyperparameters are updated by iteratively calculating the intermediate weights of the acquired updated model, and then a randomly sampled combination of iteration rounds is obtained. Finally, the intermediate weights and hyperparameters are updated to generate proof data corresponding to the combination of iteration and training rounds. This solves the technical problem that the current use of prior knowledge of existing attacks to detect opponents causes performance degradation and is vulnerable to damage from adaptive backdoor attacks.

[0184] At the same time, a random sampling strategy is adopted to improve the efficiency of proof generation and cope with the challenge of backdoor attacks. In order to successfully implement a backdoor attack, the damaged data samples and the number of training iterations need to reach a certain ratio. Drawing on the low-cost sampling QAP equation, after receiving the server model, the client randomly samples (l, e) pairs and challenges the server for proof to reduce the traversal check time cost of the entire model training. To prevent the leakage of intermediate weights and hyperparameters, the joint modeling trusted technology requires the server to submit all intermediate weights and hyperparameters H to the client before conducting a proof challenge.

[0185] At the same time, in this embodiment, an iteration round number combination including the number of iterations and the corresponding specific data sample is obtained by random sampling, and the updated intermediate weights and hyperparameters are calculated by calling the proof function, the obtained public key and the zero-knowledge proof algorithm, and the proof data corresponding to the iteration round number combination is generated, so that the client verifies the proof data to determine whether it passes the verification, which solves the technical problem that if the MLaaS provider generates a proof to prove its integrity in the calculation (i.e., model training), the MLaaS provider also wants to prevent the hyperparameters of their training scheme from being leaked to competitors, and meets the beneficial effect of the provider's need to protect private hyperparameters.

[0186] In this embodiment, zero-knowledge proof, zero-trust modeling, two-round challenge-response protocol and random sampling technology are combined to take into account the common interests of MLaaS providers and customers. This design realizes the verification of the model training process. Customers can randomly select some iterations and batches for verification and can ask the server to provide corresponding proofs to verify the integrity of the training process. In addition, this solution also meets the provider's need to protect private hyperparameters, while greatly reducing the time cost required for training verification generation.

[0187] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0188] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0189] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0190] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0191] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0192] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0193] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (full name in English: Read-Only Memory, English abbreviation: ROM), random access memory (full name in English: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program codes.

[0194] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A cross-industry risk control model verification method, characterized in that: include: Perform iterative calculation on the intermediate weights of the acquired update model to generate updated intermediate weights and hyperparameters; Get the combination of random sampling iteration rounds; Proof data corresponding to the combination of the iteration and the number of training rounds is generated by updating the intermediate weights and the hyperparameters.

2. The cross-industry risk control model verification method according to claim 1 is characterized in that: The intermediate weights of the obtained update model are iteratively calculated to generate updated intermediate weights and hyperparameters, specifically including: Get the updated model and the intermediate weights calculated by the previous iterative weights; The updated intermediate weights and the hyperparameters are generated by a submission function.

3. The cross-industry risk control model verification method according to claim 2 is characterized in that: Generating the updated intermediate weights and the hyperparameters by submitting a function specifically includes: Generate the hyperparameters by submitting a function; Calculate the updated intermediate weight by using the number of layers corresponding to the knowledge distillation algorithm and the weight corresponding to the number of layers; The final model is determined according to the updated intermediate weights.

4. The cross-industry risk control model verification method according to claim 1 is characterized in that: The combinations of random sampling iteration rounds are as follows: Obtaining the combination of iteration round numbers generated by random sampling; The number of iterations in the iteration round number combination and the corresponding specific data sample are determined.

5. The cross-industry risk control model verification method according to claim 4 is characterized in that: Generating the certification data corresponding to the combination of the iteration and the number of training rounds by updating the intermediate weights and the hyperparameters specifically includes: Calculating the updated intermediate weight and the hyperparameter by using the called proof function, the obtained public key and the zero-knowledge proof algorithm; Generate proof data corresponding to the combination of the number of iteration rounds.

6. The cross-industry risk control model verification method according to claim 5 is characterized in that: After generating the proof data corresponding to the combination of the number of iterations, the method further includes: Sending the certification data to the client, so that the client verifies the certification data to determine whether it passes the verification; A verification result corresponding to the certification data is obtained.

7. The cross-industry risk control model verification method according to claim 5, characterized in that: Before the intermediate weights of the acquired update model are iteratively calculated to generate updated intermediate weights and hyperparameters, the following steps are also included: Establishing a verification loop corresponding to the client including a training proof and a commitment proof; A public key corresponding to the update model is obtained.

8. A cross-industry risk control model verification device, characterized in that: include: A first acquisition unit is used to perform iterative calculation of intermediate weights on the acquired update model to generate updated intermediate weights and hyperparameters; A second acquisition unit is used to acquire a combination of iterative rounds of random sampling; A proof data generating unit is used to generate proof data corresponding to the combination of the iteration and the number of training rounds by updating the intermediate weights and the hyperparameters.

9. A cross-industry risk control model verification device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the cross-industry risk control model verification method described in any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and the program code is used to execute the cross-industry risk control model verification method described in any one of claims 1-7.

Images