A model training system, a model training method, and a business execution method

Abstract

The specification discloses a model training system, a model training method and a business execution method. The model training system comprises a first training end and a second training end, and a to-be-trained model comprises a first sub-model and a second sub-model. The first sub-model is deployed in a trusted computing domain provided by the first training end, and the second sub-model is deployed in a trusted computing domain provided by the second training end. The first training end sends intermediate features output by the first sub-model to the second training end. The first sub-model is updated in parameters according to received gradient information. The second training end inputs the received intermediate features into the second sub-model to obtain a prediction result. A loss value is determined according to the prediction result and label data, and the loss value is fed back to the second sub-model. Gradient information for the first sub-model is determined and sent to the first training end. The second sub-model is updated in parameters according to the loss value. The scheme improves the model training efficiency while effectively ensuring data security in the model training process.

Description

Technical Field

[0001] This specification relates to one or more embodiments in the field of computer technology, and in particular to a model training system, a model training method, and a business execution method. Background Technology

[0002] With the rapid development of artificial intelligence technology, neural network models are being widely applied in various fields such as image recognition, natural language processing, information recommendation, and risk control. Model training is a crucial step in ensuring model performance. During model training, a large amount of private data is often involved, including model parameters and sample data. Failure to protect this data can lead to data breaches and potential losses for users.

[0003] However, current training methods often fail to balance training efficiency and data security. For example, while distributed learning can accelerate training by using distributed computing clusters, multiple nodes can actually increase the risk of data leakage, thus compromising data privacy and security.

[0004] Therefore, how to improve model training efficiency while ensuring data security during the training process is an urgent problem to be solved. Summary of the Invention

[0005] This specification provides a model training system, a model training method, and a business execution method to partially solve the problems existing in the prior art.

[0006] This specification provides a model training system, which includes at least one first training terminal and a second training terminal. The model to be trained includes a first sub-model and a second sub-model. The first sub-model is deployed in a first trusted computing domain provided by the first training terminal, and the second sub-model is deployed in a second trusted computing domain provided by the second training terminal.

[0007] The at least one first training end is used to: encrypt the intermediate features output by the first sub-model during training in a first trusted computing domain, transmit the obtained encrypted intermediate features from the first trusted computing domain, and send them to the second training end; and to: transmit the received encrypted gradient data into the first trusted computing domain, decrypt the encrypted gradient data in the first trusted computing domain, and update the parameters of the first sub-model according to the gradient information obtained after decryption.

[0008] The second training end is used to: input the received encrypted intermediate features into the second trusted computing domain; decrypt the encrypted intermediate features in the second trusted computing domain; and determine the loss value of the model to be trained based on the prediction result output by the second sub-model based on the decrypted intermediate features; encrypt the gradient information for the intermediate features calculated based on the loss value in the second trusted computing domain; transmit the obtained encrypted gradient data from the second trusted computing domain and send it to the first training end; and update the parameters of the second sub-model based on the loss value.

[0009] This specification provides a model training method, which is applied to a first training endpoint. A first sub-model of the model to be trained is deployed in a first trusted computing domain provided by the first training endpoint. The method includes:

[0010] In the first trusted computing domain, the intermediate features output by the first sub-model during training are encrypted to obtain encrypted intermediate features;

[0011] The encrypted intermediate feature is transmitted from the first trusted computing domain and sent to the second training end, so that the second training end inputs the received encrypted intermediate feature into the second trusted computing domain provided by the second training end. In the second trusted computing domain, the encrypted intermediate feature is decrypted, and the loss value of the model to be trained is determined based on the prediction result output by the second sub-model deployed in the second trusted computing domain based on the decrypted intermediate feature. The gradient information for the intermediate feature calculated based on the loss value is encrypted in the second trusted computing domain, and the obtained encrypted gradient data is transmitted from the second trusted computing domain and sent to the first training end. And, the parameters of the second sub-model are updated according to the loss value.

[0012] The received encrypted gradient data is passed into the first trusted computing domain, the encrypted gradient data is decrypted in the first trusted computing domain, and the parameters of the first sub-model are updated based on the gradient information obtained after decryption.

[0013] This specification provides a model training method applied to a second training endpoint, wherein a second sub-model of the model to be trained is deployed in a second trusted computing domain provided by the second training endpoint, and the method includes:

[0014] The received encrypted intermediate feature is passed to the second trusted computing domain, and the encrypted intermediate feature is decrypted in the second trusted computing domain; wherein, the encrypted intermediate feature is the intermediate feature output by the first sub-model deployed in the first trusted computing domain during the training process, which is encrypted by the first training end in the first trusted computing domain provided by the first training end, and then transmitted from the first trusted computing domain to the second training end;

[0015] Based on the prediction results output by the second sub-model using the intermediate features obtained through decryption, the loss value of the model to be trained is determined.

[0016] In the second trusted computing domain, the gradient information for the intermediate feature calculated based on the loss value is encrypted. The obtained encrypted gradient data is transmitted from the second trusted computing domain and sent to the first training end, so that the first training end can input the received encrypted gradient data into the first trusted computing domain, decrypt the encrypted gradient data in the first trusted computing domain, and update the parameters of the first sub-model according to the decrypted gradient information; and update the parameters of the second sub-model according to the loss value.

[0017] This specification provides a business execution method, including:

[0018] Obtain business data;

[0019] The business data is input into a pre-trained business model to determine a prediction result for the business data, and the business is executed based on the prediction result; the business model is trained using the model training method described above.

[0020] This specification provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described method.

[0021] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0022] This specification provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the methods described above.

[0023] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects:

[0024] In this embodiment of the specification, the model training system includes: a first training terminal and a second training terminal. The model to be trained includes: a first sub-model and a second sub-model. The first sub-model is deployed in a trusted computing domain provided by the first training terminal, and the second sub-model is deployed in a trusted computing domain provided by the second training terminal. The first training terminal sends the intermediate features output by the first sub-model to the second training terminal. The first sub-model is trained based on the received gradient information. The second training terminal inputs the received intermediate features into the second sub-model to obtain a prediction result. A loss value is determined based on the prediction result and label data, and the loss value is fed back to the second sub-model. Gradient information for the first sub-model is determined and sent to the first training terminal. The second sub-model is trained based on the loss value.

[0025] As can be seen from the above embodiments, this specification divides the business model into two parts: an upper-layer model and a lower-layer model, and deploys them respectively in the trusted computing domains provided by the second training end and the first training end. By training each part of the model through multiple training ends, the computing resources provided by each training end can be effectively utilized, thereby improving the training efficiency of the model. Furthermore, the entire training process of the model will be executed in the trusted computing domain. The data encryption and decryption mechanism provided by the trusted computing domain protects the interactive data between the first training end and the second training end. Even if the data of one of the holding ends is leaked by others, the leaker will still not be able to obtain all the model data. While improving the model training efficiency, it also strengthens the protection of data privacy during the model training process and further avoids data security risks during the model training process. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall architecture of a model training system provided in this specification;

[0027] Figure 2 This specification provides a schematic diagram of a model training process based on a model training system.

[0028] Figure 3(a) is a schematic diagram of the splitting process of model training provided in this specification;

[0029] Figure 3(b) is a schematic diagram of the splitting process of model training provided in this specification;

[0030] Figure 4 This is a schematic diagram of the data transmission architecture of a model training system provided in this specification.

[0031] Figure 5 This is a schematic diagram of the device interaction process in a first training terminal provided in this specification.

[0032] Figure 6This is a schematic diagram of the device interaction process in a first training terminal provided in this specification.

[0033] Figure 7 This is a flowchart illustrating a model training method applied to the first training end as provided in this specification.

[0034] Figure 8 This is a flowchart illustrating a model training method applied to the second training end as provided in this specification.

[0035] Figure 9 This is a flowchart illustrating one of the business execution methods provided in this specification;

[0036] Figure 10 This is a schematic structural diagram of a device provided in this specification;

[0037] Figure 11 This is a schematic diagram of a model training device provided in this specification;

[0038] Figure 12 This is a schematic diagram of a model training device provided in this specification. Detailed Implementation

[0039] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of related data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0040] Split learning, as an efficient model training method, allows a model to be divided into multiple parts without sharing the original data. Each part is trained independently on different training endpoints, exchanging only the weights or outputs of intermediate split layers, thus ensuring data security. In this process, the model is typically split into a bottom-level model and a top-level model, deployed on separate training endpoints. One endpoint uses local data to compute the output of the bottom-level model and transmits it to the other endpoint. The other endpoint then computes the top-level model, obtaining the prediction result or loss value, and performs backpropagation to update the model. This method not only improves training efficiency but also ensures data security.

[0041] A Trusted Domain (TD) is a security mechanism built into a computer system. It aims to isolate the data from the main operating system and other applications through software and hardware technologies, thereby providing a secure environment for data computation and transmission. In practice, TDs are typically built based on technologies such as Trusted Execution Environments (TEEs). This involves using the isolation and encryption protection mechanisms provided by TEEs to create a secure TD domain. Its unique privacy protection mechanisms ensure that data is encrypted when leaving the domain and is only decrypted upon entering the domain, thus effectively guaranteeing data security.

[0042] Based on this, this specification provides a model training system that splits the model into two parts, a first sub-model and a second sub-model, and deploys them in trusted computing domains provided by different training endpoints. The system trains the model based on split learning and TEE technology, thereby improving model training efficiency while fully ensuring data security during the model training process.

[0043] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0044] Figure 1 This is a schematic diagram of the overall architecture of a model training system provided in this specification, such as... Figure 1 As shown.

[0045] The model training system includes at least one first training terminal and one second training terminal. The first training terminal has a first trusted computing domain, and the second training terminal has a second trusted computing domain. Before training the model, the second training terminal can divide the model to be trained into two parts, a first sub-model and a second sub-model, within the trusted computing domain. The first sub-model is then sent to the first trusted computing domain for deployment, and the second sub-model is deployed within the second trusted computing domain.

[0046] In this process, after the model to be trained is split, each model can correspond to a splitting layer (i.e., a first splitting layer and a second splitting layer). Before the model to be trained is split, the first splitting layer and the second splitting layer are connected, and the features output by the first splitting layer are input into the second splitting layer. The intermediate features output by the first splitting layer, after being encrypted (i.e., encrypted intermediate features), are transmitted from the first trusted computing domain and sent to the second trusted computing domain. The gradient information output by the second splitting layer, after being encrypted (i.e., encrypted gradient data), is transmitted from the second trusted computing domain and sent to the first trusted computing domain.

[0047] It should be noted that when there are multiple first training ends, each first training end can further split the first sub-model into several third sub-models. The intermediate features obtained by each third sub-model are passed sequentially from the first sub-models of each first training end to the second training end.

[0048] The first sub-model can be a model structure consisting of network layers from the input layer to the first split layer of the model to be trained, and the second sub-model can be a model structure consisting of network layers from the second split layer to the output layer of the model.

[0049] In practical applications, the model to be trained can be a business model used to perform different business operations, including: risk control model, information recommendation model, image recognition model, and natural language processing model. In addition, for different business models, there are also various types of input sample data, including: image data, text data, and audio data, etc. This specification does not make specific limitations on this.

[0050] Furthermore, this specification provides a schematic diagram of the model training process based on the above model training system, such as... Figure 2 As shown.

[0051] Figure 2 This specification provides a schematic diagram of a model training process based on a model training system, which includes the following steps:

[0052] S200: Encrypt the intermediate features output by the first sub-model during training in the first trusted computing domain.

[0053] When training the model to be trained, the first training end can input sample data into the first sub-model to obtain the intermediate features output by the first sub-model.

[0054] In this process, sample data is first input into the input layer of the first sub-model, and then intermediate features are output by the last network layer (i.e., the first split layer) of the first sub-model. Specifically, the intermediate features output by the first sub-model can be derived from the last network layer (i.e., the first split layer) of the first sub-model.

[0055] S202: The obtained encrypted intermediate features are transmitted from the first trusted computing domain and sent to the second training end.

[0056] After determining the intermediate feature, the first training end can encrypt the intermediate feature in the first trusted computing domain to obtain the encrypted intermediate feature. Then, the encrypted intermediate feature is transmitted out of the first trusted computing domain for data export and sent to the second training end.

[0057] S204: Pass the received encrypted intermediate feature into the second trusted computing domain, and decrypt the encrypted intermediate feature in the second trusted computing domain;

[0058] S206: Determine the loss value of the model to be trained based on the prediction result output by the second sub-model based on the intermediate features obtained by decryption;

[0059] S208: Encrypt the gradient information for the intermediate feature calculated based on the loss value in the second trusted computing domain.

[0060] After receiving the encrypted intermediate feature, the second training end can pass it into the second trusted computing domain and decrypt the encrypted intermediate feature in the second trusted computing domain to obtain the intermediate feature.

[0061] In this process, the intermediate features are first input into the first network layer (i.e., the second split layer) of the second sub-model, and the prediction result is output by the final output layer.

[0062] The output results will vary depending on the sample data and the business model. For example, for a risk control model, the prediction result may include whether there is a security risk or risk control strategy, while for an image recognition model, the prediction result may be the classification information of the image.

[0063] After the second training end determines the prediction result, it can determine the loss value of the model to be trained based on the deviation between the prediction result and the label data of the sample data.

[0064] In this specification, sample data can be pre-stored in a first trusted computing domain, and label data can be pre-stored in a second trusted computing domain. In this way, during model training, the first training end can directly obtain sample data from the local first trusted computing domain, and the second training end can directly obtain label data from the local second trusted computing domain. However, it is necessary to ensure that the sample data and label data used in each round of training correspond to each other.

[0065] After determining the loss value, the second training end can feed the loss value back to the second sub-model, and then use the second sub-model to backpropagate the gradient information to determine the gradient information for the first sub-model.

[0066] During this process, the second training end can update the parameters of the second sub-model based on the backpropagation of the loss value. When determining the gradient information for the first sub-model, the second training end first feeds the loss value back to the output layer of the second sub-model, performs backpropagation through its output layer, and outputs the gradient information propagated to the next network layer (i.e., the first split layer) as the gradient information for the first sub-model.

[0067] After determining the gradient information, the second training end can encrypt the gradient information in the second trusted computing domain to obtain encrypted gradient data.

[0068] S210: The obtained encrypted gradient data is transmitted from the second trusted computing domain and sent to the first training end.

[0069] After obtaining the encrypted gradient data, the second training end can output the encrypted gradient data into the second trusted computation domain for out-of-domain processing and send it to the first training end.

[0070] S212: The received encrypted gradient data is passed into the first trusted computing domain, and the encrypted gradient data is decrypted in the first trusted computing domain;

[0071] S214: Update the parameters of the first sub-model based on the gradient information obtained after decryption;

[0072] S216: Update the parameters of the second sub-model based on the loss value.

[0073] After receiving the encrypted gradient data, the first training end can pass it into the first trusted computation domain and decrypt it there to obtain the gradient information. Then, the first training end can feed the gradient information back to the first sub-model. Starting from the first split layer of the first sub-model, the gradient information is backpropagated, ending at the input layer of the first sub-model, thereby updating the parameters of the first sub-model.

[0074] Furthermore, during the transmission of encrypted gradient data and encrypted intermediate features, the first and second training ends can each send a remote verification request for the other's trusted computing domain to the verification end, ensuring that the other party is a trusted device before transmitting data. That is, the first training end can send the encrypted intermediate features to the second training end after confirming that the second trusted computing domain has passed trusted verification, and the second training end can send the encrypted gradient data to the first training end after confirming that the first trusted computing domain has passed trusted verification.

[0075] Specifically, before sending the encrypted intermediate features to the second training end, the first training end can send a remote proof request to the second trusted computing domain through the first trusted computing domain. The second trusted computing domain can send the verification information of the second trusted computing domain to the verification end based on the remote proof request sent by the first trusted computing domain, so that the verification end can determine the verification result for the second trusted computing domain based on the verification information of the second trusted computing domain, and send the verification result for the second trusted computing domain to the first trusted computing domain.

[0076] If the first training end determines that the second trusted computing domain has passed the trusted verification based on the verification result for the second trusted computing domain, then the encrypted intermediate feature can be sent to the verification end.

[0077] In practical applications, the second trusted computing domain can calculate the hash value of its code data and encrypt it using a pre-programmed private key to obtain signature data. The code data and signature data are then sent to the verification end as verification information.

[0078] The verification end can perform a hash operation on the received code data to obtain a reference hash value, and decrypt the signature data using the local public key. If the hash value obtained after decryption matches the reference hash value, the first trusted computing domain is authenticated. If the signature data cannot be decrypted using the local public key, or if the hash value obtained after decryption does not match the reference hash value, the first trusted computing domain fails to be authenticated.

[0079] Similarly, before sending encrypted gradient data to the first training end, the second training end can send a remote proof request to the first trusted computing domain through the second trusted computing domain. The first trusted computing domain can send the verification information of the first trusted computing domain to the verification end based on the remote proof request sent by the second trusted computing domain, so that the verification end can determine the verification result for the first trusted computing domain based on the verification information of the first trusted computing domain, and send the verification result for the first trusted computing domain to the second trusted computing domain.

[0080] If the second training end determines that the first trusted computing domain has passed the trusted verification based on the verification result for the first trusted computing domain, then the encrypted gradient data can be sent to the second training end.

[0081] The first trusted computing domain can calculate the hash value of its code data and encrypt it using a pre-burned private key to obtain signature data. It then sends the signature data and code data as verification information to the verification end.

[0082] The verification end can perform a hash operation on the received code data to obtain a reference hash value, and decrypt the signature data using the local public key. If the hash value obtained after decryption matches the reference hash value, the second trusted computing domain is authenticated. If the signature data cannot be decrypted using the local public key, or if the hash value obtained after decryption does not match the reference hash value, the second trusted computing domain fails to be authenticated.

[0083] In addition, when receiving data, the first and second trusted computing domains can also send a verification request for the data source to the verification end in the above manner, so as to encrypt and decrypt the data in the trusted computing domain if the data source is confirmed to have passed trusted verification.

[0084] The first training end can transmit the encrypted gradient data to the first trusted computing domain if it is determined that the data source of the encrypted gradient data has been verified as trusted. The data source of the encrypted gradient data may include the second trusted computing domain and / or the second training end.

[0085] The second training end can transmit the encrypted intermediate feature to the second trusted computing domain if it is determined that the data source of the encrypted intermediate feature has been verified as trusted. The data source of the encrypted intermediate feature may include the first trusted computing domain and / or the first training end.

[0086] The first and second training ends can train the first and second sub-models through multiple rounds of interaction with intermediate features and gradient information until the training conditions are met. The first training end can then send the trained first sub-model to the second training end so that the second training end can combine the trained first sub-model and the trained second sub-model into a trained model.

[0087] The training conditions mentioned above may include reaching a preset number of training iterations or the convergence of the first and second sub-models to a preset range, etc., which are not specifically limited in this specification. For ease of understanding, this specification provides a schematic diagram of the model training splitting process, as shown in Figure 3(a).

[0088] In this process, the second training end (A end) splits the model to be trained (Modle S) into a first sub-model (Modle B) and a second sub-model (Modle A). The first training end (B end) inputs the sample data into the first sub-model to obtain its intermediate output features (Embedding B), and sends them to the second training end (A end). The A end can determine the prediction result (Predict A) based on Embedding B and label data, and train Modle A based on the prediction result to obtain gradient information for Modle B.

[0089] Once the training conditions are met, the B-end sends the trained Model B to the A-end, which then merges Model B and Model A to obtain the trained Model S.

[0090] In this process, it is necessary to ensure that the training rounds are aligned in each round of interaction between end A and end B, and that the sample data in end B matches the label data in end A.

[0091] Of course, the first and second sub-models that have completed training in this specification can also be deployed on the first and second training ends respectively, and the subsequent model inference tasks can be performed through the first and second training ends. In this process, the first sub-model does not need to send the completed first sub-model to the second training end.

[0092] In addition, in practical applications, a third-party server can also be used to complete the splitting and merging process of the model. For ease of understanding, this manual provides a schematic diagram of the model splitting and training process, as shown in Figure 3(b).

[0093] In this process, the server (S-end) splits the model to be trained (Modle S) into a first sub-model (Modle B) and a second sub-model (Modle A). The first training end (B-end) inputs the sample data into the first sub-model to obtain its intermediate output features (Embedding B), which are then sent to the second training end (A-end) through the server. The A-end can determine the prediction result (Predict A) based on Embedding B and label data. Based on this prediction result, the A-end is trained to obtain gradient information for the B-end, which is then sent to the B-end through the server.

[0094] Once the training conditions are met, B and A can send the trained Model B and Model A to the server, respectively. The server then merges Model B and Model A to obtain the trained Model S.

[0095] In this process, it is necessary to ensure that the training rounds are aligned in each round of interaction between end A and end B, and that the sample data in end B matches the label data in end A.

[0096] During the transmission of feature data and gradient information, the first trusted computing domain and the second trusted computing domain respectively maintain mutually negotiated symmetric keys for symmetric encryption and decryption of intermediate features and gradient information.

[0097] Alternatively, the first and second trusted computing domains maintain their respective private keys and the other party's public key, which are used for asymmetric encryption and decryption or envelope-based encryption and decryption of intermediate features and gradient information.

[0098] Specifically, in the asymmetric encryption and decryption process, the first training end can use its private key to encrypt the intermediate feature in the first trusted computing domain to obtain the encrypted intermediate feature. Then, the encrypted intermediate feature is transmitted out of the first trusted computing domain and sent to the second training end.

[0099] The second training end can pass the encrypted intermediate feature sent by the first training end into the second trusted computing domain, and decrypt the encrypted intermediate feature in the second trusted computing domain using the corresponding public key to obtain the intermediate feature.

[0100] In addition, the second training end can use a private key to encrypt the gradient information in the second trusted computing domain to obtain encrypted gradient data, and then transmit the encrypted gradient data from the second trusted computing domain to the first training end.

[0101] The first training end can pass the encrypted gradient data sent by the second training end into the first trusted computing domain, and decrypt the encrypted gradient data in the first trusted computing domain using the corresponding public key to obtain the gradient information.

[0102] In the envelope-based encryption and decryption process, the first training end can encrypt the intermediate feature in the first trusted computing domain using a symmetric key to obtain an encrypted intermediate feature. Simultaneously, it encrypts the symmetric key using a private key to obtain an encryption key. Then, the first training end can transmit the encrypted envelope, consisting of the encryption key and the encrypted intermediate feature, from the first trusted computing domain to the second training end.

[0103] The second training end can pass the encrypted envelope into the second trusted computing domain, and use the public key to decrypt the encryption key in the second trusted computing domain to obtain the symmetric key. Then, the symmetric key can be used to decrypt the encrypted intermediate feature to obtain the intermediate feature.

[0104] Alternatively, the second training end can also encrypt the gradient information in the second trusted computing domain using another symmetric key to obtain encrypted gradient data; then encrypt the symmetric key using a private key to obtain an encryption key, and finally transmit the encrypted envelope consisting of the encryption key and the encrypted gradient data from the second trusted computing domain to the first training end.

[0105] The first training end can pass the encrypted envelope into the first trusted computing domain, and use the corresponding public key to decrypt the second encryption key in the first trusted computing domain to obtain the symmetric key; then use the symmetric key to decrypt the encrypted gradient data to obtain the gradient information.

[0106] To further improve model training efficiency and data security during training, multiple training devices can be deployed in both the first and second training ends. These devices each provide a corresponding trusted computing domain. The first and second training ends can train the first and second sub-models respectively through their respective training device clusters, thereby improving training efficiency while further ensuring data security during training. For ease of understanding, this specification provides a schematic diagram of the data transmission architecture of a model training system, as shown below. Figure 4 As shown.

[0107] Figure 4 This is a schematic diagram of the data transmission architecture of a model training system provided in this specification.

[0108] The first training end may include: a first scheduling device and at least two associated first computing devices; the first trusted computing domain includes: a first type of sub-trusted computing domain provided by the first scheduling device and a second type of sub-trusted computing domain provided by each of the second computing devices, and a first sub-model is deployed in each of the second type of sub-trusted computing domains; similarly, the second training end may include: a second scheduling device and at least two associated second computing devices; the second trusted computing domain includes: a third type of sub-trusted computing domain provided by the second scheduling device and a fourth type of sub-trusted computing domain provided by each of the second computing devices, and a second sub-model is deployed in each of the third type of sub-trusted computing domains.

[0109] Each training device, including computing and scheduling devices, may include a guest operating system (Guest OS) and a host operating system (Host OS), with the trusted computing domain provided by each device isolated from the host OS.

[0110] In addition, the trusted computing domain of the scheduling device can run a Ray Head process, which schedules the computing tasks during model training. The trusted computing domain of the computing device can also run a Ray Work process, which executes the computing tasks during model training.

[0111] For each first computing device in the first training end, the first computing device can input its local sample data into the first sub-model in its corresponding second type of trusted computing domain, encrypt the branch features output by the model in the second type of trusted computing domain, transmit the obtained encrypted branch features out of the second type of trusted computing domain, and send them to the associated first scheduling device.

[0112] The first scheduling device can input the received encrypted branch features into the first type of trusted computing domain, where they are decrypted into the corresponding branch features. In this case, the second training end can determine the branch prediction results output by the second sub-model for each branch feature, select the corresponding label data according to the sample data used during training for each branch feature, determine the branch loss value corresponding to each branch prediction result, and further calculate the branch gradient information for each branch feature; and perform comprehensive calculation on all the calculated branch gradient information to obtain the gradient information for the intermediate feature.

[0113] It should be noted that the intermediate features mentioned above are a set that includes all branch features.

[0114] For ease of understanding, this specification provides a schematic diagram of the device interaction process in the first training terminal, such as... Figure 5 As shown.

[0115] Figure 5 This is a schematic diagram of the device interaction process in a first training terminal provided in this specification, including the following steps:

[0116] S500: Input the sample data from the computing device into the first sub-model deployed in the second type of trusted computing domain to obtain branch features;

[0117] S502: Encrypt the branch features to obtain encrypted branch features;

[0118] S504: Transmit the encrypted branch feature from the second type of sub-trusted computing domain and send it to the scheduling device.

[0119] Specifically, for each computing device in the first training end, the computing device can input the sample data in the computing device into the first sub-model deployed in the second type of sub-trusted computing domain to obtain branch features. Then, the branch features are encrypted to obtain encrypted branch features. Subsequently, the encrypted branch features are output to the second type of sub-trusted computing domain and sent to the scheduling device.

[0120] S506: The encrypted branch features sent by each computing device are passed into the first type of sub-trusted computing domain, and the encrypted branch features are decrypted in the first type of sub-trusted computing domain to obtain the branch features;

[0121] S508: Determine intermediate features based on the features of each branch.

[0122] The scheduling device can pass the encrypted branch features sent by each computing device into the first type of sub-trusted computing domain, and decrypt each encrypted branch feature in the first type of sub-trusted computing domain to obtain each branch feature, and then determine the intermediate feature based on each branch feature.

[0123] In this context, the first type of sub-trusted computing domain and the second type of sub-trusted computing domain can each maintain: their respective private keys and the other party's public key, for use in asymmetric encryption / decryption or envelope-based encryption / decryption of branch features; or,

[0124] In both the first and second sub-trusted computing domains, mutually negotiated symmetric keys can be maintained for symmetric encryption and decryption of branch features.

[0125] For details on asymmetric encryption / decryption or envelope-based encryption / decryption methods, please refer to the above text; this manual will not elaborate further on these methods.

[0126] In addition, the first type of sub-trusted computing domain and the second type of sub-trusted computing domain can constitute a first sub-trusted computing domain cluster, and each sub-trusted computing domain in the first sub-trusted computing domain cluster maintains a unified symmetric key for symmetric encryption and decryption of branch features. This unified symmetric key can be generated and distributed by the main trusted computing domain in the first sub-trusted computing domain cluster.

[0127] For each second computing device in the second training end, the second computing device can input the branch features sent to it by the second scheduling device into the second sub-model in its corresponding fourth type of sub-trusted computing domain. Then, based on the branch prediction results output by the second sub-model and the label data corresponding to the received branch features, the branch loss value is determined, and the branch gradient information for the received branch features is further calculated. The branch gradient information is encrypted in the fourth type of sub-trusted computing domain, and the encrypted branch gradient data is transmitted out of the fourth type of sub-trusted computing domain and sent to the associated scheduling device.

[0128] The second scheduling device can transmit the received encrypted branch gradient data to the third type of sub-trusted computing domain, decrypt it into the corresponding branch gradient information in the third type of sub-trusted computing domain, and then perform joint calculation on all branch gradient information (such as calculating the mean) to obtain gradient information for the intermediate features.

[0129] In this scenario, the first training end can update the parameters of the first sub-model in the second type of sub-trusted computing domain corresponding to each first computing device based on the gradient information for the intermediate features.

[0130] For ease of understanding, this manual provides a schematic diagram of the device interaction process in the training terminal, such as... Figure 6 As shown.

[0131] Figure 6 This is a schematic diagram of the device interaction process in a first training terminal provided in this specification, including the following steps:

[0132] S600: Input the received encrypted intermediate features into the second sub-model deployed in the fourth type of trusted computing domain to obtain the branch prediction result;

[0133] S602: Determine the branch gradient information based on the branch prediction results and the local label data of the computing device;

[0134] S604: Encrypt the branch gradient information to obtain encrypted branch gradient data;

[0135] S606: Transmit the encrypted branch gradient data from the fourth type of sub-trusted computing domain and send it to the scheduling device.

[0136] Specifically, for each computing device in the second training end, the computing device inputs the received encrypted intermediate features into the second sub-model deployed in the fourth type of trusted computing domain to obtain the branch prediction result. Based on the branch prediction result and the local label data of the computing device, the branch loss value is determined, and then the branch loss value is fed back to the local second sub-model to determine the branch gradient information. After that, the branch gradient information is encrypted to obtain encrypted branch gradient data, and then the encrypted branch gradient data is transmitted out from the fourth type of trusted computing domain and sent to the scheduling device.

[0137] S608: Transmit the encrypted branch gradient data sent by each computing device into the third type of trusted computing domain, and decrypt each encrypted branch gradient data in the third type of trusted computing domain to obtain the gradient information of each branch;

[0138] S610: Determine the gradient information for the first sub-model based on the gradient information of each branch.

[0139] The third and fourth sub-trusted computing domains can each maintain their own private key and the other's public key, for use in asymmetric encryption / decryption or envelope-based encryption / decryption of branch gradient information; or...

[0140] The third and fourth sub-trusted computing domains can each maintain mutually negotiated symmetric keys for symmetric encryption and decryption of branch gradient information.

[0141] For details on specific asymmetric encryption / decryption methods or envelope-based encryption / decryption methods, please refer to the above text. This manual will not elaborate further on these methods.

[0142] In addition, the third type of sub-trusted computing domain and the fourth type of sub-trusted computing domain can also form a third sub-trusted computing domain cluster, and each sub-trusted computing domain in the third sub-trusted computing domain cluster maintains a unified symmetric key for symmetric encryption and decryption of branch gradient information. This unified symmetric key can be generated and distributed by the main trusted computing domain in the third sub-trusted computing domain cluster.

[0143] It should be noted that in both the first and second training ends, before the scheduling device sends data to the computing device or the computing device sends data to the scheduling device, it can also send a remote proof to the trusted computing domain of the other party in advance, and send data to the other party only after confirming that the other party is trustworthy.

[0144] For each training endpoint, all computing devices can remotely verify whether the scheduling device is operating in a trusted environment. Similarly, the scheduling device can repeat a similar process to verify that the computing devices are also operating in a trusted environment. When two computing devices need to communicate with each other, they can also verify each other's trusted environment based on remote verification. Thus, all devices involved within an organization can mutually trust each other to operate in trusted environments.

[0145] For different training endpoints, the scheduling devices within their respective clusters can verify in advance whether the other is running in a trusted environment. During this process, a communication key can be negotiated. Subsequently, when computing devices in each trusted computing domain communicate with computing devices in the other's trusted computing domain using the negotiated key, they can directly use the key for communication. At this point, it is no longer necessary to verify whether the other party is running in a trusted device through remote authentication.

[0146] In practical applications, the role of remote authentication can include: ensuring that the other party's physical computer environment is trustworthy, that the operating system running in the trustworthy environment meets the requirements (such as a specific version), that the software version running in the trustworthy environment meets the requirements (such as a specific version), and that the data has not been tampered with.

[0147] In addition, each computing device in the second training end may hold different labels. In this case, for each computing device in the second training end, the intermediate feature sent to that computing device may be a branch feature corresponding to the sample data corresponding to the label held by that computing device.

[0148] Furthermore, in practical applications, only one training device can be set up in both the first and second training ends. In this case, only the intermediate features and loss values ​​need to be encrypted using the private and public keys. Since only one training device is set up in each of the first and second training ends, there is no data transmission or encryption process involved within the first and second training ends.

[0149] The above is a description of the model training system provided in this manual from the perspective of the overall system. For ease of understanding, the following will explain the model training method provided in this manual from the perspectives of the first training end and the second training end, respectively. Figure 7 or Figure 8 As shown.

[0150] Figure 7 This is a flowchart illustrating a model training method applied to the first training end provided in this specification, including the following steps:

[0151] S700: Encrypt the intermediate features output by the first sub-model during training in the first trusted computing domain to obtain encrypted intermediate features;

[0152] S702: The encrypted intermediate feature is transmitted from the first trusted computing domain and sent to the second training end, so that the second training end inputs the received encrypted intermediate feature into the second trusted computing domain provided by the second training end, decrypts the encrypted intermediate feature in the second trusted computing domain, and determines the loss value of the model to be trained based on the prediction result output by the second sub-model deployed in the second trusted computing domain based on the decrypted intermediate feature; the gradient information for the intermediate feature calculated based on the loss value is encrypted in the second trusted computing domain, the obtained encrypted gradient data is transmitted from the second trusted computing domain and sent to the first training end; and the parameters of the second sub-model are updated according to the loss value.

[0153] S704: The received encrypted gradient data is passed into the first trusted computing domain, the encrypted gradient data is decrypted in the first trusted computing domain, and the parameters of the first sub-model are updated according to the gradient information obtained after decryption.

[0154] Figure 8 This is a flowchart illustrating a model training method applied to the second training end provided in this specification, including the following steps:

[0155] S800: The received encrypted intermediate feature is passed to the second trusted computing domain, and the encrypted intermediate feature is decrypted in the second trusted computing domain; wherein, the encrypted intermediate feature is the intermediate feature output by the first sub-model deployed in the first trusted computing domain during the training process, which is encrypted by the first training end in the first trusted computing domain provided by the first training end, and then transmitted from the first trusted computing domain to the second training end;

[0156] S802: Determine the loss value of the model to be trained based on the prediction result output by the second sub-model based on the intermediate features obtained by decryption;

[0157] S804: In the second trusted computing domain, the gradient information for the intermediate feature calculated based on the loss value is encrypted, the obtained encrypted gradient data is transmitted out of the second trusted computing domain and sent to the first training end, so that the first training end transmits the received encrypted gradient data into the first trusted computing domain, decrypts the encrypted gradient data in the first trusted computing domain, and updates the parameters of the first sub-model according to the gradient information obtained after decryption; and updates the parameters of the second sub-model according to the loss value.

[0158] Furthermore, this specification also provides a business execution method applied to the above-mentioned trained model, such as... Figure 9 As shown.

[0159] Figure 9 This is a flowchart illustrating one business execution method provided in this specification, including the following steps:

[0160] S900, to acquire business data;

[0161] S902, the business data is input into a pre-trained business model to determine a prediction result for the business data through the business model, and the business is executed according to the prediction result.

[0162] In the actual execution of business operations, the server can use the obtained business model to train the business model trained by the aforementioned model training system and model training method, thereby determining the prediction result for the business data through the business model, and executing the business based on the prediction result.

[0163] The business model can be a risk control model, and the business data can be image data, text data, and audio data in the risk control scenario. After the business data is input into the risk control model, the risk control model can output a prediction result. The prediction result can include a risk control strategy for the current business. Then, risk control business for the current business can be executed based on the risk control strategy output by the risk control model.

[0164] Of course, the above business model can also be other models such as image recognition model, natural language processing model, information recommendation model, etc. For example, when the business model is an image recognition model in the autonomous driving scenario, the image recognition model can determine the type of obstacles around the target driving device and the distance between it and the target driving device, and then perform navigation tasks for the target driving device based on this information.

[0165] Figure 10 This is a schematic structural diagram of a device provided in this manual. Please refer to it. Figure 10At the hardware level, the device includes a processor 1002, an internal bus 1004, a network interface 1006, memory 1008, and non-volatile memory 1010, and may also include other hardware required for its functions. One or more embodiments of this specification can be implemented in software, for example, the processor 1002 reads the corresponding computer program from the non-volatile memory 1010 into memory 1008 and then runs it. Of course, besides software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution entity of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0166] Please refer to Figure 11 or Figure 12 A model training device can be applied to, for example Figure 10 The device shown is used to implement the technical solution described in this specification.

[0167] Figure 11 This is a schematic diagram of a model training device provided in this specification. The model training device may include:

[0168] The encryption module 1100 is used to encrypt the intermediate features output by the first sub-model during training in the first trusted computing domain to obtain encrypted intermediate features.

[0169] The sending module 1100 is configured to transmit the encrypted intermediate feature from the first trusted computing domain and send it to the second training end, so that the second training end can input the received encrypted intermediate feature into the second trusted computing domain provided by the second training end, decrypt the encrypted intermediate feature in the second trusted computing domain, and determine the loss value of the model to be trained based on the prediction result output by the second sub-model deployed in the second trusted computing domain based on the decrypted intermediate feature; encrypt the gradient information for the intermediate feature calculated based on the loss value in the second trusted computing domain, transmit the obtained encrypted gradient data from the second trusted computing domain and send it to the first training end; and update the parameters of the second sub-model according to the loss value.

[0170] The decryption module 1102 is used to pass the received encrypted gradient data into the first trusted computing domain, decrypt the encrypted gradient data in the first trusted computing domain, and update the parameters of the first sub-model based on the gradient information obtained after decryption.

[0171] Figure 12 This is a schematic diagram of a model training device provided in this specification. The model training device may include:

[0172] The receiving module 1200 is used to transmit the received encrypted intermediate feature to the second trusted computing domain and decrypt the encrypted intermediate feature in the second trusted computing domain; wherein, the encrypted intermediate feature is the intermediate feature output by the first sub-model deployed in the first trusted computing domain during the training process, which is encrypted by the first training end in the first trusted computing domain provided by the first training end, and then transmitted from the first trusted computing domain to the second training end;

[0173] The determination module 1202 is used to determine the loss value of the model to be trained based on the prediction result output by the second sub-model based on the intermediate features obtained by decryption.

[0174] The update module 1204 is used to encrypt the gradient information for the intermediate feature calculated based on the loss value in the second trusted computing domain, transmit the obtained encrypted gradient data from the second trusted computing domain and send it to the first training end, so that the first training end can input the received encrypted gradient data into the first trusted computing domain, decrypt the encrypted gradient data in the first trusted computing domain, and update the parameters of the first sub-model according to the gradient information obtained after decryption; and update the parameters of the second sub-model according to the loss value.

[0175] Based on the same concept as the methods described above, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described in any of the above embodiments by executing the executable instructions.

[0176] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0177] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

Claims

1. A model training system, the model training system comprising: The at least one first training end and the second training end, the model to be trained comprises: a first sub-model and a second sub-model, the first sub-model is deployed in a first trusted computing domain provided by the first training end, and the second sub-model is deployed in a second trusted computing domain provided by the second training end; The at least one first training end is configured to: encrypt intermediate features output by the first sub-model in a training process in the first trusted computing domain, transmit the obtained encrypted intermediate features out of the first trusted computing domain, and send the encrypted intermediate features to the second training end; and transmit received encrypted gradient data into the first trusted computing domain, decrypt the encrypted gradient data in the first trusted computing domain, and update parameters of the first sub-model according to gradient information obtained after decryption; The second training end is configured to: transmit the received encrypted intermediate features into the second trusted computing domain, decrypt the encrypted intermediate features in the second trusted computing domain, and determine a loss value of the model to be trained according to a prediction result output by the second sub-model based on the intermediate features obtained after decryption; encrypt gradient information for the intermediate features calculated based on the loss value in the second trusted computing domain, transmit the obtained encrypted gradient data out of the second trusted computing domain, and send the encrypted gradient data to the first training end; and update parameters of the second sub-model according to the loss value.

2. The model training system of claim 1, The first trusted computing domain and the second trusted computing domain respectively maintain mutually negotiated symmetric keys for symmetric encryption and decryption of the intermediate features and the gradient information; or The first trusted computing domain and the second trusted computing domain respectively maintain: respective private keys and public keys of the other party, for asymmetric encryption and decryption or envelope-based encryption and decryption of the intermediate features and the gradient information.

3. The model training system of claim 1, The first training end is specifically configured to: in a case where it is determined that the second trusted computing domain has passed trusted verification, send the encrypted intermediate features to the second training end; The second training end is specifically configured to: in a case where it is determined that the first trusted computing domain has passed trusted verification, send the encrypted gradient data to the first training end.

4. The model training system of claim 1, The first training end includes: A first scheduling device and at least two first computing devices associated with the first scheduling device; The first trusted computing domain comprises: a first type of sub-trusted computing domain provided by the first scheduling device and second type of sub-trusted computing domains respectively provided by the second computing devices; and each second type of sub-trusted computing domain has the first sub-model deployed therein; Each first computing device is configured to: input sample data in the first computing device into the first sub-model in a second type of sub-trusted computing domain corresponding to the first computing device, encrypt branch features output by the model in the second type of sub-trusted computing domain, transmit the obtained encrypted branch features out of the second type of sub-trusted computing domain, and send the encrypted branch features to the associated first scheduling device. The first scheduling device is configured to: transmit the received encrypted branch feature into the first type of sub-trusted computing domain, and decrypt the branch feature into the corresponding branch feature in the first type of sub-trusted computing domain; wherein the intermediate feature is a set of all branch features; The second training end is specifically configured to: determine branch prediction results respectively output by the second sub-model for each branch feature; select corresponding label data according to sample data used by each branch feature during training, to respectively determine branch loss values corresponding to each branch prediction result, and further calculate branch gradient information for each branch feature; and comprehensively calculate all the calculated branch gradient information to obtain gradient information for the intermediate feature.

5. The model training system of claim 4, The first type of sub-trusted computing domain and the second type of sub-trusted computing domain each maintain: a private key thereof and a public key of the other, to perform asymmetric encryption and decryption or envelope-based encryption and decryption on the branch feature; or, The first type of sub-trusted computing domain and the second type of sub-trusted computing domain each maintain a symmetric key negotiated with the other, to perform symmetric encryption and decryption on the branch feature; or, The first type of sub-trusted computing domain and the second type of sub-trusted computing domain constitute a first sub-trusted computing domain cluster, and each sub-trusted computing domain in the first sub-trusted computing domain cluster maintains a unified symmetric key for performing symmetric encryption and decryption on the branch feature, which is generated and distributed by a master trusted computing domain in the first sub-trusted computing domain cluster.

6. The model training system of claim 4, The second training end includes: A second scheduling device and at least two second computing devices associated with the second scheduling device; The second trusted computing domain includes: third type of sub-trusted computing domains provided by the second scheduling device and fourth type of sub-trusted computing domains respectively provided by the second computing devices; wherein the second sub-model is deployed in each third type of sub-trusted computing domain; Each second computing device is configured to: input a branch feature sent by the second scheduling device to the second computing device into a second sub-model in a fourth type of sub-trusted computing domain corresponding to the second computing device, determine a branch loss value according to a branch prediction result output by the second sub-model and label data corresponding to the received branch feature, and further calculate branch gradient information for the received branch feature; encrypt the branch gradient information in the fourth type of sub-trusted computing domain, transmit the obtained encrypted branch gradient data out of the fourth type of sub-trusted computing domain, and send the encrypted branch gradient data to the associated scheduling device; The second scheduling device is configured to: transmit the received encrypted branch gradient data into the third type of sub-trusted computing domain, and decrypt the branch gradient data into corresponding branch gradient information in the third type of sub-trusted computing domain; and comprehensively calculate all the branch gradient information to obtain gradient information for the intermediate feature; The first training end is specifically configured to: perform parameter updating on the first sub-model in the second type of sub-trusted computing domain corresponding to each first computing device according to the gradient information for the intermediate feature.

7. The model training system of claim 6, The third type of sub-trusted computing domain and the fourth type of sub-trusted computing domain respectively maintain: respective private keys and public keys of the other party, for asymmetric encryption and decryption or envelope-based encryption and decryption of the branch gradient information; or, The third type of sub-trusted computing domain and the fourth type of sub-trusted computing domain respectively maintain mutually negotiated symmetric keys, for symmetric encryption and decryption of the branch gradient information; or, The third type of sub-trusted computing domain and the fourth type of sub-trusted computing domain constitute a third sub-trusted computing domain cluster, and each sub-trusted computing domain in the third sub-trusted computing domain cluster maintains a unified symmetric key for symmetric encryption and decryption of the branch gradient information, which is generated and distributed by a master trusted computing domain in the third sub-trusted computing domain cluster.

8. The model training system of claim 1, The first training end is configured to: in a case where it is determined that a data source of the encrypted gradient data has passed trusted verification, transmit the encrypted gradient data into the first trusted computing domain; The second training end is specifically configured to: in a case where it is determined that a data source of the encrypted intermediate feature has passed trusted verification, transmit the encrypted intermediate feature into the second trusted computing domain.

9. A model training method, applied to a first training end, a first sub-model of a to-be-trained model being deployed in a first trusted computing domain provided by the first training end, the method comprising: encrypting, in the first trusted computing domain, an intermediate feature output by the first sub-model in a training process, to obtain encrypted intermediate feature; transmitting the encrypted intermediate feature out of the first trusted computing domain and sending it to a second training end, so that the second training end transmits the received encrypted intermediate feature into a second trusted computing domain provided by the second training end, decrypts the encrypted intermediate feature in the second trusted computing domain, and determines a loss value of the to-be-trained model according to a prediction result output by a second sub-model deployed in the second trusted computing domain based on the intermediate feature decrypted; encrypting, in the second trusted computing domain, gradient information for the intermediate feature calculated based on the loss value, transmitting the obtained encrypted gradient data out of the second trusted computing domain, and sending it to the first training end; and, updating parameters of the second sub-model according to the loss value; transmitting the received encrypted gradient data into the first trusted computing domain, decrypting the encrypted gradient data in the first trusted computing domain, and updating parameters of the first sub-model according to the gradient information obtained after decryption.

10. The method of claim 9, wherein, The first training end comprises: a first scheduling device and at least two first computing devices associated therewith; the first trusted computing domain comprises: a first type of sub-trusted computing domain provided by the first scheduling device and a second type of sub-trusted computing domain respectively provided by each second computing device; wherein each second type of sub-trusted computing domain has the first sub-model deployed therein; Before encrypting, in the first trusted computing domain, an intermediate feature output by the first sub-model in a training process, the method further comprises: For each first computing device, input sample data in the first computing device into a first sub-model in a second type of sub-trusted computing domain corresponding to the first computing device, encrypt branch features output by the model within the second type of sub-trusted computing domain, and send the obtained encrypted branch features out of the second type of sub-trusted computing domain to an associated first scheduling device; input the encrypted branch features received by the first scheduling device into the first type of sub-trusted computing domain, decrypt the corresponding branch features in the first type of sub-trusted computing domain, and determine the intermediate features according to each branch feature.

11. A model training method, the method is applied to a second training end, a second sub-model of a to-be-trained model is deployed in a second trusted computing domain provided by the second training end, and the method comprises: input the received encrypted intermediate features into the second trusted computing domain and decrypt the encrypted intermediate features in the second trusted computing domain; wherein the encrypted intermediate features are intermediate features output by a first sub-model deployed in a first trusted computing domain provided by a first training end during a training process, which are encrypted by the first training end in the first trusted computing domain, sent out of the first trusted computing domain, and sent to the second training end; determine a loss value of the to-be-trained model according to a prediction result output by the second sub-model based on the decrypted intermediate features; encrypt gradient information for the intermediate features calculated based on the loss value in the second trusted computing domain, send the obtained encrypted gradient data out of the second trusted computing domain, and send the encrypted gradient data to the first training end, so that the first training end inputs the received encrypted gradient data into the first trusted computing domain, decrypts the encrypted gradient data in the first trusted computing domain, and updates parameters of the first sub-model according to the gradient information obtained after decryption; and updates parameters of the second sub-model according to the loss value.

12. The method of claim 11, wherein, The second training end comprises a second scheduling device and at least two associated second computing devices; the second trusted computing domain comprises a third type of sub-trusted computing domain provided by the second scheduling device and a fourth type of sub-trusted computing domain respectively provided by each second computing device; wherein the second sub-model is deployed in each third type of sub-trusted computing domain; Before encrypting the gradient information for the intermediate features calculated based on the loss value in the second trusted computing domain, the method further comprises: For each second computing device, input branch features sent by the second scheduling device to the second computing device into a second sub-model in a fourth type of sub-trusted computing domain corresponding to the computing device; determine a branch loss value according to a branch prediction result output by the second sub-model and label data corresponding to the received branch features, and further calculate branch gradient information for the received branch features; encrypt the branch gradient information within the fourth type of sub-trusted computing domain, send the obtained encrypted branch gradient data out of the fourth type of sub-trusted computing domain, and send the encrypted branch gradient data to an associated scheduling device. The encrypted branch gradient data received by the second scheduling device is transmitted into the third type of sub-trusted computing domain, decrypted into corresponding branch gradient information in the third type of sub-trusted computing domain, and all branch gradient information is comprehensively calculated to obtain gradient information for the intermediate feature.

13. A business execution method, comprising: obtaining business data; inputting the business data into a pre-trained business model to determine a prediction result for the business data by the business model, and executing a business according to the prediction result; the business model is trained by the method of any one of claims 9-12.

14. An electronic device comprising: a processor; a memory for storing processor-executable instructions; wherein the processor implements the steps of the method of any one of claims 9-13 by running the executable instructions.

15. A computer-readable storage medium having computer instructions stored thereon, the instructions being executed by a processor to implement the steps of the method of any one of claims 9-13.

16. A computer program product comprising computer programs / instructions, the computer programs / instructions being executed by a processor to implement the steps of the method of any one of claims 9-13.

Images