Model hybrid distributed fine-tuning method and system based on trusted data space

Abstract

The embodiment of the application discloses a model hybrid distributed fine-tuning method and system based on a trusted data space, wherein the method comprises the following steps: obtaining to-be-trained plaintext data; splitting a large model into a first model and a second model and respectively deploying the first model and the second model on a first computing node group and a second computing node group; performing forward calculation on the first model according to the to-be-trained plaintext data and performing result cryptograph processing, transmitting the cryptograph processing data to the second computing node group; decrypting the cryptograph processing data, performing forward calculation and back propagation on the second model according to the decrypted data, obtaining feature map gradients, performing cryptograph processing on the feature map gradients, and sending the feature map gradients to a trusted execution environment; decrypting the cryptograph gradient data and performing back propagation on the first model according to the decrypted data, and obtaining update parameters of the first model according to a propagation result. The embodiment of the application provides an efficient calculation solution for fine-tuning of a large model with a parameter scale of 100 billion in a cryptograph scene.

Description

Technical Field

[0001] This invention relates to the field of privacy computing, and in particular to a model hybrid distributed dense state fine-tuning method and system based on trusted data space. Background Technology

[0002] In recent years, large-scale model-based vertical applications have rapidly developed in industries such as healthcare, finance, and government. The capabilities of large models require fine-tuning to adapt them to specific industry data in order to fully realize their performance. However, the data volume possessed by a single data provider (such as a hospital or a bank) is often insufficient to support effective fine-tuning of large models. Training a large model with a small amount of data can easily lead to model parameter collapse or overfitting. Therefore, in practice, it is necessary to combine data from multiple data sources within the industry for joint fine-tuning. For example, when training a large-scale medical consultation model, it is necessary to integrate medical data from across the province or even the country to achieve the required training scale.

[0003] In the process of joint data fine-tuning, data security and privacy protection are critical issues that must be addressed. Directly centralizing plaintext data violates data protection regulations, while simple encrypted transmission cannot guarantee data security during computation. Existing technologies have designed model structures at the algorithm level to achieve secure training, such as schemes based on differential privacy, homomorphic encryption, or secure multi-party computation. However, most of these schemes focus on small-scale models or specific algorithmic structures. When facing ultra-large models with hundreds of billions of parameters, the model itself needs to be split across multiple GPUs to accommodate it. Communication and computation efficiency in distributed training is already a huge challenge. If dense computation is further introduced (such as placing all data in a TEE), the training speed is extremely slow due to the limited computing power of the TEE (especially CPU implementations). At the same time, in dense scenarios, the frequent transmission of intermediate results and gradients between nodes results in huge communication volumes for existing parameter synchronization methods. With the added overhead of dense computation, it is difficult to meet practical requirements.

[0004] Therefore, how to efficiently complete the distributed fine-tuning of ultra-large models while ensuring the security of multi-party data density is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a hybrid distributed dense-state fine-tuning method and system for models based on a trusted data space. By splitting a large model and deploying it separately in a hybrid computing architecture of a trusted execution environment and a graphics processor environment, combined with a parameter fine-tuning mechanism, the distributed fine-tuning of ultra-large models can be efficiently completed while ensuring data density security.

[0006] To achieve the above objectives, this invention provides a model hybrid distributed dense-state fine-tuning method based on a trusted data space, comprising: acquiring plaintext data to be trained; splitting the large model to be fine-tuned into a first model and a second model; deploying the first model in multiple first computing node groups in a trusted execution environment; and deploying the second model in multiple second computing node groups in a graphics processor environment; wherein the first model includes a front continuous multilayer and a rear continuous multilayer of the large model, the second model includes the remaining middle continuous multilayer of the large model, and the number of layers in the first model is less than the number of layers in the second model; in the trusted execution environment, each first computing node group performs forward computation on the first model according to the plaintext data to be trained, and performs dense-state processing on the computation results to obtain the dense-state processed model. The data is transmitted to a graphics processing unit (GPU) environment. In the GPU environment, the encrypted data is decrypted to obtain first decrypted data. Each of the second computing nodes performs forward computation and backpropagation on the second model based on the first decrypted data to obtain feature map gradients and parameter gradients. The feature map gradients are then encrypted to obtain encrypted gradient data, which is transmitted to a trusted execution environment. In the trusted execution environment, the encrypted gradient data is decrypted to obtain second decrypted data. The first model is then backpropagated based on the second decrypted data, and the updated parameters of the first model are obtained based on the backpropagation results. In the GPU environment, the parameters of the second model are updated based on the parameter gradients. When the training epochs reach a preset number of epochs, the parameter results of the target large model fine-tuning are obtained.

[0007] Further optionally, before performing forward computation on the first model, the process includes: the master node of each first computing node group loading the plaintext data to be trained, sorting the plaintext data to be trained according to the same random seed to obtain a sorting result, and dividing the sorting result according to a preset batch size to obtain multiple data batches; the first master node in the trusted execution environment allocating a data batch interval to each first computing node group according to the number of the first computing node groups and the number of data batches; and the master node of each first computing node group obtaining the node data to be trained from the plaintext data to be trained according to its assigned data batch interval; wherein, the first master node is the master node of the master node group in the trusted execution environment.

[0008] Optionally, the forward computation of the first model and the dense processing of the computation results to obtain dense-processed data include: the first computing node of each first computing node group performs forward computation of the corresponding node sub-model based on the data of the node to be trained, and passes the computation result to the next computing node in the group; the next computing node performs forward computation of the corresponding node sub-model based on the received computation result, and passes the computation result to the next computing node in the group, until the last computing node in the group completes the forward computation, obtains the group feature map, and sends it to the master node in the group; the master node of each first computing node group transmits the group feature map of its group to the first general master node; the first general master node performs dense processing on all group feature maps to obtain dense-processed data.

[0009] Further optionally, each of the second computing node groups performs forward computation and backward propagation on the second model based on the first decrypted data to obtain the feature map gradient, including: a second master node in the graphics processor environment splits and sequentially labels the first decrypted data according to the number of each second computing node group, and then distributes it to each second computing node group; the first computing node in each second computing node group performs forward computation on the corresponding node model based on the distributed data of its group, and passes the computation result to the next computing node in the group, and so on until the last computing node in the group completes the forward computation to obtain the prediction result of the group; within each second computing node group... The last computing node calculates the loss value based on the prediction result, and calculates the gradient corresponding to the node based on the loss value. It then passes the gradient to the previous computing node. Each computing node calculates its own gradient after receiving the gradient and passes it to the previous computing node until it reaches the first computing node in the group to complete the gradient calculation, thus obtaining the inverse gradient corresponding to the group. The master node of each second computing node group collects the inverse gradient of its group and transmits it to the second master node. The second master node aggregates the inverse gradients according to the label order to obtain the feature map gradient. The second master node is the master node of the master node group in the graphics processor environment.

[0010] Optionally, obtaining the updated parameters of the first model based on the propagation results of backpropagation includes: dividing computing nodes in each first computing node group within the trusted execution environment that have loaded the same node sub-model into the same parameter update group; for any parameter update group, determining the local low-rank adaptation parameters of each computing node based on the propagation results; each computing node in the parameter update group vectorizing the local low-rank adaptation parameters and concatenating them to obtain the original parameter vector; dividing the original parameter vector into multiple parameter fragments; wherein the number of parameter fragments does not exceed the number of nodes in the parameter update group; each computing node in the parameter update group determining the parameter fragment number to be uploaded in the current round according to the current round and the index of the node in the group, and uploading the parameter fragment corresponding to the parameter fragment number; the master node in the parameter update group accumulating the parameter fragments with the same parameter fragment number uploaded within a preset round range, dividing by the number of nodes that uploaded the parameter fragment, to obtain the average parameter of the parameter fragment; concatenating all the average parameters to obtain the updated parameters, and updating the parameters of each computing node in the parameter update group based on the updated parameters.

[0011] On the other hand, the present invention also provides a model hybrid distributed dense-state fine-tuning system based on a trusted data space, comprising: a training data acquisition module for acquiring plaintext data to be trained; a model deployment module for splitting the large model to be fine-tuned into a first model and a second model, deploying the first model in multiple first computing node groups in a trusted execution environment, and deploying the second model in multiple second computing node groups in a graphics processor environment; wherein the first model includes a front continuous multilayer and a rear continuous multilayer of the large model, the second model includes the remaining middle continuous multilayer of the large model, and the number of layers in the first model is less than the number of layers in the second model; and a forward computation module for performing forward computation on the first model in the trusted execution environment according to the plaintext data to be trained, performing dense-state processing on the computation results to obtain dense-state processed numbers. The data is transmitted to the graphics processing environment; the backpropagation module is used to decrypt the encrypted processing data in the graphics processing environment to obtain first decrypted data. Each second computing node group performs forward computation and backpropagation on the second model according to the first decrypted data to obtain feature map gradients and parameter gradients. The feature map gradients are then processed in an encrypted manner to obtain encrypted gradient data, which is transmitted to the trusted execution environment; the parameter update module is used to decrypt the encrypted gradient data in the trusted execution environment to obtain second decrypted data. The first model is then backpropagated according to the second decrypted data, and the updated parameters of the first model are obtained according to the backpropagation results; the parameters of the second model are updated according to the parameter gradients in the graphics processing environment; the large model determination module is used to obtain the parameter results of the target large model fine-tuning when the training rounds reach a preset number of rounds.

[0012] Further optionally, the forward computation module includes: a training data partitioning submodule, used by the master node of each first computing node group to load the training plaintext data, sort the training plaintext data according to the same random seed, obtain a sorting result, and divide the sorting result according to a preset batch size to obtain multiple data batches; a training data allocation submodule, used by the first master node in the trusted execution environment to allocate data batch intervals to each first computing node group according to the number of the first computing node groups and the number of data batches; and a training data determination submodule, used by the master node of each first computing node group to obtain training node data from the training plaintext data according to the data batch interval to which it belongs; wherein, the first master node is the master node of the master node group in the trusted execution environment.

[0013] Further optionally, the forward computation module includes: a forward computation submodule, used by the first computation node of each first computation node group to perform forward computation of the corresponding node sub-model based on the data of the node to be trained, and to pass the computation result to the next computation node in the group; a group feature map computation submodule, used by the next computation node to perform forward computation of the corresponding node sub-model based on the received computation result, and to pass the computation result to the next computation node in the group, until the last computation node in the group completes the forward computation, obtains the group feature map, and sends it to the master node in the group; a feature transmission submodule, used by the master node of each first computation node group to transmit the group feature map of its group to the first overall master node; and a dense state processing submodule, used by the first overall master node to perform dense state processing on all group feature maps to obtain dense state processed data.

[0014] Further optionally, the backpropagation module includes: a decryption data allocation submodule, used by the second master node in the graphics processor environment to split and sequentially label the first decrypted data according to the number of each second computing node group, and then allocate it to each second computing node group; a prediction result calculation submodule, used by the first computing node in each second computing node group to perform forward calculation of the corresponding node model based on the allocated data of the group, and pass the calculation result to the next computing node in the group, and so on until the last computing node in the group completes the forward calculation to obtain the prediction result of the group; and a back gradient calculation submodule, used by the last computing node in each second computing node group. The node calculates the loss value based on the prediction result, and calculates the gradient corresponding to the node based on the loss value. The gradient is then passed to the previous computing node. Each computing node calculates its own gradient after receiving the gradient and passes it to the previous computing node until it is passed to the first computing node in the group to complete the gradient calculation, thus obtaining the inverse gradient corresponding to the group. The gradient aggregation submodule is used by the master node of each second computing node group to collect the inverse gradient of the group and transmit it to the second master node. The second master node aggregates the inverse gradients according to the label order to obtain the feature map gradient. The second master node is the master node of the master node group in the graphics processor environment.

[0015] Further optionally, the parameter update module includes: a parameter update group partitioning submodule, used to partition computing nodes with the same node sub-model loaded in each first computing node group in the trusted execution environment into the same parameter update group; and to partition computing nodes with the same node sub-model loaded in each second computing node group in the graphics processor environment into the same parameter update group; and a parameter fragment segmentation submodule, used to determine the local low-rank adaptation parameters of each computing node in any parameter update group according to the propagation result, wherein each computing node in the parameter update group vectorizes the local low-rank adaptation parameters and concatenates them to obtain the original parameter vector, and divides the original parameter vector into multiple parameter fragments; wherein the number of parameter fragments does not exceed The parameter update group has a number of nodes; a parameter fragment upload submodule is used for each computing node in the parameter update group to determine the parameter fragment number to be uploaded in the current round according to the current round and the index of the node in the group, and upload the parameter fragment corresponding to the parameter fragment number; an average parameter calculation submodule is used for the main node in the parameter update group to accumulate the parameter fragments with the same parameter fragment number uploaded within a preset round range, and divide by the number of nodes that uploaded the parameter fragment to obtain the average parameter of the parameter fragment; a parameter update submodule is used to concatenate all average parameters to obtain the update parameters, and update the parameters of each computing node in the parameter update group according to the update parameters.

[0016] The above technical solution has the following beneficial effects: By splitting the large model into a first model part deployed in a trusted execution environment and a second model part deployed in a graphics processing unit environment, sensitive computations are placed in a secure area while a large number of matrix operations are completed by high-performance GPUs, avoiding the performance bottleneck of fully dense computation; by using serial forward and backward propagation within each computing node group and transmitting only feature maps and gradients, combined with low-rank adaptation fine-tuning that only updates a small number of parameters, the amount of communication between nodes is significantly reduced; by dividing the parameter vector equally during the parameter update stage and uploading different segments according to the rotation rule, and synchronizing the parameters after aggregation and averaging, the amount of communication in a single round is compressed to a fraction of the original parameters, greatly reducing the distributed synchronization overhead; by encrypting the transmission of data to the trusted execution environment from each data provider and transmitting densely processed data between the TEE and the GPU, the privacy and security of data are ensured throughout the entire process of transmission, storage, and computation. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the model hybrid distributed dense-state fine-tuning method based on trusted data space provided in the embodiments of the present invention; Figure 2 This is a flowchart of the plaintext data allocation method for training provided in an embodiment of the present invention; Figure 3 This is a flowchart of the model forward computation method provided in the embodiments of the present invention; Figure 4 This is a flowchart of the feature map gradient calculation method provided in the embodiments of the present invention; Figure 5 This is a flowchart of the parameter update method provided in the embodiments of the present invention; Figure 6 This is a schematic diagram of the structure of the model hybrid distributed dense-state fine-tuning system based on trusted data space provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of the sub-module for data allocation in the forward computing module provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of the sub-module used for forward computation in the forward computation module provided in the embodiment of the present invention; Figure 9 This is a schematic diagram of the backpropagation module provided in an embodiment of the present invention; Figure 10 This is a schematic diagram of the parameter update module provided in an embodiment of the present invention.

[0019] Figure labeling: 100-Training data acquisition module; 200-Model deployment module; 300-Forward computation module; 3001-Training data partitioning submodule; 3002-Training data allocation submodule; 3003-Training data determination submodule; 3004-Forward computation submodule; 3005-Group feature map calculation submodule; 3006-Feature transmission submodule; 3007-Dense state processing submodule; 400-Backpropagation module; 4001-Decryption data allocation submodule; 4002-Prediction result calculation submodule; 4003-Backward gradient calculation submodule; 4004-Gradient aggregation submodule; 500-Parameter update module; 5001-Parameter update group partitioning submodule; 5002-Parameter fragment segmentation submodule; 5003-Parameter fragment upload submodule; 5004-Average parameter calculation submodule; 5005-Parameter update submodule; 600-Large model determination module. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] To address the challenge of efficiently performing distributed fine-tuning of ultra-large models while ensuring data security in existing technologies, this invention provides a hybrid distributed dense-state fine-tuning method for models based on a trusted data space. Figure 1 This is a flowchart of the hybrid distributed dense-state fine-tuning method for models based on trusted data space provided in this embodiment of the invention, as shown below. Figure 1 As shown, the method includes: S1. Obtain the plaintext data to be trained.

[0022] Each data provider preprocesses the raw data according to a unified data format and encrypts the processed data locally. The encrypted data is then transmitted to the data user's connector through their respective trusted data space connectors. The data user receives the encrypted data in a trusted execution environment, decrypts it in the encrypted memory of that environment, and integrates the decrypted plaintext data to obtain the plaintext data to be trained for subsequent model fine-tuning.

[0023] For example, when training a large medical consultation model, multiple hospitals act as data providers. Each hospital encrypts its consultation records in a unified format and sends them to the trusted execution environment (TEE) of the model trainer (such as a provincial medical AI center) through a trusted data space connector. The model trainer decrypts and integrates the data from all hospitals within the TEE to obtain a plaintext dataset containing medical consultation records from across the province, which is then used for fine-tuning the large model.

[0024] S2. Split the large model to be fine-tuned into a first model and a second model. Deploy the first model in multiple first computing node groups in a trusted execution environment, and deploy the second model in multiple second computing node groups in a graphics processor environment. The first model includes the front and rear continuous multi-layers of the large model, and the second model includes the remaining middle continuous multi-layers of the large model. The number of layers in the first model is less than the number of layers in the second model.

[0025] The large model to be fine-tuned is structurally divided into the first model. Second Model The first model part contains only a small portion of the entire large model, such as about 10%. The first model includes the first few layers (i.e., the first continuous layers near the input of the large model) and the last few layers (i.e., the last continuous layers near the output of the large model). It is responsible for the feature extraction of the input data and the mapping of the final output. This part involves sensitive information of the original data and intermediate features and must be placed in a secure environment. The second model part contains most of the middle layers of the model (i.e., the remaining middle continuous layers after removing the first and last continuous layers). It is responsible for the main nonlinear transformation and feature abstraction. It has a huge amount of computation, but its direct sensitivity to the data is relatively low.

[0026] The first model is deployed across multiple clusters of first compute nodes within a Trusted Execution Environment (TEE). The TEE provides a secure computing region based on hardware isolation, ensuring that code and data are not accessed externally during runtime. Within each cluster of first compute nodes, the first model is further divided into multiple model blocks, with each block assigned to a compute node (Rank) within the cluster. The compute nodes (Rank) in the TEE can be either CPUs or GPUs, but in practice, CPU computing resources are more commonly used.

[0027] Each primary computing node cluster contains multiple computing nodes. One of these nodes is selected as the master node of the cluster, responsible for task scheduling, data loading, result aggregation, and communication with other nodes. The remaining nodes in the cluster are slave nodes, which perform specific model block calculations in the forward and backward propagation processes. Multiple primary computing node clusters are coordinated through a master node (i.e., the master node of the master node cluster), forming a hierarchical distributed computing architecture.

[0028] The second model is deployed across multiple clusters of second compute nodes within a graphics processing unit (GPU) environment. The GPU environment contains substantial GPU computing resources to accelerate matrix operations. Similar to the trusted execution environment, each cluster of second compute nodes comprises multiple GPU nodes, with one node selected as the master node and the rest as slave nodes. The second model is layered into multiple model blocks, which are then sequentially assigned to the GPU nodes within the cluster. Likewise, the multiple clusters of second compute nodes are centrally scheduled and data distributed through a master node within the GPU environment (i.e., the master node of the master node cluster).

[0029] In one specific embodiment, in a trusted execution environment, three computing units form a first computing node group, and the number of first computing node groups is 3. In a graphics processor, three computing units form a second computing node group, and the number of second computing node groups is 4.

[0030] It should be noted that the batch size of the model training parameters must satisfy the following condition: the number of the first computing node group * the batch size must be divisible by the number of the second computing node group.

[0031] For example, in a trusted execution environment, the master node (first master node) in the master node group is... The other compute nodes in the master node group are , The master node in a group of slave nodes is From other computing nodes in the node group , Additionally, the master node in the node group is... From other computing nodes in the node group , So, in the first model During deployment, , and The corresponding sub-model is the node. , and ; , , Corresponding allocation node sub-model , and Other first-order computing node groups are similar.

[0032] Similarly, in a graphics processor environment, the master node (secondary master node) in the master node group is... The rest are slave nodes, and the master node of the other three slave node groups is... , and The node sub-model carried by each computing node in the master node group is as follows: , and .

[0033] S3. In the trusted execution environment, each first computing node group performs forward computation on the first model based on the plaintext data to be trained, performs dense processing on the computation results, obtains dense processing data, and transmits it to the graphics processor environment.

[0034] Before data training, the master node of the master node group in the first computing node group starts up the service and starts up each slave node group, and the graphics processor environment is also kept consistent.

[0035] Each first computing node group loads the model, and the master node of each first computing node group loads the data; in the graphics processor environment, each second computing node group loads the model.

[0036] Multiple computing nodes within each first computing node group are divided according to the structure of the first model. Based on the plaintext data to be trained, they sequentially complete the computation of their respective sub-models in a pipeline manner. Finally, the master node of each group collects the group feature map output by the group, which is the computation result of the forward computation.

[0037] The group feature maps of all first computing node groups are aggregated to the first master node (i.e., the master node of the master node group) in the trusted execution environment. The first master node performs uniform encrypted processing on these group feature maps (e.g., using a combination of differential privacy and homomorphic encryption), so that the processed data can still be used for subsequent computations without revealing plaintext features.

[0038] Then, the first master node transmits the encrypted data to the graphics processor environment via a secure communication protocol.

[0039] S4. In the graphics processor environment, the encrypted processing data is decrypted to obtain the first decrypted data. Each second computing node group performs forward calculation and backward propagation on the second model based on the first decrypted data to obtain the feature map gradient and parameter gradient. The feature map gradient is then processed in an encrypted manner to obtain the encrypted gradient data, which is then transmitted to the trusted execution environment.

[0040] The graphics processing unit environment decrypts the received encrypted data, recovers the plaintext feature data that can be computed, and distributes the decrypted data, i.e. the first decrypted data, to each of the second computing node groups according to preset rules.

[0041] Based on the allocated data, each second computing node cluster sequentially performs forward computation and backward propagation on the second model portion deployed locally. Through serial transmission, it completes the computation and gradient solving of the model blocks carried by each node within the cluster, obtaining the feature map gradient corresponding to its cluster and the parameter gradients calculated by each computing node. The master node of each second computing node cluster collects the feature map gradients of its cluster and converges them to the second master node in the graphics processor environment. The second master node aggregates the feature map gradients of each cluster to obtain the complete feature map gradient.

[0042] Subsequently, the gradient of the feature map is processed in a dense state (e.g., using differential privacy and homomorphic encryption) to form dense gradient data, which is then transmitted back to the trusted execution environment via a secure communication protocol.

[0043] S5. In a trusted execution environment, decrypt the encrypted gradient data to obtain the second decrypted data, perform backpropagation on the first model based on the second decrypted data, and obtain the updated parameters of the first model based on the propagation results of the backpropagation; in a graphics processor environment, update the parameters of the second model based on the parameter gradient.

[0044] The Trusted Execution Environment receives encrypted gradient data from the graphics processor environment, decrypts the data, and recovers the plaintext gradient data, i.e., the second decrypted data.

[0045] The first master node in the Trusted Execution Environment (TEE) continues backpropagation based on the second decrypted data (i.e., feature map gradients). Starting with these feature map gradients, it sequentially transmits gradients to each computing node within each first computing node group in the reverse order of the forward computation on the TEE side. Each computing node calculates the parameter gradients of the first model portion it carries based on the received gradients, until the backpropagation of all model blocks within the first computing node groups is completed. After backpropagation, the propagation result is obtained, which is the parameter gradients corresponding to each computing node in the first model portion.

[0046] For the first model part, each computing node in the first computing node group updates its LoRA parameters based on the backpropagation results, combined with the preset optimizer and learning rate.

[0047] For the second model part, each computing node in the second computing node group updates its LoRA parameters according to the parameter gradient, using the same optimizer and learning rate as the first model part.

[0048] It should be noted that both the first and second models use the low-rank adaptation (LoRA) fine-tuning method, which means that only the low-rank matrix parameters A and B injected into the model are updated, while the main parameters of the original large model are frozen.

[0049] S6. When the training rounds reach the preset number of rounds, the parameter results of the fine-tuning of the target large model are obtained.

[0050] Repeat steps S3-S5 above until the loss function converges or the preset number of training rounds is reached (each complete forward calculation, backpropagation, and gradient calculation constitutes a complete training round), to complete the fine-tuning of the entire large model. Merge the first and second models after fine-tuning to obtain the target large model, and use the target large model for subsequent inference tasks.

[0051] In practical applications, the ultimate goal is to deploy the large-scale model in various industry business scenarios. For example, in a medical consultation system, the model outputs diagnostic suggestions or medication recommendations when the patient's symptom description is input; in a financial risk control scenario, the model outputs a risk assessment report when the company information is input; and in a government affairs Q&A system, the model outputs policy interpretations or service guidelines when the user's question is input.

[0052] As an optional implementation method, Figure 2 This is a flowchart of the plaintext data allocation method for training provided in an embodiment of the present invention, such as... Figure 2 As shown, before performing forward computation on the first model, the following steps are included: S301. The master node of each first computing node group loads the plaintext data to be trained, sorts the plaintext data to be trained according to the same random seed, obtains the sorting result, and divides the sorting result according to the preset batch size to obtain multiple data batches.

[0053] In the trusted execution environment, all master nodes in the first cluster of computing nodes load the same plaintext data to be trained. To ensure that all nodes process the data in a completely consistent order, each master node initializes its own pseudo-random number generator using the same random seed (e.g., a common seed broadcast by the first master node). This ensures that when each master node independently randomizes the dataset, it obtains the exact same sorting result.

[0054] Each master node divides the sorting results into multiple data batches according to a preset batch size. Each batch contains a fixed number of consecutive samples, numbered sequentially as batch 0, batch 1, batch 2, and so on, until all data is covered.

[0055] S302. The first master node in the trusted execution environment allocates data batch intervals to each first computing node group based on the number of first computing node groups and the number of data batches.

[0056] The first master node obtains the total number of generated data batches (e.g., if there are 1536 plaintext data entries to be trained, and the preset batch size is 128, then the total number of data batches is 12). Simultaneously, the first master node determines the number of first computing node groups currently participating in the training (e.g., 3). The first master node divides the total number of data batches equally according to the number of node groups, and calculates the length of the consecutive batch interval that each first computing node group should be responsible for. For example, if there are 12 total batches and 3 node groups, then each node group is responsible for 4 consecutive batches.

[0057] The first master node allocates consecutive intervals of starting batch numbers according to the inherent order of the node groups (e.g., master node group, slave node group 1, slave node group 2, etc.): node group 1 is responsible for batches 0 to 3, node group 2 is responsible for batches 4 to 7, and node group 3 is responsible for batches 8 to 11. If the total number of data batches is not divisible by the number of node groups, the remaining batches are allocated sequentially to the first few node groups to ensure that each node group receives at least an integer number of batches and that the batch intervals do not overlap.

[0058] The first master node will broadcast the allocation results (i.e., the start and end numbers of the batch interval for each node group) to the master nodes of each first computing node group via internal communication.

[0059] S303. The master nodes of each first computing node group obtain the training node data from the plaintext data to be trained according to the data batch interval to which they belong; among them, the first general master node is the master node of the master node group in the trusted execution environment.

[0060] After completing global sorting and batch partitioning, each master node in the first computing node group has a complete batch list locally (e.g., batches 0 to 11). When the first master node notifies each master node of the batch range (e.g., node group 1 is responsible for batches 0 to 3), each master node extracts only the sample data corresponding to those batches within its own range from its locally maintained batch list; these are the data to be trained. For example, the master node of node group 1 loads all samples from batches 0, 1, 2, and 3; the master node of node group 2 loads batches 4, 5, 6, and 7; and the master node of node group 3 loads batches 8, 9, 10, and 11. Each master node organizes the extracted data into an internal training queue, which serves as the input data source for subsequent forward computations within its group.

[0061] The first master node is the master node of the master node group in the trusted execution environment (e.g., It is responsible for global scheduling, batch allocation, and aggregation and distribution of dense data.

[0062] As an optional implementation method, Figure 3 This is a flowchart of the model forward computation method provided in the embodiments of the present invention, such as... Figure 3As shown, forward computation is performed on the first model, and the computation results are processed in a dense state to obtain dense state processed data, including: S304. The first computing node of each first computing node group performs forward computation of the corresponding node sub-model based on the data of the node to be trained, and passes the computation result to the next computing node in the group.

[0063] Each group of first computing nodes obtains the training data for the nodes it is responsible for, based on the first model. The partitioning method involves the first computation node (usually the master node) carrying the first few layers of the first model part (e.g., the first block of the model). This computation node takes the data assigned to the training node in its own group as input, loads the model parameters on this node (including the original large model frozen parameters and LoRA low-rank adaptation parameters), performs forward computation, and obtains the output result, namely the intermediate feature map. This feature map is a high-dimensional representation of the input data after being extracted by the first block of the model.

[0064] After completing the computation, the first computing node transmits the intermediate feature map to the second computing node in the cluster via inter-node communication (e.g., high-speed interconnect or message passing interface). That is, the bearing model The computing nodes.

[0065] S305. The next computing node performs forward computation of the corresponding node sub-model based on the received computation result, and passes the computation result to the next computing node in the group until the last computing node in the group completes the forward computation, obtains the group feature map, and sends it to the master node in the group.

[0066] The second computing node in the group Receive the intermediate feature map passed from the first computing node, use this feature map as the input of this node, and load the node sub-model of the first model part carried by this node (e.g., The forward computation is performed, generating a new intermediate feature map, which is then passed on to the third computation node in the group. That is, the carrier sub-model The nodes. And so on, each computing node (the third, the fourth... up to the second to last) performs the same operation: receives the feature map from the previous node, calculates the sub-model of this node, and then passes the output feature map to the next node.

[0067] When it is passed to the last computing node in this group, such as At that time, the node receives the feature map from the previous node and loads the last sub-model of the first model part it carries (e.g., This completes the final forward computation. The output of this node is the final feature map of this first computation node group, called the group feature map. The group feature map represents a high-dimensional abstract representation of the data batch processed by this group after passing through all sub-models of the first model part.

[0068] After the last computing node completes its calculation, it sends the group feature map to the group's master node (i.e., the first computing node in the group, which is also the group's scheduling and aggregation node) via intra-group communication.

[0069] S306. The master node of each first computing node group transmits the group characteristic map of its group to the first general master node.

[0070] After receiving the group feature map sent by the last computing node in its group, the master node of each first computing node group (i.e., the first computing node in the group) sends the group feature map of its group to the first master node (i.e., the master node of the master node group) through the internal high-speed communication network. The first master node is responsible for receiving and aggregating the group feature maps from all the first computing nodes to form a complete set of intermediate results.

[0071] S307. The first master node performs dense state processing on all group feature maps to obtain dense state processed data.

[0072] The first master node aggregates group feature maps from each of the first compute node groups. These feature maps are in plaintext format and contain intermediate features extracted from the TEE-side model. To protect data privacy when transmitting data to the graphics processor environment and prevent theft or leakage during cross-environment transmission, the first master node performs encrypted processing on these group feature maps.

[0073] Encryption processing includes, but is not limited to, two techniques: differential privacy and homomorphic encryption. Differentiated privacy adds random noise following a specific distribution to the feature map, making it impossible for attackers to distinguish the contribution of individual samples, thus protecting the privacy of the original data. Homomorphic encryption encrypts the feature map using homomorphic encryption algorithms, allowing the encrypted data to be directly used for preset computational operations (such as addition and multiplication) in the graphics processing unit (GPU) environment without decryption. The decrypted result is identical to the result of directly calculating the plaintext. In practical implementations, these techniques can be flexibly combined according to security requirements and computational overhead.

[0074] After encrypted processing, the original plaintext group feature map is converted into encrypted processed data, which can be securely transmitted to the graphics processing unit (GPU) environment without revealing valid information. The first master node then sends the encrypted processed data to the second master node in the GPU environment via a secure communication protocol (such as gRPC). .

[0075] As an optional implementation method, Figure 4 This is a flowchart of the feature map gradient calculation method provided in the embodiments of the present invention, such as... Figure 4 As shown, each group of second computing nodes performs forward computation and backward propagation on the second model based on the first decrypted data to obtain the feature map gradient, including: S401, the second master node in the graphics processor environment, splits and sequentially marks the first decrypted data according to the number of each second computing node group, and then distributes it to each second computing node group.

[0076] Second Master Node The encrypted processing data is received. The second master node uses the pre-negotiated decryption key (or the corresponding private key) to decrypt the encrypted processing data, recovering the plaintext feature map data, i.e., the first decrypted data. This first decrypted data corresponds to the plaintext form of the group feature map output by each first computing node group on the TEE side. However, since noise has been added before transmission using differential privacy, the sensitive information of the original sample will not be exposed even after decryption.

[0077] The second master node obtains the number of second computing node clusters (denoted as M, e.g., 4 node clusters) in the current graphics processor environment participating in training. The second master node then splits the decrypted feature map data into M sub-data blocks. The splitting can be based on the uniform division of the feature map along the sample dimension; for example, if the first decrypted data contains feature maps corresponding to N samples, then each sub-block is allocated approximately N / M sample feature maps. Simultaneously, the second master node sequentially labels each split sub-block (e.g., numbered 0, 1, ..., M-1) so that the original order can be restored during subsequent aggregation.

[0078] The second master node will split and mark the sub-data blocks and distribute them to the corresponding master nodes of the second computing node group. Each master node of the second computing node group receives its own sub-data block as input for subsequent forward computations of its group.

[0079] S402. The first computing node in each second computing node group performs forward calculation of the corresponding node model based on the allocated data of the group, and passes the calculation result to the next computing node in the group, and so on until the last computing node in the group completes the forward calculation, thus obtaining the prediction result of the group.

[0080] The first computing node (usually the master node) of each second computing node group loads the first model block of the second model part carried by this node (e.g., the first model block of the second model part carried by this node) according to the allocated sub-data block (i.e., the decrypted feature map data). ), perform forward computation to generate intermediate feature maps.

[0081] After completing the computation, the node transmits the intermediate feature map to the second computation node in the cluster via a high-speed communication interface (such as NVLink between GPUs or network communication). The second computation node receives the intermediate feature map from the previous node, uses it as input, loads the next model block of the second model part it carries, continues the forward computation, and outputs a new intermediate feature map, which is then passed to the third computation node. This process continues, with each subsequent computation node performing the same operation: receiving the output from the previous node, performing the forward computation of its own model block, and passing the result to the next node, until the last computation node in the cluster is reached. The last computation node receives the intermediate feature map from the previous node, loads the last model block of the second model part it carries, completes the final forward computation, and outputs the prediction result corresponding to the cluster (e.g., classification probability, regression value, etc.).

[0082] Taking a master node cluster as an example, calculate The model will then provide the calculation results to... , calculate The model will then provide the calculation results to... , calculate The model yields prediction results.

[0083] S403. The last computing node in each second computing node group calculates the loss value based on the prediction result, and calculates the corresponding gradient of the node based on the loss value. The gradient is then passed to the previous computing node. After receiving the gradient, each computing node calculates its own gradient and passes it to the previous computing node until it is passed to the first computing node in the group to complete the gradient calculation and obtain the inverse gradient of the group.

[0084] The last computing node in each second computing node group calculates the loss value based on the prediction result and the true label, and calculates the gradient of its own output based on the loss value to obtain the initial gradient of the output of this node.

[0085] Then, the last computation node passes the calculated gradient to the previous computation node. Each non-first node in the group (from the last node to the second node), after receiving the gradient from the next node, performs the following operations: using the received gradient, combined with the intermediate state saved by the node in the forward computation (such as the input feature map), calculates the parameter gradient of the model block carried by the node (for subsequent updates of LoRA parameters); at the same time, it calculates the gradient at the input of the node (i.e., the partial derivative with respect to the output feature map of the previous node), and continues to pass this input gradient to the previous computation node.

[0086] The above process proceeds sequentially: the second-to-last node receives the gradient passed from the last node, calculates its own parameter gradient and input gradient, and then passes the input gradient to the third-to-last node; and so on, until it reaches the first computation node in the group. The first computation node, upon receiving the gradient from the second node, also calculates its own parameter gradient and input gradient. The input gradient of the first computation node is the inverse gradient of the input feature map of the group, which is the inverse gradient corresponding to the group (used to be passed back to the TEE environment).

[0087] Through the above chain-like backpropagation, each second computing node group can obtain the parameter gradients of each computing node in its own group (used to update the LoRA parameters of each model block in the second model part), as well as the backpropagation gradient of the input feature map of its own group (i.e. the backpropagation output of the entire group, for subsequent cross-environment transmission).

[0088] S404. The master node of each second computing node group collects the inverse gradient of its group and transmits it to the second total master node. The second total master node aggregates the inverse gradients in the order of their labels to obtain the feature map gradient. The second total master node is the master node of the master node group in the graphics processor environment.

[0089] The master node of each second compute node group (i.e., the first compute node in its group) collects the inverse gradient of the input feature map through intra-group communication (if the master node already possesses this gradient, no additional collection is required), and then sends this inverse gradient of its group to the second master node via a high-speed network. The second master node is the master node of the master node group in the graphics processor environment.

[0090] After receiving the backward gradients from all the second computing nodes, the second master node concatenates and aggregates these backward gradients in their original order according to the marking order (i.e., the sequential marking of each sub-block during splitting). The aggregated result is the feature map gradient. This gradient data corresponds completely to the group feature map initially sent by the TEE side in spatial dimension, ensuring the correct alignment between the gradient and the feature map during subsequent backward propagation on the TEE side.

[0091] As an optional implementation method, Figure 5 This is a flowchart of the parameter update method provided in the embodiments of the present invention, such as... Figure 5 As shown, the updated parameters of the first model are obtained based on the propagation results of backpropagation, including: S501. In the trusted execution environment, computing nodes in each first computing node group that have the same node sub-model loaded are assigned to the same parameter update group.

[0092] Since each first computing node group carries the first model part according to the same partitioning method, for example, the first computing node in each group carries the first sub-model ( The second computing node carries the second sub-model ( And so on. Therefore, in different first computing node groups, computing nodes with the same sorting position (i.e., loading the same node sub-model) have consistent LoRA parameter distribution and update requirements. In order to efficiently complete parameter synchronization and reduce communication overhead, these computing nodes loading the same node sub-model across node groups are divided into the same parameter update group.

[0093] The specific division method is as follows: in all the first computing node groups, the computing nodes with sorting index 1 (i.e., the master nodes of each group) are grouped together, the computing nodes with sorting index 2 are grouped together, ..., the computing nodes with sorting index k are grouped together.

[0094] For example, in a scenario comprising three first clusters of computing nodes, each cluster having three computing nodes, , and Update the group for one parameter; , and Update the group for one parameter; , and As a group.

[0095] In this way, the number of nodes in each parameter update group is equal to the number of nodes in the first computing node group (for example, if there are 3 node groups, then there are 3 nodes in each group). A master node is selected in each group (usually the first node in the group, i.e., the corresponding node in the master node group) to be responsible for scheduling, aggregating and broadcasting parameter updates.

[0096] Similarly, the same parameter update grouping is performed in the graphics processing unit (GPU) environment, grouping computation nodes that carry the same node sub-model into the same parameter update group. For example, nodes carrying the same node sub-model... Four computing nodes , , and They were grouped into the same parameter update group.

[0097] S502. For any parameter update group, determine the local low-rank adaptation parameters of each computing node based on the propagation results. Each computing node in the parameter update group vectorizes the local low-rank adaptation parameters and concatenates them to obtain the original parameter vector. The original parameter vector is then divided into multiple parameter fragments. The number of parameter fragments does not exceed the number of nodes in the parameter update group.

[0098] Each computation node calculates new values ​​for its local low-rank adaptation parameters using an optimizer (such as Adam or SGD) based on the parameter gradients in the propagation results. The low-rank adaptation parameters consist of two low-rank matrices A and B injected in the LoRA method, and their total number of parameters is much smaller than that of the original large model (e.g., only about 1%).

[0099] Within the same parameter update group, each computing node vectorizes its local low-rank adaptation parameters (i.e. matrices A and B): matrices A and B are expanded into one-dimensional vectors by rows or columns, and then the two vectors are concatenated end to end to form a long vector, denoted as the original parameter vector F.

[0100] Each computing node divides the original parameter vector F into multiple equal parameter segments, denoted as N. For example, if N is 3, then the original parameter vector of each computing node... .

[0101] When dividing the vector F into equal parts, the vector F is uniformly cut along its length, and each parameter segment contains consecutive parameter elements. The number of parameter segments N must satisfy the constraint that N does not exceed the total number of nodes in the parameter update group (i.e., the number of nodes in the first node group). This constraint ensures that in subsequent round-robin uploads, each segment has at least one node responsible for uploading, avoiding blank segments with no one uploading them.

[0102] S503. Each computing node in the parameter update group determines the parameter fragment number that needs to be uploaded in the current round according to the current round and the index of the node in the group, and uploads the parameter fragment corresponding to the parameter fragment number.

[0103] Each parameter update group contains multiple computation nodes, and the nodes within the group have a fixed sorting index (e.g., 1, 2, 3, ..., M). , and Taking a parameter update group as an example, where, The group index is 1. The group index is 2. The index is 3.

[0104] For the t-th training round (t starts counting from 0, i.e., t is 0 in the first training round), the node with index 'index' in the group needs to upload parameter fragment number i in this round according to the following rotation rule:

[0105] Here, `mod` is the modulo operation, and the result ranges from 0 to N-1. This rule ensures that in the same round, different nodes upload different parameter fragments; in different rounds, the same node uploads all fragments in turn, forming a loop.

[0106] For example, suppose N=3, and the node indices within the group are 1, 2, and 3. Then, in the four training rounds, the parameter fragments uploaded by the computation node with indices 1 within the group are respectively... , , , The parameter fragments uploaded by the compute node with index 2 within the group are as follows: , , , The parameter fragments uploaded by the compute node with index 3 within the group are as follows: , , , .

[0107] S504. The master node in the parameter update group accumulates the parameter fragments with the same parameter fragment number uploaded within the preset round range, and then divides the sum by the number of nodes that uploaded the parameter fragment to obtain the average parameter of the parameter fragment.

[0108] The preset round range indicates the number of rounds before performing a cumulative average of parameter segments. This value is manually set and can be 1, meaning a cumulative average is performed in each training round, or a value greater than 1, such as 3, meaning a cumulative average is performed every three training rounds.

[0109] A round-robin upload mechanism is adopted, where each node uploads only one parameter fragment per round. For example, when N=3, after 3 rounds of training, each fragment will be uploaded once by all 3 nodes in the group.

[0110] After receiving parameter fragments uploaded by each node in each round, the master node within the group maintains an independent accumulator and counter for each fragment number. When a fragment is received, the master node adds the value of the fragment to the corresponding accumulator and increments the counter by 1. When the preset value of the preset round range is reached, the master node calculates the average parameter of all parameter fragments corresponding to the same number: the sum in the accumulator divided by the counter value (i.e., the number of times the parameter fragment appears).

[0111] For example, if the preset epoch range is set to 4, and each original update parameter is divided into 3 parameter fragments with indices 1, 2, and 3 within the group, then the parameter fragment uploaded by the compute node with indic 1 within the group in the four training epochs will be as follows: , , , The parameter fragments uploaded by the compute node with index 2 within the group are as follows: , , , The parameter fragments uploaded by the compute node with index 3 within the group are as follows: , , , Based on this, parameter fragment numbered 1 is used. For example, if a parameter is uploaded four times in four training rounds, then the sum of these four parameter segments is divided by 4 to obtain the average parameter of that parameter segment. Similarly, the average parameters of the other two parameter segments can be obtained. , .

[0112] S505. Concatenate all average parameters to obtain updated parameters, and update the parameters of each calculation node in the parameter update group according to the updated parameters.

[0113] The master node within the group has obtained the average parameters of all parameter fragments (e.g., , , corresponding , , These segments correspond to the consecutive parts of the original parameter vector F after it has been divided into equal parts. The master node concatenates these average segments in ascending order of their segment numbers to form a new complete vector, denoted as . ,For example , , .

[0114] The master node will concatenate the complete update parameters. Distributed to all other compute nodes within the group via intra-group communication (such as broadcast or point-to-point transmission). For the master node itself, it is directly distributed according to... Overwrite the local parameter vector. Specifically, restore it by performing the inverse vectorization process: The LoRA parameters on the current node are split into matrices A and B, which match the original low-rank adaptation parameter structure. These new matrices are then used to replace the LoRA parameters on the current node. After the replacement, the low-rank adaptation parameters of all computing nodes in the group reach a consistent state (i.e., the global average). This constitutes a complete parameter synchronization update.

[0115] The above describes the parameter update method for the first model using the TEE side as an example, while the parameter update method for the second model on the GPU side is consistent with it. For each parameter update group divided within the graphics processing unit (GPU) environment and trusted execution environment, the operations in steps S502-S505 are performed to complete the parameter update. In the GPU environment, the parameters of the second model are updated according to the parameter gradient. Specifically, for any parameter update group, the local low-rank adaptation parameters of each computing node are determined based on the corresponding parameter gradient. Each computing node within the parameter update group vectorizes its local low-rank adaptation parameters and concatenates them to obtain the original parameter vector. The original parameter vector is then divided into multiple parameter segments; the number of parameter segments does not exceed the number of nodes within the parameter update group.

[0116] On the other hand, embodiments of the present invention also provide a model hybrid distributed dense-state fine-tuning system based on a trusted data space. Figure 6 This is a schematic diagram of the structure of the model hybrid distributed dense-state fine-tuning system based on trusted data space provided in the embodiments of the present invention, as shown below. Figure 6 As shown, the system includes: The training data acquisition module 100 is used to acquire plaintext data to be trained.

[0117] Each data provider preprocesses the raw data according to a unified data format and encrypts the processed data locally. The encrypted data is then transmitted to the data user's connector through their respective trusted data space connectors. The data user receives the encrypted data in a trusted execution environment, decrypts it in the encrypted memory of that environment, and integrates the decrypted plaintext data to obtain the plaintext data to be trained for subsequent model fine-tuning.

[0118] For example, when training a large medical consultation model, multiple hospitals act as data providers. Each hospital encrypts its consultation records in a unified format and sends them to the trusted execution environment (TEE) of the model trainer (such as a provincial medical AI center) through a trusted data space connector. The model trainer decrypts and integrates the data from all hospitals within the TEE to obtain a plaintext dataset containing medical consultation records from across the province, which is then used for fine-tuning the large model.

[0119] The model deployment module 200 is used to split the large model to be fine-tuned into a first model and a second model, deploy the first model in multiple first computing node groups in a trusted execution environment, and deploy the second model in multiple second computing node groups in a graphics processor environment; wherein, the first model includes a front continuous multi-layer and a rear continuous multi-layer of the large model, the second model includes the remaining middle continuous multi-layer of the large model, and the number of layers in the first model is less than the number of layers in the second model.

[0120] The large model to be fine-tuned is structurally divided into the first model. Second Model The first model part contains only a small portion of the entire large model, such as about 10%. The first model includes the first few layers (i.e., the first continuous layers near the input of the large model) and the last few layers (i.e., the last continuous layers near the output of the large model). It is responsible for the feature extraction of the input data and the mapping of the final output. This part involves sensitive information of the original data and intermediate features and must be placed in a secure environment. The second model part contains most of the middle layers of the model (i.e., the remaining middle continuous layers after removing the first and last continuous layers). It is responsible for the main nonlinear transformation and feature abstraction. It has a huge amount of computation, but its direct sensitivity to the data is relatively low.

[0121] The first model is deployed across multiple clusters of first compute nodes within a Trusted Execution Environment (TEE). The TEE provides a secure computing region based on hardware isolation, ensuring that code and data are not accessed externally during runtime. Within each cluster of first compute nodes, the first model is further divided into multiple model blocks, with each block assigned to a compute node (Rank) within the cluster. The compute nodes (Rank) in the TEE can be either CPUs or GPUs, but in practice, CPU computing resources are more commonly used.

[0122] Each primary computing node cluster contains multiple computing nodes. One of these nodes is selected as the master node of the cluster, responsible for task scheduling, data loading, result aggregation, and communication with other nodes. The remaining nodes in the cluster are slave nodes, which perform specific model block calculations in the forward and backward propagation processes. Multiple primary computing node clusters are coordinated through a master node (i.e., the master node of the master node cluster), forming a hierarchical distributed computing architecture.

[0123] The second model is deployed across multiple clusters of second compute nodes within a graphics processing unit (GPU) environment. The GPU environment contains substantial GPU computing resources to accelerate matrix operations. Similar to the trusted execution environment, each cluster of second compute nodes comprises multiple GPU nodes, with one node selected as the master node and the rest as slave nodes. The second model is layered into multiple model blocks, which are then sequentially assigned to the GPU nodes within the cluster. Likewise, the multiple clusters of second compute nodes are centrally scheduled and data distributed through a master node within the GPU environment (i.e., the master node of the master node cluster).

[0124] In one specific embodiment, in a trusted execution environment, three computing units form a first computing node group, and the number of first computing node groups is 3. In a graphics processor, three computing units form a second computing node group, and the number of second computing node groups is 4.

[0125] It should be noted that the batch size of the model training parameters must satisfy the following condition: the number of the first computing node group * the batch size must be divisible by the number of the second computing node group.

[0126] For example, in a trusted execution environment, the master node (first master node) in the master node group is... The other compute nodes in the master node group are , The master node in a group of slave nodes is From other computing nodes in the node group , Additionally, the master node in the node group is... From other computing nodes in the node group , So, in the first model During deployment, , and The corresponding sub-model is the node. , and ; , , Corresponding allocation node sub-model , and Other first-order computing node groups are similar.

[0127] Similarly, in a graphics processor environment, the master node (secondary master node) in the master node group is... The rest are slave nodes, and the master node of the other three slave node groups is... , and The node sub-model carried by each computing node in the master node group is as follows: , and .

[0128] The forward computation module 300 is used in a trusted execution environment, where each group of first computing nodes performs forward computation on the first model based on the plaintext data to be trained, performs dense processing on the computation results, obtains dense processing data, and transmits it to the graphics processor environment.

[0129] Before data training, the master node of the master node group in the first computing node group starts up the service and starts up each slave node group, and the graphics processor environment is also kept consistent.

[0130] Each first computing node group loads the model, and the master node of each first computing node group loads the data; in the graphics processor environment, each second computing node group loads the model.

[0131] Multiple computing nodes within each first computing node group are divided according to the structure of the first model. Based on the plaintext data to be trained, they sequentially complete the computation of their respective sub-models in a pipeline manner. Finally, the master node of each group collects the group feature map output by the group, which is the computation result of the forward computation.

[0132] The group feature maps of all first computing node groups are aggregated to the first master node (i.e., the master node of the master node group) in the trusted execution environment. The first master node performs uniform encrypted processing on these group feature maps (e.g., using a combination of differential privacy and homomorphic encryption), so that the processed data can still be used for subsequent computations without revealing plaintext features.

[0133] Then, the first master node transmits the encrypted data to the graphics processor environment via a secure communication protocol.

[0134] The backpropagation module 400 is used to decrypt the encrypted processing data in the graphics processor environment to obtain the first decrypted data. Each second computing node group performs forward calculation and backpropagation on the second model based on the first decrypted data to obtain the feature map gradient and parameter gradient. The feature map gradient is then processed in encrypted form to obtain the encrypted gradient data and transmit it to the trusted execution environment.

[0135] The graphics processing unit environment decrypts the received encrypted data, recovers the plaintext feature data that can be computed, and distributes the decrypted data, i.e. the first decrypted data, to each of the second computing node groups according to preset rules.

[0136] Based on the allocated data, each second computing node group sequentially performs forward computation and backward propagation on the second model portion deployed locally. Through serial transmission, it completes the computation and gradient solving of the model blocks carried by each node within the group, obtaining the feature map gradient corresponding to its group and the parameter gradients calculated by each computing node. The master node of each second computing node group collects the feature map gradients of its group and converges them to the second master node in the graphics processor environment. The second master node aggregates the feature map gradients of each group to obtain the complete feature map gradient.

[0137] Subsequently, the gradient of the feature map is processed in a dense state (e.g., using differential privacy and homomorphic encryption) to form dense gradient data, which is then transmitted back to the trusted execution environment via a secure communication protocol.

[0138] The parameter update module 500 is used to decrypt the encrypted gradient data to obtain the second decrypted data in a trusted execution environment, perform backpropagation on the first model based on the second decrypted data, and obtain the updated parameters of the first model based on the propagation result of the backpropagation; and to update the parameters of the second model based on the parameter gradient in a graphics processor environment.

[0139] The Trusted Execution Environment receives encrypted gradient data from the graphics processor environment, decrypts the data, and recovers the plaintext gradient data, i.e., the second decrypted data.

[0140] The first master node in the Trusted Execution Environment (TEE) continues backpropagation based on the second decrypted data (i.e., feature map gradients). Starting with these feature map gradients, it sequentially transmits gradients to each computing node within each first computing node group in the reverse order of the forward computation on the TEE side. Each computing node calculates the parameter gradients of the first model portion it carries based on the received gradients, until the backpropagation of all model blocks within the first computing node groups is completed. After backpropagation, the propagation result is obtained, which is the parameter gradients corresponding to each computing node in the first model portion.

[0141] For the first model part, each computing node in the first computing node group updates its LoRA parameters based on the backpropagation results, combined with the preset optimizer and learning rate.

[0142] For the second model part, each computing node in the second computing node group updates its LoRA parameters according to the parameter gradient, using the same optimizer and learning rate as the first model part.

[0143] It should be noted that both the first and second models use the low-rank adaptation (LoRA) fine-tuning method, which means that only the low-rank matrix parameters A and B injected into the model are updated, while the main parameters of the original large model are frozen.

[0144] The large model determination module 600 is used to obtain the parameter results of fine-tuning the target large model when the training rounds reach a preset number of rounds.

[0145] Repeat the operations of the forward computation module, backpropagation module, and parameter update module until the loss function converges or the preset number of training rounds is reached (each completion of forward computation, backpropagation, and gradient calculation constitutes a complete training round). This completes the fine-tuning of the entire large model. The first and second models after fine-tuning are then merged to obtain the target large model, which is then used for subsequent inference tasks.

[0146] In practical applications, the ultimate goal is to deploy the large-scale model in various industry business scenarios. For example, in a medical consultation system, the model outputs diagnostic suggestions or medication recommendations when the patient's symptom description is input; in a financial risk control scenario, the model outputs a risk assessment report when the company information is input; and in a government affairs Q&A system, the model outputs policy interpretations or service guidelines when the user's question is input.

[0147] As an optional implementation method, Figure 7 This is a schematic diagram of the data allocation submodule in the forward computing module provided in an embodiment of the present invention, as shown below. Figure 7 As shown, the forward computing module 300 includes: The training data partitioning submodule 3001 is used to load the plaintext data to be trained into the master nodes of each first computing node group, sort the plaintext data to be trained according to the same random seed, obtain the sorting result, and divide the sorting result according to the preset batch size to obtain multiple data batches.

[0148] In the trusted execution environment, all master nodes in the first cluster of computing nodes load the same plaintext data to be trained. To ensure that all nodes process the data in a completely consistent order, each master node initializes its own pseudo-random number generator using the same random seed (e.g., a common seed broadcast by the first master node). This ensures that when each master node independently randomizes the dataset, it obtains the exact same sorting result.

[0149] Each master node divides the sorting results into multiple data batches according to a preset batch size. Each batch contains a fixed number of consecutive samples, numbered sequentially as batch 0, batch 1, batch 2, and so on, until all data is covered.

[0150] The training data allocation submodule 3002 is used by the first master node in the trusted execution environment to allocate data batch intervals to each first computing node group according to the number of first computing node groups and the number of data batches.

[0151] The first master node obtains the total number of generated data batches (e.g., if there are 1536 plaintext data entries to be trained, and the preset batch size is 128, then the total number of data batches is 12). Simultaneously, the first master node determines the number of first computing node groups currently participating in the training (e.g., 3). The first master node divides the total number of data batches equally according to the number of node groups, and calculates the length of the consecutive batch interval that each first computing node group should be responsible for. For example, if there are 12 total batches and 3 node groups, then each node group is responsible for 4 consecutive batches.

[0152] The first master node allocates consecutive intervals of starting batch numbers according to the inherent order of the node groups (e.g., master node group, slave node group 1, slave node group 2, etc.): node group 1 is responsible for batches 0 to 3, node group 2 is responsible for batches 4 to 7, and node group 3 is responsible for batches 8 to 11. If the total number of data batches is not divisible by the number of node groups, the remaining batches are allocated sequentially to the first few node groups to ensure that each node group receives at least an integer number of batches and that the batch intervals do not overlap.

[0153] The first master node will broadcast the allocation results (i.e., the start and end numbers of the batch interval for each node group) to the master nodes of each first computing node group via internal communication.

[0154] The training data determination submodule 3003 is used by the master nodes of each first computing node group to obtain the training node data from the plaintext data to be trained according to the data batch interval to which they belong; wherein, the first general master node is the master node of the master node group in the trusted execution environment.

[0155] After completing global sorting and batch partitioning, each master node in the first computing node group has a complete batch list locally (e.g., batches 0 to 11). When the first master node notifies each master node of the batch range (e.g., node group 1 is responsible for batches 0 to 3), each master node extracts only the sample data corresponding to those batches within its own range from its locally maintained batch list; these are the data to be trained. For example, the master node of node group 1 loads all samples from batches 0, 1, 2, and 3; the master node of node group 2 loads batches 4, 5, 6, and 7; and the master node of node group 3 loads batches 8, 9, 10, and 11. Each master node organizes the extracted data into an internal training queue, which serves as the input data source for subsequent forward computations within its group.

[0156] The first master node is the master node of the master node group in the trusted execution environment (e.g., It is responsible for global scheduling, batch allocation, and aggregation and distribution of dense data.

[0157] As an optional implementation method, Figure 8 This is a schematic diagram of the structure of the sub-module used for forward computation in the forward computation module provided in an embodiment of the present invention, as shown below. Figure 8 As shown, the forward computing module 300 also includes: The forward computation submodule 3004 is used as the first computation node of each first computation node group. It performs forward computation of the corresponding node sub-model based on the data of the node to be trained, and passes the computation result to the next computation node in the group.

[0158] Each group of first computing nodes obtains the training data for the nodes it is responsible for, based on the first model. The partitioning method involves the first computation node (usually the master node) carrying the first few layers of the first model part (e.g., the first block of the model). This computation node takes the data assigned to the training node in its own group as input, loads the model parameters on this node (including the original large model frozen parameters and LoRA low-rank adaptation parameters), performs forward computation, and obtains the output result, namely the intermediate feature map. This feature map is a high-dimensional representation of the input data after being extracted by the first block of the model.

[0159] After completing the computation, the first computing node transmits the intermediate feature map to the second computing node in the cluster via inter-node communication (e.g., high-speed interconnect or message passing interface). That is, the bearing model The computing nodes.

[0160] The group feature map calculation submodule 3005 is used by the next calculation node to perform forward calculation of the corresponding node sub-model based on the received calculation result, and pass the calculation result to the next calculation node in the group until the last calculation node in the group completes the forward calculation, obtains the group feature map, and sends it to the master node in the group.

[0161] The second computing node in the group Receive the intermediate feature map passed from the first computing node, use this feature map as the input of this node, and load the node sub-model of the first model part carried by this node (e.g., The forward computation is performed, generating a new intermediate feature map, which is then passed on to the third computation node in the swarm. That is, the carrier sub-model The nodes. And so on, each computing node (the third, the fourth... up to the second to last) performs the same operation: receives the feature map from the previous node, calculates the sub-model of this node, and then passes the output feature map to the next node.

[0162] When it is passed to the last computing node in this group, such as At that time, the node receives the feature map from the previous node and loads the last sub-model of the first model part it carries (e.g., This completes the final forward computation. The output of this node is the final feature map of this first computation node group, called the group feature map. The group feature map represents a high-dimensional abstract representation of the data batch processed by this group after passing through all sub-models of the first model part.

[0163] After the last computing node completes its calculation, it sends the group feature map to the group's master node (i.e., the first computing node in the group, which is also the group's scheduling and aggregation node) via intra-group communication.

[0164] The feature transmission submodule 3006 is used by the master node of each first computing node group to transmit the group feature map of its group to the first general master node.

[0165] After receiving the group feature map sent by the last computing node in its group, the master node of each first computing node group (i.e., the first computing node in the group) sends the group feature map of its group to the first master node (i.e., the master node of the master node group) through the internal high-speed communication network. The first master node is responsible for receiving and aggregating the group feature maps from all the first computing nodes to form a complete set of intermediate results.

[0166] The dense state processing submodule 3007 is used by the first master node to perform dense state processing on all group feature maps to obtain dense state processed data.

[0167] The first master node aggregates group feature maps from each of the first compute node groups. These feature maps are in plaintext format and contain intermediate features extracted from the TEE-side model. To protect data privacy when transmitting data to the graphics processor environment and prevent theft or leakage during cross-environment transmission, the first master node performs encrypted processing on these group feature maps.

[0168] Encryption processing includes, but is not limited to, two techniques: differential privacy and homomorphic encryption. Differentiated privacy adds random noise following a specific distribution to the feature map, making it impossible for attackers to distinguish the contribution of individual samples, thus protecting the privacy of the original data. Homomorphic encryption encrypts the feature map using homomorphic encryption algorithms, allowing the encrypted data to be directly used for preset computational operations (such as addition and multiplication) in the graphics processing unit (GPU) environment without decryption. The decrypted result is identical to the result of directly calculating the plaintext. In practical implementations, these techniques can be flexibly combined according to security requirements and computational overhead.

[0169] After encrypted processing, the original plaintext group feature map is converted into encrypted processed data, which can be securely transmitted to the graphics processing unit (GPU) environment without revealing valid information. The first master node then sends the encrypted processed data to the second master node in the GPU environment via a secure communication protocol (such as gRPC). .

[0170] As an optional implementation method, Figure 9 This is a schematic diagram of the backpropagation module provided in an embodiment of the present invention, as shown below. Figure 9 As shown, the backpropagation module 400 includes: The decryption data allocation submodule 4001 is used in the second master node of the graphics processor environment to split and sequentially mark the first decryption data according to the number of each second computing node group, and then allocate it to each second computing node group.

[0171] Second Master Node The encrypted processing data is received. The second master node uses the pre-negotiated decryption key (or the corresponding private key) to decrypt the encrypted processing data, recovering the plaintext feature map data, i.e., the first decrypted data. This first decrypted data corresponds to the plaintext form of the group feature map output by each first computing node group on the TEE side. However, since noise has been added before transmission using differential privacy, the sensitive information of the original sample will not be exposed even after decryption.

[0172] The second master node obtains the number of second computing node clusters (denoted as M, e.g., 4 node clusters) in the current graphics processor environment participating in training. The second master node then splits the decrypted feature map data into M sub-data blocks. The splitting can be based on the uniform division of the feature map along the sample dimension; for example, if the first decrypted data contains feature maps corresponding to N samples, then each sub-block is allocated approximately N / M sample feature maps. Simultaneously, the second master node sequentially labels each split sub-block (e.g., numbered 0, 1, ..., M-1) so that the original order can be restored during subsequent aggregation.

[0173] The second master node will split and mark the sub-data blocks and distribute them to the corresponding master nodes of the second computing node group. Each master node of the second computing node group receives its own sub-data block as input for subsequent forward computations of its group.

[0174] The prediction result calculation submodule 4002 is used by the first computing node in each second computing node group to perform forward calculation of the corresponding node model based on the allocated data of the group, and pass the calculation result to the next computing node in the group, and so on until the last computing node in the group completes the forward calculation to obtain the prediction result of the group.

[0175] The first computing node (usually the master node) of each second computing node group loads the first model block of the second model part carried by this node (e.g., the first model block of the second model part carried by this node) according to the allocated sub-data block (i.e., the decrypted feature map data). ), perform forward computation to generate intermediate feature maps.

[0176] After completing the computation, the node transmits the intermediate feature map to the second computation node in the cluster via a high-speed communication interface (such as NVLink between GPUs or network communication). The second computation node receives the intermediate feature map from the previous node, uses it as input, loads the next model block of the second model part it carries, continues the forward computation, and outputs a new intermediate feature map, which is then passed to the third computation node. This process continues, with each subsequent computation node performing the same operation: receiving the output from the previous node, performing the forward computation of its own model block, and passing the result to the next node, until the last computation node in the cluster is reached. The last computation node receives the intermediate feature map from the previous node, loads the last model block of the second model part it carries, completes the final forward computation, and outputs the prediction result corresponding to the cluster (e.g., classification probability, regression value, etc.).

[0177] Taking a master node cluster as an example, calculate The model will then provide the calculation results to... , calculate The model will then provide the calculation results to... , calculate The model yields prediction results.

[0178] The backward gradient calculation submodule 4003 is used by the last calculation node in each second calculation node group to calculate the loss value based on the prediction result, calculate the gradient corresponding to the node based on the loss value, and pass the gradient to the previous calculation node. After receiving the gradient, each calculation node calculates its own gradient and passes its own gradient to the previous calculation node until it is passed to the first calculation node in the group to complete the gradient calculation and obtain the backward gradient corresponding to the group.

[0179] The last computing node in each second computing node group calculates the loss value based on the prediction result and the true label, and calculates the gradient of its own output based on the loss value to obtain the initial gradient of the output of this node.

[0180] Then, the last computation node passes the calculated gradient to the previous computation node. Each non-first node in the group (from the last node to the second node), after receiving the gradient from the next node, performs the following operations: using the received gradient, combined with the intermediate state saved by the node in the forward computation (such as the input feature map), calculates the parameter gradient of the model block carried by the node (for subsequent updates of LoRA parameters); at the same time, it calculates the gradient at the input of the node (i.e., the partial derivative with respect to the output feature map of the previous node), and continues to pass this input gradient to the previous computation node.

[0181] The above process proceeds sequentially: the second-to-last node receives the gradient passed from the last node, calculates its own parameter gradient and input gradient, and then passes the input gradient to the third-to-last node; and so on, until it reaches the first computation node in the group. The first computation node, upon receiving the gradient from the second node, also calculates its own parameter gradient and input gradient. The input gradient of the first computation node is the inverse gradient of the input feature map of the group, which is the inverse gradient corresponding to the group (used to be passed back to the TEE environment).

[0182] Through the above chain-like backpropagation, each second computing node group can obtain the parameter gradients of each computing node in its own group (used to update the LoRA parameters of each model block in the second model part), as well as the backpropagation gradient of the input feature map of its own group (i.e. the backpropagation output of the entire group, for subsequent cross-environment transmission).

[0183] The gradient aggregation submodule 4004 is used by the master nodes of each second computing node group to collect the backward gradients of their group and transmit them to the second master node. The second master node aggregates the backward gradients in the order of their labels to obtain the feature map gradients. The second master node is the master node of the master node group in the graphics processor environment.

[0184] The master node of each second compute node group (i.e., the first compute node in its group) collects the inverse gradient of the input feature map through intra-group communication (if the master node already possesses this gradient, no additional collection is required), and then sends this inverse gradient of its group to the second master node via a high-speed network. The second master node is the master node of the master node group in the graphics processor environment.

[0185] After receiving the backward gradients from all the second computing nodes, the second master node concatenates and aggregates these backward gradients in their original order according to the marking order (i.e., the sequential marking of each sub-block during splitting). The aggregated result is the feature map gradient. This gradient data corresponds completely to the group feature map initially sent by the TEE side in spatial dimension, ensuring the correct alignment between the gradient and the feature map during subsequent backward propagation on the TEE side.

[0186] As an optional implementation method, Figure 10 This is a schematic diagram of the parameter update module provided in an embodiment of the present invention, as shown below. Figure 10 As shown, the parameter update module 500 includes: The parameter update group division submodule 5001 is used to divide computing nodes with the same node sub-model loaded in each first computing node group in the trusted execution environment into the same parameter update group; and to divide computing nodes with the same node sub-model loaded in each second computing node group in the graphics processor environment into the same parameter update group.

[0187] Since each first computing node group carries the first model part according to the same partitioning method, for example, the first computing node in each group carries the first sub-model ( The second computing node carries the second sub-model ( And so on. Therefore, in different first computing node groups, computing nodes with the same sorting position (i.e., loading the same node sub-model) have consistent LoRA parameter distribution and update requirements. In order to efficiently complete parameter synchronization and reduce communication overhead, these computing nodes loading the same node sub-model across node groups are divided into the same parameter update group.

[0188] The specific division method is as follows: in all the first computing node groups, the computing nodes with sorting index 1 (i.e., the master nodes of each group) are grouped together, the computing nodes with sorting index 2 are grouped together, ..., the computing nodes with sorting index k are grouped together.

[0189] For example, in a scenario comprising three first clusters of computing nodes, each cluster having three computing nodes, , and Update the group for one parameter; , and Update the group for one parameter; , and As a group.

[0190] In this way, the number of nodes in each parameter update group is equal to the number of nodes in the first computing node group (for example, if there are 3 node groups, then there are 3 nodes in each group). A master node is selected in each group (usually the first node in the group, i.e., the corresponding node in the master node group) to be responsible for scheduling, aggregating and broadcasting parameter updates.

[0191] Similarly, the same parameter update grouping is performed in the graphics processing unit (GPU) environment, grouping computation nodes that carry the same node sub-model into the same parameter update group. For example, nodes carrying the same node sub-model... Four computing nodes , , and They were grouped into the same parameter update group.

[0192] The parameter fragment segmentation submodule 5002 is used to determine the local low-rank adaptation parameters of each computing node for any parameter update group based on the propagation results. Each computing node in the parameter update group vectorizes the local low-rank adaptation parameters and concatenates them to obtain the original parameter vector. The original parameter vector is then divided into multiple parameter fragments. The number of parameter fragments does not exceed the number of nodes in the parameter update group.

[0193] Each computing node calculates new values ​​for its local low-rank adaptation parameters using an optimizer (such as Adam or SGD) based on the parameter gradients obtained from backpropagation. The low-rank adaptation parameters consist of two low-rank matrices A and B injected in the LoRA method, and their total number of parameters is much smaller than that of the original large model (e.g., only about 1%).

[0194] Within the same parameter update group, each computing node vectorizes its local low-rank adaptation parameters (i.e. matrices A and B): matrices A and B are expanded into one-dimensional vectors by rows or columns, and then the two vectors are concatenated end to end to form a long vector, denoted as the original parameter vector F.

[0195] Each computing node divides the original parameter vector F into multiple equal parameter segments, denoted as N. For example, if N is 3, then the original parameter vector of each computing node... .

[0196] When dividing the vector F into equal parts, the vector F is uniformly cut along its length, and each parameter segment contains consecutive parameter elements. The number of parameter segments N must satisfy the constraint that N does not exceed the total number of nodes in the parameter update group (i.e., the number of nodes in the first node group). This constraint ensures that in subsequent round-robin uploads, each segment has at least one node responsible for uploading, avoiding blank segments with no one uploading them.

[0197] The parameter fragment upload submodule 5003 is used to determine the parameter fragment number that each computing node in the parameter update group needs to upload in the current round according to the current round and the index of the node in the group, and then upload the parameter fragment corresponding to the parameter fragment number.

[0198] Each parameter update group contains multiple computation nodes, and the nodes within the group have a fixed sorting index (e.g., 1, 2, 3, ..., M). , and Taking a parameter update group as an example, where, The group index is 1. The group index is 2. The index is 3.

[0199] For the t-th training round (t starts counting from 0, i.e., t is 0 in the first training round), the node with index 'index' in the group needs to upload parameter fragment number i in this round according to the following rotation rule:

[0200] Here, `mod` is the modulo operation, and the result ranges from 0 to N-1. This rule ensures that in the same round, different nodes upload different parameter fragments; in different rounds, the same node uploads all fragments in turn, forming a loop.

[0201] For example, suppose N=3, and the node indices within the group are 1, 2, and 3. Then, in the four training rounds, the parameter fragments uploaded by the computation node with indices 1 within the group are respectively... , , , The parameter fragments uploaded by the compute node with index 2 within the group are as follows: , , , The parameter fragments uploaded by the compute node with index 3 within the group are as follows: , , , .

[0202] The average parameter calculation submodule 5004 is used by the master node in the parameter update group to accumulate the parameter fragments with the same parameter fragment number uploaded within the preset round range, and then divide by the number of nodes that uploaded the parameter fragment to obtain the average parameter of the parameter fragment.

[0203] The preset round range indicates the number of rounds before performing a cumulative average of parameter segments. This value is manually set and can be 1, meaning a cumulative average is performed in each training round, or a value greater than 1, such as 3, meaning a cumulative average is performed every three training rounds.

[0204] A round-robin upload mechanism is adopted, where each node uploads only one parameter fragment per round. For example, when N=3, after 3 rounds of training, each fragment will be uploaded once by all 3 nodes in the group.

[0205] After receiving parameter fragments uploaded by each node in each round, the master node within the group maintains an independent accumulator and counter for each fragment number. When a fragment is received, the master node adds the value of the fragment to the corresponding accumulator and increments the counter by 1. When the preset value of the preset round range is reached, the master node calculates the average parameter of all parameter fragments corresponding to the same number: the sum in the accumulator divided by the counter value (i.e., the number of times the parameter fragment appears).

[0206] For example, if the preset epoch range is set to 4, and each original update parameter is divided into 3 parameter fragments with indices 1, 2, and 3 within the group, then the parameter fragment uploaded by the compute node with indic 1 within the group in the four training epochs will be as follows: , , , The parameter fragments uploaded by the compute node with index 2 within the group are as follows: , , , The parameter fragments uploaded by the compute node with index 3 within the group are as follows: , , , Based on this, parameter fragment numbered 1 is used. For example, if a parameter is uploaded four times in four training rounds, then the sum of these four parameter segments is divided by 4 to obtain the average parameter of that parameter segment. Similarly, the average parameters of the other two parameter segments can be obtained. , .

[0207] The parameter update submodule 5005 is used to concatenate all average parameters to obtain updated parameters, and then update the parameters of each computing node in the parameter update group based on the updated parameters.

[0208] The master node within the group has obtained the average parameters of all parameter fragments (e.g., , , corresponding , , These segments correspond to the consecutive parts of the original parameter vector F after it has been divided into equal parts. The master node concatenates these average segments in ascending order of their segment numbers to form a new complete vector, denoted as . ,For example , , .

[0209] The master node will concatenate the complete update parameters. Distributed to all other compute nodes within the group via intra-group communication (such as broadcast or point-to-point transmission). For the master node itself, it is directly distributed according to... Overwrite the local parameter vector. Specifically, restore it by performing the inverse vectorization process: The LoRA parameters on the current node are split into matrices A and B, which match the original low-rank adaptation parameter structure. These new matrices are then used to replace the LoRA parameters on the current node. After the replacement, the low-rank adaptation parameters of all computing nodes in the group reach a consistent state (i.e., the global average). This constitutes a complete parameter synchronization update.

[0210] The above describes the parameter update method for the first model using the TEE side as an example, while the parameter update method for the second model on the GPU side is consistent with it. For each parameter update group divided within the graphics processing unit (GPU) environment and trusted execution environment, the operations in submodules 5002-5005 are executed to complete the parameter update. In the GPU environment, the parameters of the second model are updated according to the parameter gradient. Specifically, for any parameter update group, the local low-rank adaptation parameters of each computing node are determined based on the corresponding parameter gradient. Each computing node within the parameter update group vectorizes its local low-rank adaptation parameters and concatenates them to obtain the original parameter vector. The original parameter vector is then divided into multiple parameter segments; the number of parameter segments does not exceed the number of nodes within the parameter update group.

[0211] The above technical solution has the following beneficial effects: By splitting the large model into a first model part deployed in a trusted execution environment and a second model part deployed in a graphics processing unit environment, sensitive computations are placed in a secure area while a large number of matrix operations are completed by high-performance GPUs, avoiding the performance bottleneck of fully dense computation; by using serial forward and backward propagation within each computing node group and transmitting only feature maps and gradients, combined with low-rank adaptation fine-tuning that only updates a small number of parameters, the amount of communication between nodes is significantly reduced; by dividing the parameter vector equally during the parameter update stage and uploading different segments according to the rotation rule, and synchronizing the parameters after aggregation and averaging, the amount of communication in a single round is compressed to a fraction of the original parameters, greatly reducing the distributed synchronization overhead; by encrypting the transmission of data to the trusted execution environment from each data provider and transmitting densely processed data between the TEE and the GPU, the privacy and security of data are ensured throughout the entire process of transmission, storage, and computation.

[0212] The above-described specific embodiments of the invention further illustrate the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above content is only for specific embodiments of the invention and is not intended to limit the scope of protection of the invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A hybrid distributed dense-state fine-tuning method for models based on trusted data space, characterized in that, include: Obtain the plaintext data to be trained; The large model to be fine-tuned is split into a first model and a second model. The first model is deployed in multiple first computing node groups in a trusted execution environment, and the second model is deployed in multiple second computing node groups in a graphics processor environment. The first model includes the front and rear continuous multi-layers of the large model, and the second model includes the remaining middle continuous multi-layers of the large model. The number of layers in the first model is less than the number of layers in the second model. In a trusted execution environment, each first computing node group performs forward computation on the first model based on the plaintext data to be trained, performs dense processing on the computation results, obtains dense processing data, and transmits it to the graphics processor environment. In the graphics processor environment, the encrypted processing data is decrypted to obtain first decrypted data. Each second computing node group performs forward calculation and backward propagation on the second model based on the first decrypted data to obtain feature map gradient and parameter gradient. The feature map gradient is then processed in an encrypted manner to obtain encrypted gradient data, which is then transmitted to the trusted execution environment. In a trusted execution environment, the dense gradient data is decrypted to obtain second decrypted data. The first model is backpropagated based on the second decrypted data, and the updated parameters of the first model are obtained based on the backpropagation results. In a graphics processor environment, the parameters of the second model are updated based on the parameter gradient. The step of obtaining the updated parameters of the first model based on the propagation results of backpropagation includes: dividing computing nodes in each first computing node group within the trusted execution environment that have loaded the same node sub-model into the same parameter update group; for any parameter update group, determining the local low-rank adaptation parameters of each computing node based on the propagation results; each computing node in the parameter update group vectorizing the local low-rank adaptation parameters and concatenating them to obtain the original parameter vector; dividing the original parameter vector into multiple parameter fragments; wherein the number of parameter fragments does not exceed the number of nodes in the parameter update group; each computing node in the parameter update group determining the parameter fragment number to be uploaded in the current round according to the current round and the index of the node in the group, and uploading the parameter fragment corresponding to the parameter fragment number; the master node in the parameter update group accumulating the parameter fragments with the same parameter fragment number uploaded within the preset round range, dividing by the number of nodes that uploaded the parameter fragment, to obtain the average parameter of the parameter fragment; concatenating all the average parameters to obtain the updated parameters, and updating the parameters of each computing node in the parameter update group according to the updated parameters. When the training rounds reach the preset number of rounds, the parameter results of the fine-tuning of the target large model are obtained.

2. The model hybrid distributed dense-state fine-tuning method based on trusted data space according to claim 1, characterized in that, Before performing forward computation on the first model, the following steps are included: The master node of each first computing node group loads the plaintext data to be trained, sorts the plaintext data to be trained according to the same random seed, obtains the sorting result, and divides the sorting result according to the preset batch size to obtain multiple data batches. The first master node in the trusted execution environment allocates data batch ranges to each first computing node group based on the number of the first computing node groups and the number of data batches. Each master node of the first computing node group obtains the training node data from the plaintext data to be trained according to its data batch interval; wherein, the first general master node is the master node of the master node group in the trusted execution environment.

3. The model hybrid distributed dense-state fine-tuning method based on trusted data space according to claim 2, characterized in that, The step of performing forward computation on the first model and performing dense-state processing on the computation results to obtain dense-state processed data includes: The first computing node in each first computing node group performs forward computation of the corresponding node sub-model based on the data of the node to be trained, and passes the computation result to the next computing node in the group; The next computing node performs forward computation of the corresponding node sub-model based on the received computation result, and passes the computation result to the next computing node in the group until the last computing node in the group completes the forward computation, obtains the group feature map, and sends it to the master node in the group. The master node of each first computing node group transmits the group feature map of its group to the first general master node; The first master node performs dense state processing on all group feature maps to obtain dense state processed data.

4. The model hybrid distributed dense-state fine-tuning method based on trusted data space according to claim 1, characterized in that, Each of the second computing node groups performs forward computation and backward propagation on the second model based on the first decrypted data to obtain the feature map gradient, including: The second master node in the graphics processor environment splits and sequentially marks the first decrypted data according to the number of each second computing node group, and then distributes it to each second computing node group. The first computing node in each second computing node group performs forward calculation of the corresponding node model based on the allocated data of the group, and passes the calculation result to the next computing node in the group, and so on until the last computing node in the group completes the forward calculation and obtains the prediction result of the group. The last computing node in each second computing node group calculates the loss value based on the prediction result, and calculates the gradient corresponding to the node based on the loss value. The gradient is then passed to the previous computing node. After receiving the gradient, each computing node calculates its own gradient and passes it to the previous computing node until it is passed to the first computing node in the group to complete the gradient calculation and obtain the inverse gradient corresponding to the group. Each master node of the second computing node group collects the inverse gradient of its group and transmits it to the second master node. The second master node aggregates the inverse gradients in the order of their labels to obtain the feature map gradients. The second master node is the master node of the master node group in the graphics processor environment.

5. A model-hybrid distributed dense-state fine-tuning system based on a trusted data space, characterized in that, include: The training data acquisition module is used to acquire plaintext data to be trained. The model deployment module is used to split the large model to be fine-tuned into a first model and a second model, deploy the first model in multiple first computing node groups in a trusted execution environment, and deploy the second model in multiple second computing node groups in a graphics processor environment; wherein, the first model includes the front continuous multi-layer and the rear continuous multi-layer of the large model, the second model includes the remaining middle continuous multi-layer of the large model, and the number of layers of the first model is less than the number of layers of the second model. The forward computation module is used in a trusted execution environment, where each group of first computing nodes performs forward computation on the first model based on the plaintext data to be trained, performs dense processing on the computation results, obtains dense processing data, and transmits it to the graphics processor environment. The backpropagation module is used to decrypt the encrypted processing data in the graphics processor environment to obtain first decrypted data. Each second computing node group performs forward calculation and backpropagation on the second model according to the first decrypted data to obtain feature map gradient and parameter gradient. The feature map gradient is then processed in encrypted form to obtain encrypted gradient data and transmit it to the trusted execution environment. The parameter update module is used to decrypt the dense gradient data to obtain second decrypted data in a trusted execution environment, perform backpropagation on the first model based on the second decrypted data, and obtain the updated parameters of the first model based on the backpropagation results; and to update the parameters of the second model based on the parameter gradient in a graphics processor environment. The parameter update module includes: a parameter update group partitioning submodule, used to partition computing nodes with the same node submodel loaded in each first computing node group in the trusted execution environment into the same parameter update group; and to partition computing nodes with the same node submodel loaded in each second computing node group in the graphics processor environment into the same parameter update group; and a parameter fragment segmentation submodule, used to determine the local low-rank adaptation parameters of each computing node in any parameter update group according to the propagation result, wherein each computing node in the parameter update group vectorizes the local low-rank adaptation parameters and concatenates them to obtain the original parameter vector, and divides the original parameter vector into multiple parameter fragments; wherein the number of parameter fragments does not exceed the specified number of parameter fragments. The parameter update group has the following submodules: Number of nodes in the parameter update group; Parameter fragment upload submodule, used by each computing node in the parameter update group to determine the parameter fragment number to be uploaded in the current round according to the current round and the node's index in the group, and then upload the parameter fragment corresponding to that parameter fragment number; Average parameter calculation submodule, used by the main node in the parameter update group to accumulate the parameter fragments with the same parameter fragment number uploaded within a preset round range, and then divide by the number of nodes that uploaded that parameter fragment to obtain the average parameter of that parameter fragment; Parameter update submodule, used to concatenate all average parameters to obtain the update parameters, and update the parameters of each computing node in the parameter update group according to the update parameters; The large model determination module is used to obtain the parameter results of fine-tuning the target large model when the training rounds reach a preset number of rounds.

6. The model hybrid distributed dense-state fine-tuning system based on trusted data space according to claim 5, characterized in that, The forward computation module includes: The training data partitioning submodule is used to load the training plaintext data into the master node of each first computing node group, sort the training plaintext data according to the same random seed, obtain the sorting result, and divide the sorting result according to the preset batch size to obtain multiple data batches. The training data allocation submodule is used by the first master node in the trusted execution environment to allocate data batch ranges to each first computing node group according to the number of the first computing node groups and the number of data batches. The training data determination submodule is used by the master nodes of each first computing node group to obtain the training node data from the plaintext data to be trained according to the data batch interval to which they belong; wherein, the first master node is the master node of the master node group in the trusted execution environment.

7. The model hybrid distributed dense-state fine-tuning system based on trusted data space according to claim 6, characterized in that, The forward computation module includes: The forward computation submodule is used by the first computation node of each first computation node group to perform forward computation of the corresponding node sub-model based on the data of the node to be trained, and pass the computation result to the next computation node in the group. The group feature map calculation submodule is used by the next calculation node to perform forward calculation of the corresponding node sub-model based on the received calculation result, and pass the calculation result to the next calculation node in the group until the last calculation node in the group completes the forward calculation, obtains the group feature map, and sends it to the master node in the group. The feature transmission submodule is used by the master node of each first computing node group to transmit the group feature map of its own group to the first general master node. The dense state processing submodule is used by the first master node to perform dense state processing on all group feature maps to obtain dense state processed data.

8. The model hybrid distributed dense-state fine-tuning system based on trusted data space according to claim 5, characterized in that, The backpropagation module includes: The decryption data distribution submodule is used in the second master node of the graphics processor environment to split and sequentially mark the first decryption data according to the number of each second computing node group and distribute it to each second computing node group. The prediction result calculation submodule is used by the first computing node in each second computing node group to perform forward calculation of the corresponding node model based on the allocated data of the group, and pass the calculation result to the next computing node in the group, and so on until the last computing node in the group completes the forward calculation to obtain the prediction result of the group. The inverse gradient calculation submodule is used to calculate the loss value based on the prediction result by the last calculation node in each second calculation node group, and calculate the gradient corresponding to the node based on the loss value. The gradient is then passed to the previous calculation node. After receiving the gradient, each calculation node calculates its own gradient and passes it to the previous calculation node until it is passed to the first calculation node in the group to complete the gradient calculation and obtain the inverse gradient corresponding to the group. The gradient aggregation submodule is used by the master nodes of each second computing node group to collect the inverse gradients of their group and transmit them to the second total master node. The second total master node aggregates the inverse gradients according to the label order to obtain the feature map gradient. The second total master node is the master node of the master node group in the graphics processor environment.

Images