Blockchain-based large model training method, apparatus and device, and storage medium

Abstract

Provided in the present application are a blockchain-based large model training method, apparatus and device, and a storage medium. A blockchain comprises a central node and a plurality of consortium nodes, wherein each consortium node corresponds to one data terminal. The method comprises: a central node acquiring a pre-trained large model; acquiring an adapter module corresponding to the large model, and sending the adapter module to each consortium node; each consortium node adding the adapter module to a trusted execution environment; acquiring encrypted training data from a blockchain; in the trusted execution environment, decrypting the encrypted training data, and using decrypted training data to train fine-tuning model parameters of the adapter module, and uploading updated fine-tuning model parameters to the central node; the central node performing aggregation processing on the fine-tuning model parameters; and on the basis of aggregated fine-tuning model parameters, updating the pre-trained large model to obtain an updated large model until the updated large model converges.

Description

Blockchain-based large model training methods, devices, equipment, and storage media

[0001] This application claims priority to Chinese Patent Application No. 202410854205.X, filed on June 27, 2024, entitled “Method, Apparatus, Device and Storage Medium for Large Model Training Based on Blockchain”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of big data technology, specifically to a blockchain-based method, apparatus, device, and storage medium for training large models. Background Technology

[0003] With the further development of large-scale modeling technology and big data, prioritizing data privacy and security has become a global trend. In the financial sector, an increasing number of generative artificial intelligence applications based on large-scale models are being implemented, and the financial data involved is often closely related to personal privacy and industry secrets.

[0004] In related technologies, to ensure data security, the traditional federated learning approach is adopted, which uses multiple devices or data sources to collaboratively train a large machine learning model, and uses multi-party secure computation protocols such as homomorphic encryption to encrypt the data during the model training process.

[0005] However, there are technical issues regarding data security during the training of large models. Summary of the Invention

[0006] This application provides a blockchain-based method, apparatus, device, and storage medium for training large models, thereby solving the problem of low data security during the training of large models.

[0007] Firstly, this application provides a large-scale model training method based on blockchain, wherein the blockchain includes a central node and multiple consortium nodes; wherein each consortium node corresponds to a data terminal, and the method includes:

[0008] The central node acquires a pre-trained large model;

[0009] The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node; wherein, the adapter module includes the fine-tuned model parameters of the pre-trained large model;

[0010] Each alliance node adds the adapter module to the trusted execution environment;

[0011] Each consortium node acquires encrypted training data on the blockchain; wherein, the training data is acquired by the data terminal corresponding to each consortium node, encrypted, and then sent to the consortium node, which stores it on the blockchain.

[0012] Each consortium node, in a trusted execution environment, decrypts the encrypted training data and uses the decrypted training data to train the fine-tuning model parameters of the adapter module to obtain updated fine-tuning model parameters; each consortium node then uploads the updated fine-tuning model parameters to the central node.

[0013] The central node aggregates the fine-tuned model parameters uploaded by all alliance nodes to obtain aggregated fine-tuned model parameters.

[0014] The central node updates the pre-trained large model based on the aggregated fine-tuned model parameters to obtain the updated large model.

[0015] If the updated large model does not converge, the central node continues to use the updated large model as the pre-trained large model and restarts the steps of "obtaining the adapter module corresponding to the large model and sending the adapter module to each alliance node" until the updated large model converges to complete the training of the large model.

[0016] This application proposes a blockchain-based method for training large models. The entire training process is conducted on the blockchain. A central node distributes the adapter module corresponding to the large model to consortium nodes. Upon receiving the adapter module, the consortium nodes join a Trusted Execution Environment (TEE). Training data is encrypted by a data terminal and uploaded to the consortium nodes. Within the TEE, the consortium nodes decrypt the training data and efficiently fine-tune the parameters of the adapter module using the decrypted training data. After training, the data is sent to the central node for aggregation processing to complete the training of the entire large model. By training in the TEE environment, the training process and plaintext training data of a consortium node are invisible to other blockchain participants, ensuring the security of training data, model data, and data during training. Furthermore, this application trains using plaintext data, which is more efficient than model parameter aggregation based on homomorphic encryption and other multi-party secure computation protocols, thus improving the efficiency of model training.

[0017] Optionally, each consortium node, before obtaining encrypted training data on the blockchain, further includes:

[0018] The data terminal acquires training data; the data terminal encrypts the training data to obtain encrypted training data; the data terminal sends the encrypted training data to the consortium node; the consortium node stores the encrypted training data on the blockchain.

[0019] Here, the data terminal of this application corresponds to a consortium node on the blockchain. After encrypting the training data, the data terminal sends it to the corresponding consortium node on the blockchain, and the consortium node stores it on the blockchain. The consortium node can replace the data terminal to perform federated learning of the parameters of the large model. The data terminal cannot know the data of the large model, which ensures the security of the large model. Moreover, due to the encryption of the data, the consortium nodes on the blockchain cannot know the training data of the data terminals corresponding to other consortium nodes, so the data security is high. Furthermore, after receiving the training data, the consortium node of this application stores the encrypted training data on the blockchain, realizing the traceability of the data source and improving the data security.

[0020] Optionally, the data terminal encrypts the training data to obtain encrypted training data, including:

[0021] The data terminal uses a pre-issued public key to encrypt the training data to obtain encrypted training data.

[0022] The data terminal of this application can use the issued public key to encrypt the training data. The encryption method is simple, and the public key encryption scheme is secure and reliable, which can effectively ensure the data security of the training data.

[0023] Optionally, each consortium node, in a trusted execution environment, decrypts the encrypted training data, including:

[0024] Each consortium node, in a trusted execution environment, uses a pre-issued private key to decrypt the encrypted training data to obtain the decrypted training data; wherein the private key and the public key belong to a public-private key pair.

[0025] In this application, the process of decrypting encrypted training data by the consortium node is performed in the TEE. The TEE provides reliable protection for the training data. In order to decrypt the training data encrypted with the public key, the consortium node uses a private key that belongs to the same public-private key pair as the public key for decryption. This results in high data security, simple encryption and decryption methods, and improved training efficiency for large models.

[0026] Optionally, it also includes:

[0027] Each data terminal initiates a blockchain registration request to the central node through its corresponding consortium node; the central node generates a public-private key pair based on the registration request; the public key of the public-private key pair is distributed to the corresponding data terminal, and the private key of the public-private key pair is distributed to the corresponding consortium node.

[0028] Here, the data terminal of this application can initiate registration with the central node of the blockchain through its corresponding consortium node. The central node, as a Certificate Authority (CA for short), can generate a public-private key pair for the data terminal and the corresponding consortium node after receiving the registration request. The public key is then distributed to the data terminal and the private key is distributed to the consortium node. The data terminal and the consortium node can achieve secure and reliable data transmission based on the public-private key pair. The training data encrypted with the public-private key pair is not visible to other devices and nodes, which improves the security of the training data.

[0029] Optionally, the consortium nodes store the encrypted training data on the blockchain, including:

[0030] The consortium nodes store the encrypted training data on-chain in the interplanetary file system of the blockchain.

[0031] This application enables the storage of encrypted training data on the blockchain based on the InterPlanetary File System (IPFS) distributed storage protocol. IPFS is completely decentralized, requiring no user trust in the identity and permissions of any node, nor requiring any node to use specific encryption and authentication mechanisms. User data can only be controlled and accessed by the user and will not be leaked by any node, thus protecting the user's privacy and rights. User data is verified through hash values ​​and digital signatures, ensuring that it will not be tampered with or deleted by any node, guaranteeing the reliability and recoverability of user data. Therefore, the security of training data stored on the blockchain is guaranteed.

[0032] Optionally, after the consortium node stores the encrypted training data on-chain in the InterPlanetary File System (IPS) of the blockchain, the method further includes:

[0033] The consortium nodes calculate a first message digest of the encrypted training data; after adding a timestamp to the first message digest, they package it into a block and store it in the block record section of the blockchain.

[0034] Here, this application processes the training data by batch hashing to obtain the hash value of the training data as a first information digest. After timestamping the first information digest, it packages it into blocks and stores them in the block record part of the blockchain. This realizes the recording of the source of the training data and prevents the training data from being tampered with. Due to the irreversibility of the data after hashing, the traceability of the data is guaranteed, as well as the security of the data.

[0035] Optionally, after training the fine-tuning model parameters of the adapter module using the training data in each consortium node within the trusted execution environment to obtain the updated fine-tuning model parameters, the method further includes:

[0036] Each consortium node calculates a second information digest of the training data; after adding a timestamp to the second information digest and the digest of the adapter module, they are packaged into a block and stored in the block record section of the blockchain.

[0037] Here, during training, the blockchain records the correspondence between the hash of the training data used in this training and the summary of the adapter module, and adds a timestamp, realizing the traceability of data throughout the training process, preventing data from being tampered with during training, and preventing data leakage due to the irreversibility of data after hash processing, further ensuring the data security of large model training.

[0038] Optionally, the central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node, including:

[0039] The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node based on the transport layer security protocol.

[0040] In this application, the central node sends the adapter module to each consortium node through Transport Layer Security (TLS). This protocol protects the security and privacy of network communication through mechanisms such as encryption, data integrity, and authentication, thereby improving the data transmission security of large models and further enhancing data security during the training process of large models.

[0041] Optionally, the central node aggregates the fine-tuned model parameters uploaded by all alliance nodes to obtain aggregated fine-tuned model parameters, including:

[0042] The central node calculates the average value of the fine-tuned model parameters uploaded by all alliance nodes, and determines the average value as the aggregated fine-tuned model parameters.

[0043] In this application, the central node can use the method of calculating the average value to aggregate the fine-tuned model parameters uploaded by the alliance nodes. This method can quickly aggregate the fine-tuned model parameters sent by multiple alliance nodes, thus improving the efficiency of large model training.

[0044] Optionally, the central node aggregates the fine-tuned model parameters uploaded by all alliance nodes to obtain aggregated fine-tuned model parameters, including:

[0045] The central node obtains the pre-stored weight coefficients of each alliance node; the central node calculates a weighted average based on the weight coefficients of each alliance node and the fine-tuning model parameters, and determines the weighted average as the aggregated fine-tuning model parameters.

[0046] In this application, the central node can calculate a weighted average value by pre-stored weight coefficients of each alliance node to aggregate the fine-tuned model parameters uploaded by the alliance nodes. The weight coefficients can be assigned based on the amount of local data on the data terminal, i.e. the size of the training data. The more data a data terminal has, the greater the weight of the corresponding alliance node. This fully considers the impact of the amount of data on the training results and further improves the accuracy of large model training.

[0047] Secondly, this application provides a blockchain-based large-scale model training device, wherein the blockchain includes a central node and multiple consortium nodes; wherein each consortium node corresponds to a data terminal, and the device includes:

[0048] The first acquisition module is used by the central node to acquire a pre-trained large model;

[0049] The first processing module is used for the central node to obtain the adapter module corresponding to the large model and send the adapter module to each alliance node; wherein, the adapter module includes fine-tuned model parameters of the pre-trained large model;

[0050] The second processing module is used by each alliance node to add the adapter module to the trusted execution environment;

[0051] The second acquisition module is used by each consortium node to acquire encrypted training data on the blockchain; wherein the training data is acquired by the data terminal corresponding to each consortium node, encrypted, and then sent to the consortium node, and stored on the blockchain by the consortium node.

[0052] The third processing module is used by each alliance node to decrypt the encrypted training data in a trusted execution environment, and to use the decrypted training data to train the fine-tuning model parameters of the adapter module to obtain the updated fine-tuning model parameters.

[0053] An upload module is used by each alliance node to upload the updated fine-tuned model parameters to the central node;

[0054] The aggregation module is used by the central node to aggregate the fine-tuning model parameters uploaded by all alliance nodes to obtain aggregated fine-tuning model parameters.

[0055] The update module is used by the central node to update the pre-trained large model according to the aggregated fine-tuned model parameters, so as to obtain the updated large model.

[0056] The fourth processing module is used to, if the updated large model does not converge, continue to use the updated large model as the pre-trained large model and restart the steps of "obtaining the adapter module corresponding to the large model and sending the adapter module to each alliance node" until the updated large model converges, so as to complete the training of the large model.

[0057] Optionally, before the second acquisition module is used by each consortium node to acquire encrypted training data on the blockchain, the method further includes:

[0058] The third acquisition module is used by the data terminal to acquire training data;

[0059] An encryption module is used by the data terminal to encrypt the training data to obtain encrypted training data;

[0060] A sending module is used by the data terminal to send the encrypted training data to the consortium node;

[0061] The first upload module is used by the alliance node to upload the encrypted training data to the blockchain.

[0062] Optionally, the encryption module is specifically used for:

[0063] The data terminal uses a pre-issued public key to encrypt the training data to obtain encrypted training data.

[0064] Optionally, the third processing module is specifically used for:

[0065] Each consortium node, in a trusted execution environment, uses a pre-issued private key to decrypt the encrypted training data to obtain the decrypted training data; wherein the private key and the public key belong to a public-private key pair.

[0066] Optionally, it also includes:

[0067] The registration module is used by each data terminal to initiate a blockchain registration request to the central node through the corresponding alliance node;

[0068] The public-private key pair distribution module is used by the central node to generate a public-private key pair according to the registration request; distribute the public key of the public-private key pair to the corresponding data terminal; and distribute the private key of the public-private key pair to the corresponding consortium node.

[0069] Optionally, the first upload module is specifically used for:

[0070] The consortium nodes store the encrypted training data on-chain in the interplanetary file system of the blockchain.

[0071] Optionally, after the first upload module stores the encrypted training data on the blockchain's InterPlanetary File System, the device further includes a second upload module for:

[0072] The consortium nodes calculate a first message digest of the encrypted training data; after adding a timestamp to the first message digest, they package it into a block and store it in the block record section of the blockchain.

[0073] Optionally, after the third processing module is used by each consortium node in the trusted execution environment to train the fine-tuning model parameters of the adapter module using the training data to obtain the updated fine-tuning model parameters, a third upload module is further included, used for:

[0074] Each consortium node calculates a second information digest of the training data; after adding a timestamp to the second information digest and the digest of the adapter module, they are packaged into a block and stored in the block record section of the blockchain.

[0075] Optionally, the first processing module is specifically used for:

[0076] The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node based on the transport layer security protocol.

[0077] Optionally, the aggregation module is specifically used for:

[0078] The central node calculates the average value of the fine-tuned model parameters uploaded by all alliance nodes, and determines the average value as the aggregated fine-tuned model parameters.

[0079] Optionally, the aggregation module is further specifically used for:

[0080] The central node obtains the pre-stored weight coefficients of each alliance node; the central node calculates a weighted average based on the weight coefficients of each alliance node and the fine-tuning model parameters, and determines the weighted average as the aggregated fine-tuning model parameters.

[0081] Thirdly, this application provides a large model training device based on blockchain, comprising: at least one processor and memory;

[0082] The memory stores computer-executed instructions;

[0083] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the blockchain-based large model training method as described in the first aspect and various possible designs of the first aspect.

[0084] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the blockchain-based large model training method described in the first aspect and various possible designs of the first aspect.

[0085] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the blockchain-based large model training method described in the first aspect and various possible designs of the first aspect.

[0086] This application provides a blockchain-based method, apparatus, device, and storage medium for training large models. The entire large model training process is conducted on the blockchain. A central node distributes the corresponding adapter module to consortium nodes. Upon receiving the adapter module, the consortium nodes add it to a TEE (Training Environment). Training data is encrypted by a data terminal and uploaded to the consortium nodes. Within the TEE, the consortium nodes decrypt the training data and efficiently fine-tune the parameters of the adapter module using the decrypted training data. After training, the data is sent to the central node for aggregation processing to complete the training of the entire large model. Training in the TEE environment ensures that the training process and plaintext training data within a consortium node are invisible to other blockchain participants, guaranteeing the security of training data, model data, and data during training. Furthermore, this application trains using plaintext data, which is more efficient than model parameter aggregation based on homomorphic encryption and other multi-party secure computation protocols, thus improving the efficiency of model training. Attached Figure Description

[0087] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0088] Figure 1 is a schematic diagram of the structure of a large model training system based on blockchain provided in an embodiment of this application;

[0089] Figure 2 is a schematic diagram of a structure for efficient fine-tuning of a consortium node through a TEE execution model provided in an embodiment of this application;

[0090] Figure 3 is a flowchart illustrating a large model training method based on blockchain provided in an embodiment of this application;

[0091] Figure 4 is a flowchart illustrating another blockchain-based large model training method provided in an embodiment of this application;

[0092] Figure 5 is a schematic diagram of a large model training device based on blockchain provided in an embodiment of this application;

[0093] Figure 6 is a schematic diagram of the structure of a large-scale model training device based on blockchain according to an embodiment of this application. The above figures illustrate specific embodiments of this application, which will be described in more detail below. These figures and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0094] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0095] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0096] Data security is paramount for large-scale models. Several training methods exist for large models, including: For example, blockchain-based federated learning systems and methods update model parameters through multiple rounds of parameter aggregation until convergence. Alternatively, a centralized server role is replaced by nodes randomly selected from the blockchain. Another approach is blockchain-based federated learning defense methods. Each participating node retrieves the latest model from the blockchain and downloads it locally, trains it using local data, and broadcasts the training time on the blockchain upon completion. Noise is added to the local model during training, and two validators are created for each participating node. An authoritative institution obtains and aggregates all validated updated models from the block nodes. These solutions largely employ traditional federated learning, collaboratively training a large machine learning model across multiple devices or data sources. However, existing technologies for training large models primarily suffer from low data security.

[0097] To address the aforementioned issues, this application provides a blockchain-based method, apparatus, device, and medium for training large-scale models. The entire large-scale model training process is conducted on the blockchain. A central node distributes the corresponding adapter module to consortium nodes. Upon receiving the adapter module, the consortium nodes add it to the TEE (Training Equipment Environment). Training data is encrypted by a data terminal and uploaded to the consortium nodes. The consortium nodes decrypt the training data within the TEE and efficiently fine-tune the parameters of the adapter module using the decrypted training data. After training, the data is sent to the central node for aggregation processing to complete the training of the entire large-scale model.

[0098] Optionally, the blockchain in this application embodiment is a consortium blockchain. As a blockchain technology among specific groups, consortium blockchains have the characteristics of high efficiency, security, privacy, and scalability, which can improve the security and efficiency of large model training.

[0099] Optionally, the embodiments of this application can achieve the following functions and effects:

[0100] (1) Data sources are traceable, avoiding data poisoning. The entire process of training the federated large model is carried out on the blockchain. The data and models used in each round of model training are timestamped, encrypted, and stored on the blockchain, ensuring that the data source is non-repudiable.

[0101] (2) Data and Model Privacy Protection. For each round of training on the consortium nodes, the model's training parameters are decrypted in the TEE, and the training data is encrypted and uploaded to the consortium nodes by the data terminal. During training, the consortium nodes read the data from the TEE using the IPFS protocol and a distributed hash table for decryption. The efficient fine-tuning of the Adapter model parameters is performed in the TEE. That is, the training process and plaintext training data of the model in a consortium node are not visible to other blockchain participants.

[0102] (3) Improve the time efficiency of model training. Performing plaintext model training in a TEE environment is more efficient than model parameter aggregation based on homomorphic encryption or multi-party secure computation. In addition, cloud servers with TEE chip capabilities are gradually becoming more widespread and will become more universal and standardized in the future.

[0103] Optionally, Figure 1 is a schematic diagram of the structure of a blockchain-based large model training system provided in an embodiment of this application. The blockchain includes a central node and multiple consortium nodes; each consortium node corresponds to a data terminal. The system is a three-layer blockchain distributed large model training mode. As shown in Figure 1, the blockchain-based large model training system provided in this embodiment of the application includes three layers: the first layer is the blockchain central layer, which includes a central node 101; the second layer is the consortium node layer, with four consortium nodes (first consortium node 102, second consortium node 103, third consortium node 104, and fourth consortium node 105) as an example in Figure 1; and the third layer is the data terminal layer, with four data terminals (first data terminal 106, second data terminal 107, third data terminal 108, and fourth data terminal 109) as an example in Figure 1. It should be understood that Figure 1 is only illustrative, and the structure of the blockchain-based large model training system and the number of consortium nodes and data terminals can be determined according to the actual situation. This embodiment of the application does not impose specific limitations.

[0104] In Figure 1, the central node and all the alliance nodes are on a blockchain / consortium blockchain, and can communicate through the blockchain / consortium blockchain.

[0105] In this system, one data terminal corresponds to one alliance node. For example, the first data terminal 106 in Figure 1 corresponds to the first alliance node 102. The data terminal and its corresponding alliance node can communicate with each other, and the data terminal can join the blockchain through its corresponding alliance node.

[0106] Based on the above structure, the embodiments of this application can achieve the following functions through various optional implementation methods:

[0107] Option 1 and its effects: Introducing a central blockchain layer, i.e., a central node, to ensure that data and models do not leave the blockchain. As shown in Figure 1, the data terminal is decoupled from the central node, and the central layer server, i.e., the central node, is located on the blockchain. After the data terminal encrypts and uploads the data to the consortium nodes, subsequent model training is executed on-chain. That is, the model is not trained locally by the data holder, which can effectively avoid faulty participants in federated learning from poisoning the global model with data.

[0108] Option 2 and its effects: Traceable ownership of training data. All training data provided by data terminals is batch-hashted, timestamped, and uploaded to the blockchain. During training, the hash-model summary correspondence of the data used in this training is recorded on the blockchain's distributed ledger and timestamped. Furthermore, data encryption and on-chain storage protects data privacy.

[0109] Optional Implementation Method Three and its Effects: Trusted Storage and Data Interoperability: This solution is based on the IPFS distributed storage protocol.

[0110] It supports on-chain data sharing, and cross-chain services can enable trusted interoperability of homogeneous / heterogeneous chain data through authorization, breaking down data silos of a single chain and ensuring that the authenticity of data is not tampered with during the circulation process.

[0111] An encrypted storage and sharing method was designed based on the original plaintext sharing in IPFS. Data is decrypted in a trusted execution environment, smart contracts are executed, and parameter-efficient fine-tuning (PEFT) is performed on the large-scale adapter parameters.

[0112] Traditional internet security is centralized, requiring users to trust the server's identity and permissions, as well as the encryption and authentication mechanisms used. This approach has several drawbacks: user data may be leaked, stolen, or sold by the server, resulting in damage to user privacy and rights; user data may be tampered with or deleted by the server, making it unreliable or unrecoverable; user data may be censored or blocked by the server, leading to a lack of freedom or fairness in user data access. IPFS security, on the other hand, is decentralized. It does not require users to trust the identity and permissions of any node, nor does it require any node to use specific encryption and authentication mechanisms. This approach has several advantages: user data is only controlled and accessed by the user, preventing leakage, theft, or sale by any node, thus protecting user privacy and rights; user data is verified through hash values ​​and digital signatures, preventing tampering or deletion by any node, ensuring data reliability and recoverability; user data is transmitted through efficient content addressing and routing mechanisms, preventing censorship or blocking by any node. IPFS supports encrypted transmission and can protect the security of transmitted data through protocols such as TLS, preventing data eavesdropping or tampering.

[0113] Optionally, Figure 2 is a schematic diagram of a consortium node performing efficient model fine-tuning via a TEE according to an embodiment of this application. As shown in Figure 2, the consortium node can add an adapter module to the trusted execution environment. The adapter module includes adapter parameters. Fine-tuning of the adapter module is performed within the TEE. After fine-tuning, the fine-tuned adapter module is taken outside the TEE to adjust the local large language model (Local LLM). Due to the security and isolation of the TEE, the training process and plaintext training data of the model in a consortium node are not visible to other blockchain participants (consortium nodes), thus improving the security of large model training.

[0114] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the architecture of a large-scale model training system based on blockchain. In other feasible embodiments of this application, the above architecture may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components, which can be determined according to the actual application scenario and is not limited here. The components shown in Figures 1 and 2 can be implemented in hardware, software, or a combination of software and hardware.

[0115] In the specific implementation process, the aforementioned central node, data terminal, and alliance node include processors or processing devices.

[0116] It should be understood that the aforementioned processing device can be implemented by a processor reading instructions from memory and executing those instructions, or it can be implemented by a chip circuit.

[0117] Furthermore, the network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0118] The technical solution of this application will be described in detail below with reference to specific embodiments:

[0119] Optionally, Figure 3 is a flowchart illustrating a blockchain-based large model training method provided in an embodiment of this application. The execution entity in this embodiment can be the blockchain-based large model training system shown in Figure 1; the specific execution entity can be determined according to the actual application scenario. As shown in Figure 3, the method includes the following steps:

[0120] S301: The central node acquires the pre-trained large model.

[0121] Among them, the pre-trained large model is the large model that needs to be trained.

[0122] S302: The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node.

[0123] The adapter module includes fine-tuning model parameters for a pre-trained large model.

[0124] Optionally, the adapter module can be queried and retrieved by using the preset insertion position of the adapter module.

[0125] Adapter modules are a technique used for fine-tuning neural networks, particularly large pre-trained language models. Adapter modules are a set of small network layers designed to be embedded into an existing pre-trained model to achieve adaptive learning for a specific task. An adapter is essentially a subset of specific parameters from the original large model, representing only a few percent or a fraction of the total parameters. Fine-tuning these parameters can achieve almost the same effect as fine-tuning the entire model. By inserting small network layers into the pre-trained large model and updating only the parameters of these layers, adapter modules can learn task-specific features without altering the main model structure, thus improving the model's flexibility and efficiency. In this embodiment, training the entire large model can be achieved simply by adjusting the parameters of the adapter modules, further ensuring the data security of the large model.

[0126] Optionally, the central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node, including:

[0127] The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node based on the transport layer security protocol.

[0128] In this embodiment, the central node sends the adapter module to each consortium node via the TLS protocol. This protocol protects the security and privacy of network communication through mechanisms such as encryption, data integrity, and authentication, thereby improving the data transmission security of large models and further enhancing data security during the training process of large models.

[0129] S303: Each consortium node adds the adapter module to the trusted execution environment.

[0130] As shown in Figure 2, in this embodiment of the application, the alliance node fine-tunes the Adapter Modules within the TEE to ensure the data security of the parameters in the Adapter Modules.

[0131] S304: Each consortium node obtains encrypted training data on the blockchain.

[0132] The training data is obtained by the data terminal corresponding to each alliance node, encrypted, and then sent to the alliance node, which then stores it on the blockchain.

[0133] S305: Each consortium node, in a trusted execution environment, decrypts the encrypted training data and uses the decrypted training data to train the fine-tuning model parameters of the adapter module to obtain the updated fine-tuning model parameters; each consortium node uploads the updated fine-tuning model parameters to the central node.

[0134] Specifically, the fine-tuned model parameters are obtained after training within the TEE. After obtaining the updated fine-tuned model parameters, the updated fine-tuned model parameters are taken to the consortium node outside the TEE and uploaded to the central node.

[0135] Optionally, each consortium node uploads the updated fine-tuned model parameters to the central node based on the TLS protocol.

[0136] S306: The central node aggregates the fine-tuned model parameters uploaded by all alliance nodes to obtain the aggregated fine-tuned model parameters.

[0137] Optionally, the central node aggregates the fine-tuned model parameters uploaded by all alliance nodes to obtain aggregated fine-tuned model parameters, including:

[0138] The central node calculates the average value of the fine-tuned model parameters uploaded by all alliance nodes, and uses this average value as the aggregated fine-tuned model parameters.

[0139] In this embodiment, the central node can use the method of calculating the average value to aggregate the fine-tuned model parameters uploaded by the alliance nodes. This method can quickly aggregate the fine-tuned model parameters sent by multiple alliance nodes, thus improving the efficiency of large model training.

[0140] Optionally, the central node aggregates the fine-tuned model parameters uploaded by all alliance nodes to obtain aggregated fine-tuned model parameters, including:

[0141] The central node obtains the pre-stored weight coefficients of each alliance node; the central node calculates the weighted average based on the weight coefficients of each alliance node and the fine-tuning model parameters, and determines the weighted average as the aggregated fine-tuning model parameters.

[0142] It is understood that the weighting coefficients here can be determined according to the actual situation, and the specific implementation of this application does not impose any specific limitations.

[0143] Optionally, the amount of training data corresponding to each alliance node can be obtained, and a weight coefficient can be determined for each alliance node based on the proportion of the amount of training data corresponding to each alliance node.

[0144] In this embodiment, the central node can calculate a weighted average value by pre-stored weight coefficients of each alliance node to aggregate the fine-tuned model parameters uploaded by the alliance nodes. The weight coefficients can be assigned based on the amount of local data on the data terminal, i.e., the size of the training data. The more data a data terminal has, the greater the weight of the alliance node. This fully considers the impact of the amount of data on the training results and further improves the accuracy of large model training.

[0145] Optionally, a Federated Averaging (FedAvg) algorithm is used for model aggregation. The central node collects weight updates from the Adapter Modules of all federated nodes, calculates the average weights, and distributes the updated weights back to each federated node. This process is repeated in multiple rounds until convergence is achieved. To ensure the accuracy of model aggregation, the FedAvg algorithm uses a weighted averaging approach. Specifically, each federated node's uploaded model parameters are assigned a weight, and then a weighted average is performed. The weights of the uploaded model parameters are assigned based on the amount of local data on the device; devices with more data have higher weights.

[0146] S307: The central node updates the pre-trained large model based on the aggregated fine-tuned model parameters to obtain the updated large model.

[0147] S308: If the updated large model does not converge, the central node continues to use the updated large model as the pre-trained large model and restarts the steps of "obtaining the adapter module corresponding to the large model and sending the adapter module to each alliance node" until the updated large model converges to complete the training of the large model.

[0148] This application proposes a blockchain-based method for training large models. The entire training process is conducted on the blockchain. A central node distributes the corresponding adapter module to consortium nodes. Upon receiving the adapter module, the consortium nodes add it to the TEE (Training Environment). Training data is encrypted by a data terminal and uploaded to the consortium nodes. Within the TEE, the consortium nodes decrypt the training data and perform efficient fine-tuning of the adapter module parameters using the decrypted training data. After training, the data is sent to the central node for aggregation processing to complete the training of the entire large model. By training in the TEE environment, the training process and plaintext training data of a consortium node are invisible to other blockchain participants, ensuring the security of training data, model data, and data during training. Furthermore, this application trains using plaintext data, which is more efficient than model parameter aggregation based on homomorphic encryption and other multi-party secure computation protocols, thus improving the efficiency of model training.

[0149] Optionally, the training data is encrypted in this embodiment of the application. Accordingly, Figure 4 is a flowchart illustrating another blockchain-based large model training method provided in this embodiment of the application. As shown in Figure 4, the method includes:

[0150] S401: The central node acquires the pre-trained large model.

[0151] S402: The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node.

[0152] The adapter module includes fine-tuning model parameters for a pre-trained large model.

[0153] S403: Each consortium node adds the adapter module to the trusted execution environment.

[0154] The implementation methods of steps S401-S403 are the same as those of steps S301 and S303, and will not be described in detail here.

[0155] S404: The data terminal acquires training data; the data terminal encrypts the training data to obtain encrypted training data.

[0156] Optionally, the training data is stored in the local storage location of the data terminal, and the data terminal can directly obtain the training data from the local storage location.

[0157] Optionally, the data terminal encrypts the training data to obtain encrypted training data, including:

[0158] The data terminal uses a pre-issued public key to encrypt the training data to obtain encrypted training data.

[0159] In this embodiment of the application, the data terminal can use the issued public key to encrypt the training data. The encryption method is simple, and the public key encryption scheme is secure and reliable, which can effectively ensure the data security of the training data.

[0160] Optionally, based on the above public-key encryption method, each consortium node, in a trusted execution environment, decrypts the encrypted training data, including:

[0161] Each consortium node, in a trusted execution environment, uses a pre-issued private key to decrypt the encrypted training data to obtain the decrypted training data; the private key and public key are a public-private key pair.

[0162] In this embodiment of the application, the process of the consortium node decrypting the encrypted training data is performed in the TEE. The TEE provides reliable protection for the training data. In order to decrypt the training data encrypted with the public key, the consortium node uses a private key that belongs to the same public-private key pair as the public key to decrypt. This results in high data security, simple encryption and decryption methods, and improved training efficiency for large models.

[0163] Optionally, it also includes:

[0164] Each data terminal initiates a blockchain registration request to the central node through its corresponding consortium node; the central node generates a public-private key pair based on the registration request; the public key of the public-private key pair is distributed to the corresponding data terminal, and the private key of the public-private key pair is distributed to the corresponding consortium node.

[0165] Here, in this embodiment of the application, the data terminal can initiate registration with the central node of the blockchain through its corresponding consortium node. The central node, as a CA, can generate a public-private key pair for the data terminal and the consortium node corresponding to the data terminal after receiving the registration request. The public key is sent to the data terminal and the private key is sent to the consortium node. The data terminal and the consortium node can realize secure and reliable data transmission based on the public-private key pair. The training data encrypted by the public-private key pair is not visible to other devices and nodes, which improves the security of the training data.

[0166] S405: The data terminal sends the encrypted training data to the consortium node.

[0167] S406: Consortium nodes store encrypted training data on the blockchain.

[0168] Optionally, consortium nodes will store encrypted training data on the blockchain, including:

[0169] Consortium nodes store encrypted training data on-chain in the interplanetary file system of the blockchain.

[0170] In this embodiment, encrypted training data can be stored on the blockchain based on the IPFS distributed storage protocol. IPFS is completely decentralized, which does not require users to trust the identity and permissions of any node, nor does it require any node to use specific encryption and authentication mechanisms. Only the user can control and access the user's data, and it will not be leaked by any node, thus protecting the user's privacy and rights. The user's data is verified through hash values ​​and digital signatures, and it will not be tampered with or deleted by any node, ensuring the reliability and recoverability of the user's data. Therefore, the security of the training data stored on the blockchain is guaranteed.

[0171] In this embodiment of the application, there is no specific restriction on the order of steps S401-S403 and steps S404-S406. Steps S404-S406 can also be performed before steps S401-S403, simultaneously, or in an overlapping manner.

[0172] Optionally, after the consortium nodes store the encrypted training data on-chain in the InterPlanetary File System (IPS) of the blockchain, the process also includes:

[0173] The consortium nodes calculate the first message digest of the encrypted training data; after adding a timestamp to the first message digest, they package it into a block and store it in the block record section of the blockchain.

[0174] Here, in this embodiment of the application, the training data is processed by batch hashing to obtain the hash value of the training data as a first information digest. The first information digest is then timestamped and packaged into a block and stored in the block record part of the blockchain. This realizes the recording of the source of the training data and prevents the training data from being tampered with. Due to the irreversibility of the data after hashing, the traceability of the data is guaranteed, as well as the security of the data.

[0175] S407: Each consortium node obtains encrypted training data on the blockchain.

[0176] The training data is obtained by the data terminal corresponding to each alliance node, encrypted, and then sent to the alliance node, which then stores it on the blockchain.

[0177] S408: Each consortium node, in a trusted execution environment, decrypts the encrypted training data and uses the decrypted training data to train the fine-tuning model parameters of the adapter module to obtain the updated fine-tuning model parameters; each consortium node uploads the updated fine-tuning model parameters to the central node.

[0178] Optionally, at each consortium node, in a trusted execution environment, after training the fine-tuning model parameters of the adapter module using training data to obtain the updated fine-tuning model parameters, the process also includes:

[0179] Each consortium node calculates a second information digest of the training data; after adding a timestamp to the second information digest and the digest of the adapter module, they are packaged into a block and stored in the block record section of the blockchain.

[0180] Here, in this embodiment of the application, during training, the corresponding relationship between the hash of the training data used in this training and the summary of the adapter module is recorded in the block record part of the blockchain, and a timestamp is added, which realizes the traceability of data throughout the training process, prevents the data from being tampered with during training, and prevents data leakage due to the irreversibility of data after hash processing, further ensuring the data security of large model training.

[0181] S409: The central node aggregates the fine-tuned model parameters uploaded by all alliance nodes to obtain the aggregated fine-tuned model parameters.

[0182] S410: The central node updates the pre-trained large model based on the aggregated fine-tuned model parameters to obtain the updated large model.

[0183] S411: If the updated large model does not converge, the central node continues to use the updated large model as the pre-trained large model and restarts the steps of "obtaining the adapter module corresponding to the large model and sending the adapter module to each alliance node" until the updated large model converges to complete the training of the large model.

[0184] The implementation methods of steps S407-S411 and steps S304-S308 are the same, and will not be described in detail here.

[0185] In this embodiment, the data terminal corresponds to a consortium node on the blockchain. The data terminal encrypts the training data and sends it to the corresponding consortium node on the blockchain, where it is stored on the blockchain. The consortium node can replace the data terminal to perform federated learning of the parameters of a large model. The data terminal cannot access the data of the large model, ensuring its security. Furthermore, due to the encryption of the data, the consortium nodes on the blockchain cannot access the training data of the data terminals corresponding to other consortium nodes, resulting in high data security. In addition, after receiving the training data, the consortium node in this embodiment stores the encrypted training data on the blockchain, achieving traceability of the data source and improving data security.

[0186] In one possible implementation, the process of training a large model based on blockchain is as follows:

[0187] Step 1: Consortium Blockchain Registration. Each consortium node and its corresponding data terminal registers and joins the consortium blockchain.

[0188] Step Two: Public Key Distribution. The authoritative and trusted certificate authority, acting as the central node, generates a public-private key pair. The public key is distributed to all consortium nodes on the consortium blockchain, and then sent to the data terminal through the consortium nodes. The private key is distributed to the consortium nodes on the consortium blockchain for safekeeping.

[0189] Step 3: Data Upload to the Blockchain. Each consortium node splits the data used to train the federated large model into batches, encrypts it using a public key, and uploads it to the IPFS storage system on the consortium blockchain.

[0190] Step 4: Federated Model Initialization. The training method used in this embodiment is a highly efficient fine-tuning training based on a modification of the traditional FedAvg algorithm. The fine-tuning method is adapter fine-tuning of the FedAdaptor. The central node server first uploads a pre-trained large model, and then freezes all parameters of the pre-trained large model except for the Adaptor. The central node server then distributes the Adaptor to all federated node servers based on the TLS security protocol.

[0191] The server collects weight updates from all consortium nodes, calculates the average weights, and distributes the updated weights back to each consortium node. This process is repeated in multiple rounds until convergence is achieved. To ensure the accuracy of model aggregation, the FedAvg algorithm uses a weighted average method for model aggregation. Specifically, each model parameter uploaded by a consortium node is assigned a weight, and then a weighted average is performed. The weights of the model parameters uploaded by consortium nodes are assigned based on the amount of local data on the device; devices with more data have higher weights.

[0192] Step 5: Alliance Node Model Verification. In the previous step, after all alliance nodes received the model from the central node, they joined the TEE environment.

[0193] Step Six: Federated Distributed Model Training. Each federated block node collects its own batch training data in the IPFS system and decrypts the data using its private key in the TEE, then performs efficient fine-tuning with the Adaptor.

[0194] Step 7: Model Aggregation and Update. Each consortium node takes the model trained within the TEE environment from the previous step and uploads it to the central node via the TLS protocol to a consortium blockchain node outside the TEE environment. After receiving the Adaptor models uploaded by each consortium node, the central node performs model parameter aggregation and update in the FedAvg algorithm.

[0195] Step 8: Iterate the model until it converges. Repeat the above steps until the model converges.

[0196] Based on the above method, secure training of large models was achieved.

[0197] Figure 5 is a schematic diagram of the structure of a blockchain-based large model training device provided in an embodiment of this application. As shown in Figure 5, the device in this embodiment includes: a first acquisition module 501, a first processing module 502, a second processing module 503, a second acquisition module 504, a third processing module 505, an upload module 506, an aggregation module 507, an update module 508, and a fourth processing module 509. The blockchain-based large model training device can be the processing device of the aforementioned central node, consortium node, or data terminal, the processor itself, or a chip or integrated circuit that implements the processor's function. It should be noted that the division of the first acquisition module 501, first processing module 502, second processing module 503, second acquisition module 504, third processing module 505, upload module 506, aggregation module 507, update module 508, and fourth processing module 509 is only a logical functional division; physically, they can be integrated or independent.

[0198] The first acquisition module is used by the central node to acquire the pre-trained large model.

[0199] The first processing module is used by the central node to obtain the adapter module corresponding to the large model and send the adapter module to each alliance node; wherein, the adapter module includes the fine-tuned model parameters of the pre-trained large model;

[0200] The second processing module is used by each alliance node to add the adapter module to the trusted execution environment.

[0201] The second acquisition module is used by each consortium node to acquire encrypted training data on the blockchain; wherein, the training data is acquired by the data terminal corresponding to each consortium node, encrypted, and then sent to the consortium node, which stores it on the blockchain.

[0202] The third processing module is used by each alliance node to decrypt the encrypted training data in a trusted execution environment, and then use the decrypted training data to train the fine-tuning model parameters of the adapter module to obtain the updated fine-tuning model parameters.

[0203] The upload module is used by each alliance node to upload the updated fine-tuned model parameters to the central node.

[0204] The aggregation module is used by the central node to aggregate the fine-tuned model parameters uploaded by all alliance nodes to obtain the aggregated fine-tuned model parameters.

[0205] The update module is used by the central node to update the pre-trained large model based on the aggregated fine-tuned model parameters, so as to obtain the updated large model.

[0206] The fourth processing module is used so that if the updated large model does not converge, the central node continues to use the updated large model as the pre-trained large model and restarts the steps of "obtaining the adapter module corresponding to the large model and sending the adapter module to each alliance node" until the updated large model converges to complete the training of the large model.

[0207] Optionally, before the second acquisition module is used by each consortium node to acquire encrypted training data on the blockchain, it also includes:

[0208] The third acquisition module is used by the data terminal to acquire training data;

[0209] The encryption module is used by the data terminal to encrypt the training data to obtain encrypted training data.

[0210] The sending module is used by the data terminal to send encrypted training data to the consortium nodes;

[0211] The first upload module is used by alliance nodes to upload encrypted training data to the blockchain.

[0212] Optionally, the encryption module is specifically used for:

[0213] The data terminal uses a pre-issued public key to encrypt the training data to obtain encrypted training data.

[0214] Optionally, the third processing module is specifically used for:

[0215] Each consortium node, in a trusted execution environment, uses a pre-issued private key to decrypt the encrypted training data to obtain the decrypted training data; the private key and public key are a public-private key pair.

[0216] Optionally, it also includes:

[0217] The registration module is used by each data terminal to initiate a blockchain registration request to the central node through the corresponding alliance node;

[0218] The public-private key pair distribution module is used by the central node to generate a public-private key pair based on the registration request; distribute the public key of the public-private key pair to the corresponding data terminal; and distribute the private key of the public-private key pair to the corresponding alliance node.

[0219] Optionally, the first upload module is specifically used for:

[0220] Consortium nodes store encrypted training data on-chain in the interplanetary file system of the blockchain.

[0221] Optionally, after the first upload module stores the encrypted training data on-chain in the InterPlanetary File System (IPS) of the blockchain, the device further includes a second upload module for:

[0222] The consortium nodes calculate the first message digest of the encrypted training data; after adding a timestamp to the first message digest, they package it into a block and store it in the block record section of the blockchain.

[0223] Optionally, after the third processing module, for each consortium node in the trusted execution environment, trains the fine-tuning model parameters of the adapter module using training data to obtain the updated fine-tuning model parameters, a third upload module is also included, for:

[0224] Each consortium node calculates a second information digest of the training data; after adding a timestamp to the second information digest and the digest of the adapter module, they are packaged into a block and stored in the block record section of the blockchain.

[0225] Optionally, the first processing module is specifically used for:

[0226] The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node based on the transport layer security protocol.

[0227] Optionally, the aggregation module is specifically used for:

[0228] The central node calculates the average value of the fine-tuned model parameters uploaded by all alliance nodes, and uses this average value as the aggregated fine-tuned model parameters.

[0229] Optionally, the aggregation module is also specifically used for:

[0230] The central node obtains the pre-stored weight coefficients of each alliance node; the central node calculates the weighted average based on the weight coefficients of each alliance node and the fine-tuning model parameters, and determines the weighted average as the aggregated fine-tuning model parameters.

[0231] Referring to Figure 6, a schematic diagram of a blockchain-based large model training device 600 suitable for implementing embodiments of the present disclosure is shown. This blockchain-based large model training device 600 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. The blockchain-based large model training device shown in Figure 6 is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present disclosure.

[0232] As shown in Figure 6, the blockchain-based large model training device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to the program stored in read-only memory (ROM) 602 or the program loaded from storage device 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the blockchain-based large model training device 600. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input-to-output (I / O) interface 605 is also connected to the bus 604.

[0233] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 608 including, for example, magnetic tape, hard disk, etc.; and communication devices 609. Communication device 609 allows the blockchain-based large model training device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 illustrates a blockchain-based large model training device 600 with various devices, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented alternatively.

[0234] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0235] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0236] The aforementioned computer-readable medium may be included in the aforementioned blockchain-based large model training device; or it may exist independently and not be assembled into the blockchain-based large model training device.

[0237] The aforementioned computer-readable medium carries one or more programs, which, when executed by the blockchain-based large model training device, cause the blockchain-based large model training device to perform the method shown in the above embodiments.

[0238] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0239] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0240] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0241] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0242] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0243] The blockchain-based large model training device in this application can be used to execute the technical solutions in the above-described method embodiments of this application. Its implementation principle and technical effect are similar, and will not be repeated here.

[0244] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement any of the above-mentioned blockchain-based large model training methods.

[0245] This application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement any of the above-mentioned blockchain-based large model training methods.

[0246] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0247] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0248] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0249] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for training large-scale models based on blockchain, characterized in that, The blockchain includes a central node and multiple consortium nodes; Each alliance node corresponds to a data terminal, and the method includes: The central node acquires a pre-trained large model; The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node; wherein, the adapter module includes the fine-tuned model parameters of the pre-trained large model; Each alliance node adds the adapter module to the trusted execution environment; Each consortium node acquires encrypted training data on the blockchain; wherein, the training data is acquired by the data terminal corresponding to each consortium node, encrypted, and then sent to the consortium node, which stores it on the blockchain. Each consortium node, in a trusted execution environment, decrypts the encrypted training data and uses the decrypted training data to train the fine-tuning model parameters of the adapter module to obtain updated fine-tuning model parameters. Each alliance node uploads the updated fine-tuned model parameters to the central node; The central node aggregates the fine-tuned model parameters uploaded by all alliance nodes to obtain aggregated fine-tuned model parameters. The central node updates the pre-trained large model based on the aggregated fine-tuned model parameters to obtain the updated large model. If the updated large model does not converge, the central node continues to use the updated large model as the pre-trained large model and restarts the steps of "obtaining the adapter module corresponding to the large model and sending the adapter module to each alliance node" until the updated large model converges to complete the training of the large model.

2. The method according to claim 1, characterized in that, Before each consortium node obtains encrypted training data from the blockchain, the process further includes: Data terminal acquires training data; The data terminal encrypts the training data to obtain encrypted training data. The data terminal sends the encrypted training data to the consortium node; The alliance nodes store the encrypted training data on the blockchain.

3. The method according to claim 2, characterized in that, The data terminal encrypts the training data to obtain encrypted training data, including: The data terminal uses a pre-issued public key to encrypt the training data to obtain encrypted training data.

4. The method according to claim 3, characterized in that, Each consortium node, in a trusted execution environment, decrypts the encrypted training data, including: Each consortium node, in a trusted execution environment, uses a pre-issued private key to decrypt the encrypted training data to obtain the decrypted training data; wherein the private key and the public key belong to a public-private key pair.

5. The method according to claim 4, characterized in that, Also includes: Each data terminal initiates a blockchain registration request to the central node through its corresponding alliance node; The central node generates a public-private key pair based on the registration request; The public key in the public-private key pair is sent to the corresponding data terminal, and the private key in the public-private key pair is sent to the corresponding consortium node.

6. The method according to claim 2, characterized in that, The consortium nodes store the encrypted training data on the blockchain, including: The consortium nodes store the encrypted training data on-chain in the interplanetary file system of the blockchain.

7. The method according to claim 6, characterized in that, After the consortium node stores the encrypted training data on-chain in the InterPlanetary File System (IPS) of the blockchain, the method further includes: The consortium nodes calculate a first information digest of the encrypted training data; After adding a timestamp to the first message digest, it is packaged into a block and stored in the block record section of the blockchain.

8. The method according to claim 6, characterized in that, In each consortium node, within the trusted execution environment, after training the fine-tuning model parameters of the adapter module using the training data to obtain the updated fine-tuning model parameters, the process further includes: Each consortium node computes a second information digest of the training data; After adding timestamps to the second message digest and the digest of the adapter module, they are packaged into a block and stored in the block record section of the blockchain.

9. The method according to any one of claims 1 to 8, characterized in that, The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node, including: The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node based on the transport layer security protocol.

10. The method according to any one of claims 1 to 8, characterized in that, The central node aggregates the fine-tuned model parameters uploaded by all alliance nodes to obtain aggregated fine-tuned model parameters, including: The central node calculates the average value of the fine-tuned model parameters uploaded by all alliance nodes, and determines the average value as the aggregated fine-tuned model parameters.

11. The method according to any one of claims 1 to 8, characterized in that, The central node aggregates the fine-tuned model parameters uploaded by all alliance nodes to obtain aggregated fine-tuned model parameters, including: The central node obtains the pre-stored weight coefficients of each alliance node; The central node calculates a weighted average value based on the weight coefficient of each alliance node and the fine-tuning model parameters, and determines the aggregated fine-tuning model parameters based on the weighted average value.

12. A training device for a large model, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 11.

Images