Blockchain-based large model training method and device, equipment and storage medium

Abstract

The application provides a large model training method and device based on a blockchain, equipment and a storage medium. The blockchain comprises a center node and a plurality of alliance nodes. Each alliance node corresponds to a data terminal. The method comprises the following steps: the center node acquires a pre-trained large model, acquires an adapter module corresponding to the large model, and sends the adapter module to each alliance node; each alliance node adds the adapter module to a trusted execution environment; encrypted training data is acquired on the blockchain; the encrypted training data is decrypted in the trusted execution environment, and the decrypted training data is used to train fine-tuning model parameters of the adapter module; the updated fine-tuning model parameters are uploaded to the center node; the center node performs aggregation processing on the fine-tuning model parameters; the pre-trained large model is updated according to the aggregated fine-tuning model parameters to obtain an updated large model, and the process is repeated until the updated large model converges.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and specifically to a large-scale model training method, apparatus, equipment, and storage medium based on blockchain. Background Art

[0002] With the further development of big model technology and big data, the emphasis on data privacy and security has become a global trend. In the financial sector, an increasing number of generative AI applications based on big models are being implemented. The financial business data involved is often closely related to personal privacy and industry secrets.

[0003] In related technologies, in order to ensure data security, the traditional federated learning method is adopted to collaboratively train a large machine learning model through multiple devices or data sources, and to encrypt data during the model training process through multi-party secure computing protocols such as homomorphic encryption.

[0004] However, during the training of large models, there is a technical problem of low data security. Summary of the Invention

[0005] The present application provides a blockchain-based large-model training method, apparatus, equipment, and storage medium, thereby solving the problem of low data security during large-model training.

[0006] In a first aspect, the present application provides a large model training method based on a blockchain, wherein the blockchain includes a central node and multiple alliance nodes; each alliance node corresponds to a data terminal, and the method includes:

[0007] The central node obtains a pre-trained large model;

[0008] The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node; wherein the adapter module includes the fine-tuning model parameters of the pre-trained large model;

[0009] Each alliance node adds the adapter module to the trusted execution environment;

[0010] Each alliance node obtains encrypted training data on the blockchain; wherein the training data is obtained by the data terminal corresponding to each alliance node, encrypted, and sent to the alliance node, and stored on the blockchain by the alliance node;

[0011] Each alliance node decrypts the encrypted training data in the trusted execution environment and uses the decrypted training data to train the fine-tuning model parameters of the adapter module to obtain updated fine-tuning model parameters; each alliance node uploads the updated fine-tuning model parameters to the central node;

[0012] The central node aggregates the fine-tuning model parameters uploaded by all alliance nodes to obtain aggregated fine-tuning model parameters;

[0013] The central node updates the pre-trained large model according to the aggregated fine-tuning model parameters to obtain an updated large model;

[0014] If the updated large model does not converge, the central node continues to use the updated large model as the pre-trained large model and restarts the step of "obtaining the adapter module corresponding to the large model and sending the adapter module to each alliance node" until the updated large model converges to complete the training of the large model.

[0015] This application proposes a large model training method based on blockchain. The entire process of large model training is carried out on the blockchain. The central node is used to send the adapter module corresponding to the large model to the alliance node. After receiving the adapter module corresponding to the large model, the alliance node joins the trusted execution environment (TEE). The training data is encrypted by the data terminal and uploaded to the alliance node. The alliance node performs decryption of the training data in the TEE and efficiently fine-tunes the parameters of the adapter module through the decrypted training data. After the training is completed, it is sent to the central node for aggregation processing to complete the training of the entire large model. Through training in the TEE environment, the training process of the model and the plaintext of the training data in a alliance node are invisible to other blockchain participants, which can ensure the security of the training data, model data and data during the training process. In addition, the present application uses plaintext data for training, which is more efficient than model parameter aggregation based on multi-party secure computing protocols such as homomorphic encryption, thereby improving the efficiency of model training.

[0016] Optionally, before each alliance node obtains the encrypted training data on the blockchain, the following steps may be performed:

[0017] The data terminal obtains training data; the data terminal encrypts the training data to obtain encrypted training data; the data terminal sends the encrypted training data to the alliance node; the alliance node stores the encrypted training data on the blockchain.

[0018] Here, the data terminal of the present application corresponds to an alliance node on the blockchain. The data terminal encrypts the training data and sends it to the corresponding alliance node on the blockchain, which is then stored on the blockchain by the alliance node. The alliance node can replace the data terminal to realize federated learning of the parameters of the large model. The data terminal cannot obtain the data of the large model, which ensures the security of the large model. Due to the encryption of the data, the alliance node on the blockchain cannot obtain the training data of the data terminals corresponding to other alliance nodes, and the data security is high. Furthermore, after receiving the training data, the alliance node of the present application stores the encrypted training data on the blockchain, which realizes the traceability of the data source and improves the security of the data.

[0019] Optionally, the data terminal encrypts the training data to obtain encrypted training data, including:

[0020] The data terminal encrypts the training data using the pre-issued public key to obtain encrypted training data.

[0021] Among them, the data terminal of the present application can use the issued public key to encrypt the training data. The encryption method is simple, and the public key encryption scheme is safe and reliable, which can effectively ensure the data security of the training data.

[0022] Optionally, each alliance node decrypts the encrypted training data in a trusted execution environment, including:

[0023] Each alliance node, in a trusted execution environment, uses a pre-issued private key to decrypt the encrypted training data to obtain decrypted training data; wherein the private key and the public key form a public-private key pair.

[0024] Among them, the process of decrypting the encrypted training data by the alliance node of this application is executed in TEE, which provides reliable protection for the training data. In order to decrypt the training data encrypted with the public key, the alliance node uses the private key belonging to the same public-private key pair as the public key to decrypt. The data security is high, the encryption and decryption methods are simple, and the training efficiency of large models is also improved.

[0025] Optionally, it also includes:

[0026] Each data terminal initiates a blockchain registration request to the central node through the corresponding alliance node; the central node generates a public-private key pair based on the registration request; sends the public key in the public-private key pair to the corresponding data terminal, and sends the private key in the public-private key pair to the corresponding alliance node.

[0027] Here, the data terminal of the present application can initiate registration with the central node of the blockchain through its corresponding alliance node. The central node acts as a certificate authority (CA for short). After receiving the registration request, it can generate a public-private key pair for the data terminal and the alliance node corresponding to the data terminal, and send the public key to the data terminal and the private key to the alliance node. The data terminal and the alliance node can achieve secure and reliable data transmission based on the public-private key pair. The training data encrypted by the public-private key pair is invisible to other devices and nodes, thereby improving the security of the training data.

[0028] Optionally, the alliance node stores the encrypted training data on the blockchain, including:

[0029] The alliance node stores the encrypted training data on-chain in the InterPlanetary File System of the blockchain.

[0030] Among them, the present application can realize the storage of encrypted training data on the blockchain based on the InterPlanetary File System (IPFS) distributed storage protocol. IPFS is completely based on decentralization. It does not require users to trust the identity and authority of any node, nor does it require any node to use specific encryption and authentication mechanisms. The user's data can only be controlled and accessed by the user himself, and will not be leaked by any node, thereby protecting the user's privacy and rights. The user's data is verified by hash value and digital signature, and will not be tampered with or deleted by any node, ensuring that the user's data is reliable and recoverable. Therefore, the security of the training data stored on the blockchain is guaranteed.

[0031] Optionally, after the alliance node stores the encrypted training data on-chain in the InterPlanetary File System of the blockchain, the process further includes:

[0032] The alliance node calculates a first information digest of the encrypted training data; adds a timestamp to the first information digest, packages it into a block, and stores it in the block record part of the blockchain.

[0033] Here, after batch hashing the training data, the present application obtains the hash value of the training data as the first information summary. The first information summary is timestamped and packaged into blocks and stored in the block record part of the blockchain, thereby recording the source of the training data and preventing the training data from being tampered with. Due to the irreversibility of the data after hashing, the traceability of the data is guaranteed, and the security of the data is also guaranteed.

[0034] Optionally, after each alliance node, in the trusted execution environment, uses the training data to train the fine-tuning model parameters of the adapter module to obtain updated fine-tuning model parameters, the method further includes:

[0035] Each alliance node calculates a second information digest of the training data; adds a timestamp to the second information digest and the digest of the adapter module, packages them into blocks, and stores them in the block record part of the blockchain.

[0036] Here, during the training of this application, the correspondence between the hash of the training data used in this training and the summary of the adapter module is recorded in the block record part of the blockchain, and a timestamp is added, thereby realizing the traceability of the data throughout the training process and preventing the data from being tampered with during the training process. Due to the irreversibility of the data after hash processing, data leakage is also prevented, further ensuring the data security of large model training.

[0037] Optionally, the central node obtains an adapter module corresponding to the large model and sends the adapter module to each alliance node, including:

[0038] The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node based on the transport layer security protocol.

[0039] Among them, the central node of this application sends the adapter module to each alliance node through the Transport Layer Security (TLS) protocol. This protocol protects the security and privacy of network communications through mechanisms such as encryption, data integrity and authentication, thereby improving the transmission security of large model data and further improving data security during large model training.

[0040] Optionally, the central node aggregates the fine-tuning model parameters uploaded by all alliance nodes to obtain aggregated fine-tuning model parameters, including:

[0041] The central node calculates an average value of the fine-tuning model parameters uploaded by all alliance nodes, and determines the average value as the aggregated fine-tuning model parameters.

[0042] Among them, the central node of this application can use the method of calculating the average value to realize the aggregation processing of the fine-tuning model parameters uploaded by the alliance nodes. This method can quickly realize the aggregation processing of the fine-tuning model parameters sent by multiple alliance nodes, thereby improving the efficiency of large model training.

[0043] Optionally, the central node aggregates the fine-tuning model parameters uploaded by all alliance nodes to obtain aggregated fine-tuning model parameters, including:

[0044] The central node obtains the pre-stored weight coefficient of each alliance node; the central node calculates a weighted average value based on the weight coefficient of each alliance node and the fine-tuning model parameters, and determines the weighted average value as the aggregated fine-tuning model parameters.

[0045] Among them, the central node of the present application can realize the aggregation processing of the fine-tuning model parameters uploaded by the alliance nodes by calculating the weighted average of the pre-stored weight coefficients of each alliance node. The weight coefficient can be assigned according to the amount of local data on the data terminal, that is, the size of the training data. The more data the data terminal has, the greater the weight of the corresponding alliance node. This can fully consider the impact of the size of the data on the training results, and further improve the accuracy of large model training.

[0046] In a second aspect, the present application provides a large-scale model training device based on a blockchain, wherein the blockchain includes a central node and multiple alliance nodes; each alliance node corresponds to a data terminal, and the device includes:

[0047] A first acquisition module is used for the central node to acquire a pre-trained large model;

[0048] A first processing module is configured for the central node to obtain an adapter module corresponding to the large model and send the adapter module to each alliance node; wherein the adapter module includes fine-tuning model parameters of the pre-trained large model;

[0049] a second processing module, configured for each federation node, to add the adapter module to a trusted execution environment;

[0050] A second acquisition module is configured for each alliance node to acquire encrypted training data on the blockchain; wherein the training data is acquired by a data terminal corresponding to each alliance node, encrypted, and sent to the alliance node, which then uploads the data to the blockchain for storage;

[0051] a third processing module, configured for each alliance node to decrypt the encrypted training data in the trusted execution environment, and train the fine-tuning model parameters of the adapter module using the decrypted training data to obtain updated fine-tuning model parameters;

[0052] An uploading module, used for each alliance node to upload the updated fine-tuning model parameters to the central node;

[0053] An aggregation module, configured for the central node to aggregate the fine-tuning model parameters uploaded by all alliance nodes to obtain aggregated fine-tuning model parameters;

[0054] An updating module, configured for the central node to update the pre-trained large model according to the aggregated fine-tuning model parameters to obtain an updated large model;

[0055] The fourth processing module is used to, if the updated large model does not converge, cause the central node to continue to use the updated large model as the pre-trained large model and restart the step of "obtaining the adapter module corresponding to the large model and sending the adapter module to each alliance node" until the updated large model converges to complete the training of the large model.

[0056] Optionally, before the second acquisition module is used by each alliance node to acquire the encrypted training data on the blockchain, the method further includes:

[0057] The third acquisition module is used for the data terminal to acquire training data;

[0058] an encryption module, configured for the data terminal to encrypt the training data to obtain encrypted training data;

[0059] a sending module, configured for the data terminal to send the encrypted training data to the alliance node;

[0060] The first uploading module is used for the alliance node to upload the encrypted training data and store it on the blockchain.

[0061] Optionally, the encryption module is specifically used to:

[0062] The data terminal encrypts the training data using the pre-issued public key to obtain encrypted training data.

[0063] Optionally, the third processing module is specifically configured to:

[0064] Each alliance node, in a trusted execution environment, uses a pre-issued private key to decrypt the encrypted training data to obtain decrypted training data; wherein the private key and the public key form a public-private key pair.

[0065] Optionally, it also includes:

[0066] A registration module is used for each data terminal to initiate a blockchain registration request to the central node through the corresponding alliance node;

[0067] The public-private key pair sending module is used for the central node to generate a public-private key pair according to the registration request; send the public key in the public-private key pair to the corresponding data terminal, and send the private key in the public-private key pair to the corresponding alliance node.

[0068] Optionally, the first uploading module is specifically configured to:

[0069] The alliance node stores the encrypted training data on-chain in the InterPlanetary File System of the blockchain.

[0070] Optionally, after the first uploading module is used to upload the encrypted training data to the InterPlanetary File System of the blockchain, the apparatus further includes a second uploading module, which is used to:

[0071] The alliance node calculates a first information digest of the encrypted training data; adds a timestamp to the first information digest, packages it into a block, and stores it in the block record part of the blockchain.

[0072] Optionally, after the third processing module is used for each alliance node of the third processing module to train the fine-tuning model parameters of the adapter module in the trusted execution environment using the training data to obtain updated fine-tuning model parameters, a third uploading module is further included, which is used to:

[0073] Each alliance node calculates a second information digest of the training data; adds a timestamp to the second information digest and the digest of the adapter module, packages them into blocks, and stores them in the block record part of the blockchain.

[0074] Optionally, the first processing module is specifically configured to:

[0075] The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node based on the transport layer security protocol.

[0076] Optionally, the aggregation module is specifically configured to:

[0077] The central node calculates an average value of the fine-tuning model parameters uploaded by all alliance nodes, and determines the average value as the aggregated fine-tuning model parameters.

[0078] Optionally, the aggregation module is further specifically configured to:

[0079] The central node obtains the pre-stored weight coefficient of each alliance node; the central node calculates a weighted average value based on the weight coefficient of each alliance node and the fine-tuning model parameters, and determines the weighted average value as the aggregated fine-tuning model parameters.

[0080] In a third aspect, the present application provides a large-scale model training device based on blockchain, comprising: at least one processor and a memory;

[0081] The memory stores computer-executable instructions;

[0082] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the blockchain-based large model training method as described in the first aspect and various possible designs of the first aspect.

[0083] In a fourth aspect, the present invention provides a computer-readable storage medium, which stores computer-executable instructions. When a processor executes the computer-executable instructions, the blockchain-based large model training method as described in the first aspect and various possible designs of the first aspect is implemented.

[0084] In a fifth aspect, the present invention provides a computer program product, comprising a computer program. When the computer program is executed by a processor, it implements the blockchain-based large model training method as described in the first aspect and various possible designs of the first aspect.

[0085] The present application provides a blockchain-based large model training method, device, equipment and storage medium, wherein the entire process of large model training of this method is carried out on the blockchain, and the central node is used to send the Adaptor module corresponding to the large model to the alliance node. After receiving the adapter module corresponding to the large model, the alliance node joins the TEE, and the training data is encrypted by the data terminal and uploaded to the alliance node. The alliance node performs decryption of the training data in the TEE and efficiently fine-tunes the parameters of the Adaptor module through the decrypted training data. After the training is completed, it is sent to the central node for aggregation processing to complete the training of the entire large model. Through training in the TEE environment, the training process of the model in a alliance node and the plaintext of the training data are invisible to other blockchain participants, which can ensure the security of the training data, model data and data during the training process. In addition, the present application uses plaintext data for training, which is more efficient than the model parameter aggregation based on multi-party secure computing protocols such as homomorphic encryption, thereby improving the efficiency of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0087] Figure 1 A schematic diagram of the structure of a large model training system based on blockchain provided in an embodiment of the present application;

[0088] Figure 2 A schematic diagram of the structure of an alliance node providing an embodiment of the present application that efficiently fine-tunes the execution model through a TEE;

[0089] Figure 3 A flowchart of a large-scale model training method based on blockchain provided in an embodiment of the present application;

[0090] Figure 4 A flowchart of another blockchain-based large model training method provided in an embodiment of the present application;

[0091] Figure 5 A schematic diagram of the structure of a large-scale model training device based on blockchain provided in an embodiment of the present application;

[0092] Figure 6 A schematic structural diagram of a blockchain-based large-model training device 600 provided in an embodiment of the present application.

[0093] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0094] Exemplary embodiments are described in detail herein, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of the present application.

[0095] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0096] Data security for large models is crucial. Related technologies offer several approaches to training large models: for example, blockchain-based federated learning systems and methods update model parameters through multiple rounds of parameter aggregation until convergence. Differently, the centralized server role is replaced with nodes randomly selected from the blockchain. Alternatively, a blockchain-based federated learning defense approach is proposed. Each participating node in the blockchain retrieves the latest model from the blockchain and downloads it locally. Training is performed using local data, and the training is broadcast to the blockchain after completion, along with the training time. During training, noise is added to the local model, and two validators are established for participating nodes. An authoritative organization obtains and aggregates all verified updated models from the blockchain nodes. Based on the aforementioned solutions, it can be seen that related technologies often employ traditional federated learning methods, using multiple devices or data sources to collaboratively train a large machine learning model. Considering the large-scale model training methods described above, the main technical issue with existing technologies during large-scale model training is the lack of data security.

[0097] In order to solve the above problems, the embodiments of the present application provide a blockchain-based large model training method, device, equipment and medium. The entire process of large model training in this method is carried out on the blockchain. The central node is used to send the Adaptor module corresponding to the large model to the alliance node. After receiving the adapter module corresponding to the large model, the alliance node adds it to the TEE. The training data is encrypted by the data terminal and uploaded to the alliance node. The alliance node performs decryption of the training data in the TEE and efficiently fine-tunes the parameters of the Adaptor module through the decrypted training data. After the training is completed, it is sent to the central node for aggregation processing to complete the training of the entire large model.

[0098] Optionally, the blockchain in the embodiment of the present application is a consortium chain. As a blockchain technology among specific groups, the consortium chain has the characteristics of high efficiency, security, privacy, and scalability, and can improve the security and efficiency of large model training.

[0099] Optionally, the embodiments of the present application can achieve the following functions and effects:

[0100] (1) Data sources are traceable to avoid data poisoning. The entire process of federated large-scale model training is carried out on the blockchain. The data and models used in each round of model training are timestamped and encrypted and stored on the blockchain, making the data source irrefutable.

[0101] (2) Data and model privacy protection. For each round of training on the alliance node, the model's training parameters are decrypted in the TEE. The training data is encrypted and uploaded to the alliance node by the data terminal. During training, the alliance node reads the data into the TEE through the IPFS protocol using a distributed hash table and decrypts it. The efficient fine-tuning of the Adaptor model parameters is performed in the TEE. In other words, the training process and plaintext training data of the model in a alliance node are not visible to other blockchain participants.

[0102] (3) Improve the time efficiency of model training. Performing plaintext model training in a TEE environment is more efficient than aggregating model parameters based on homomorphic encryption or multi-party secure computing. In addition, cloud servers with TEE chip functions are becoming increasingly popular and will become more universal and standardized in the future.

[0103] Optionally, Figure 1 This is a schematic diagram of the structure of a large-scale model training system based on blockchain provided in an embodiment of the present application. The blockchain includes a central node and multiple alliance nodes; each alliance node corresponds to a data terminal. The system is a three-layer blockchain distributed large-scale model training mode, such as Figure 1 As shown, the blockchain-based large model training system provided in the embodiment of the present application includes three layers: the first layer is the blockchain center layer, which includes the central node 101; the second layer is the layer where the alliance node is located; Figure 1 In the example, four alliance nodes (first alliance node 102, second alliance node 103, third alliance node 104 and fourth alliance node 105) are used. The third layer is where the data terminal is located. Figure 1 In the example, there are four data terminals (a first data terminal 106, a second data terminal 107, a third data terminal 108 and a fourth data terminal 109). It can be understood that: Figure 1 This is only for illustration. The structure of the blockchain-based large-scale model training system and the number of alliance nodes and data terminals can be determined according to actual conditions, and the embodiments of this application do not impose specific restrictions.

[0104] in, Figure 1 The central node and all alliance nodes are on a blockchain\alliance chain, and communication can be achieved through the blockchain\alliance chain.

[0105] Among them, one data terminal corresponds to one alliance node, for example Figure 1 The alliance node corresponding to the first data terminal 106 is the first alliance node 102. The data terminal can communicate with its corresponding alliance node, and the data terminal can join the blockchain through its corresponding alliance node.

[0106] Based on the above structure, the embodiments of the present application can achieve the following functions through various optional implementation methods:

[0107] Optional implementation method 1 and its effect: Introduce the blockchain central layer, i.e. the central node, to ensure that data and models do not leave the blockchain. Figure 1 Data terminals are decoupled from central nodes, with the central layer server (central node) located on the blockchain. After the data terminal uploads encrypted data to the federated node, subsequent model training is performed on-chain. This means that the model is not trained locally on the data holder, effectively preventing faulty participants in federated learning from poisoning the data and contaminating the global model.

[0108] Optional Implementation Method 2 and Its Effect: Training data ownership is traceable. Training data provided by all data terminals is hashed and timestamped in batches and uploaded to the blockchain. During training, the corresponding relationship between the data hash and the model summary used in that training session is recorded and timestamped on the blockchain's distributed ledger. Furthermore, data privacy is protected by encrypted data uploaded to the blockchain.

[0109] Optional implementation method three and its effect: Trusted storage and data interoperability: This solution is based on the IPFS distributed storage protocol:

[0110] Supporting on-chain data sharing, cross-chain services can enable trusted intercommunication of homogeneous / heterogeneous chain data through authorization, breaking the data silos of a single chain and ensuring the authenticity of the data is not tampered with during the circulation process.

[0111] Based on the original plaintext sharing of IPFS, an encrypted storage and sharing method was designed. Data is decrypted in a trusted execution environment, smart contracts are executed, and parameter-efficient fine-tuning (PEFT) is performed on the large model adapter.

[0112] Because traditional internet security is centralized, it requires users to trust the server's identity and authority, as well as the encryption and authentication mechanisms used by the server. This approach has several disadvantages: user data could be leaked, stolen, or sold by the server, compromising user privacy and rights; user data could be tampered with or deleted by the server, making it unreliable or irrecoverable; and user data could be censored or blocked by the server, resulting in data freedom and unfairness. IPFS, on the other hand, is decentralized and does not require users to trust the identity and authority of any node, nor does it require any node to use specific encryption and authentication mechanisms. This approach has several advantages: user data is controlled and accessible only by the user and cannot be leaked, stolen, or sold by any node, thus protecting user privacy and rights; user data is verified using hash values ​​and digital signatures, preventing it from being tampered with or deleted by any node, ensuring its reliability and recoverability; and user data is transmitted using efficient content-addressing and routing mechanisms, preventing it from being censored or blocked by any node. IPFS supports encrypted transmission and can protect the security of transmitted data using protocols such as TLS, preventing eavesdropping and tampering.

[0113] Optionally, Figure 2 A schematic diagram of a structure in which an alliance node efficiently fine-tunes a model through TEE execution is provided in an embodiment of the present application, such as Figure 2 As shown, consortium nodes can add an adapter module, which includes adapter parameters, to the Trusted Execution Environment (TEE). Fine-tuning the adapter module is then performed within the TEE. After fine-tuning, the fine-tuned adapter module is taken outside the TEE to adjust the Local Large Language Model (LLM). Due to the security and isolation of the TEE, the training process and plaintext training data of a consortium node's model are invisible to other blockchain participants (consortium nodes), improving the security of large-scale model training.

[0114] It is understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the architecture of a large-scale model training system based on blockchain. In other feasible implementations of this application, the above architecture may include more or fewer components than shown in the figure, or combine or split certain components, or arrange the components differently. The specific configuration can be determined according to the actual application scenario and is not limited here. Figure 1 and Figure 2 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0115] In the specific implementation process, the above-mentioned central node, data terminal and alliance node include a processor or processing device.

[0116] It should be understood that the above-mentioned processing device can be implemented by a processor reading instructions in a memory and executing the instructions, or it can be implemented by a chip circuit.

[0117] In addition, the network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0118] The technical solution of this application is described in detail below with reference to specific embodiments:

[0119] Optionally, Figure 3 A flowchart of a large model training method based on blockchain provided in the embodiment of this application. The execution subject of the embodiment of this application can be Figure 1 The specific execution entity of the blockchain-based large model training system can be determined according to the actual application scenario. Figure 3 As shown, the method includes the following steps:

[0120] S301: The central node obtains the pre-trained large model.

[0121] Among them, the pre-trained large model is a large model that needs to be trained.

[0122] S302: The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node.

[0123] Among them, the adapter module includes the fine-tuning model parameters of the pre-trained large model.

[0124] Optionally, the adapter module can be queried and obtained through a preset insertion position of the adapter module.

[0125] Adapter Modules for large models are a technology used for fine-tuning in neural networks, especially large pre-trained language models. Adapter Modules are a group of small network layers designed to be embedded in existing pre-trained models to achieve adaptive learning for specific tasks. Adapter is essentially a part of the specific parameters in the original large model, and its parameter amount accounts for only a few percent or a few thousandths of the entire original large model. Only fine-tuning can achieve almost the same effect as fine-tuning all parameters of the entire model. By inserting small network layers into the pre-trained large model and only updating the parameters of these layers, Adapter Modules can learn the characteristics of specific tasks without changing the main structure of the model, thereby improving the flexibility and efficiency of the model. The embodiment of the present application only needs to adjust the parameters of the Adapter Modules to achieve training of the entire large model, further ensuring the data security of the large model.

[0126] Optionally, the central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node, including:

[0127] The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node based on the transport layer security protocol.

[0128] Among them, the central node of the embodiment of the present application sends the adapter module to each alliance node through the TLS protocol. The protocol protects the security and privacy of network communications through mechanisms such as encryption, data integrity and authentication, improves the transmission security of large model data, and further improves the data security during the large model training process.

[0129] S303: Each alliance node adds the adapter module to the trusted execution environment.

[0130] like Figure 2 As shown, the alliance node in the embodiment of the present application fine-tunes the Adapter Modules within the TEE to ensure the data security of the parameters in the Adapter Modules.

[0131] S304: Each alliance node obtains encrypted training data on the blockchain.

[0132] Among them, the training data is obtained by the data terminal corresponding to each alliance node, encrypted and sent to the alliance node, and then stored on the blockchain by the alliance node.

[0133] S305: Each alliance node decrypts the encrypted training data in the trusted execution environment and uses the decrypted training data to train the fine-tuning model parameters of the adapter module to obtain updated fine-tuning model parameters; each alliance node uploads the updated fine-tuning model parameters to the central node.

[0134] Specifically, the fine-tuning model parameters here are obtained after training within the TEE. After obtaining the updated fine-tuning model parameters, the updated fine-tuning model parameters are taken to the alliance node outside the TEE and uploaded to the central node.

[0135] Optionally, each alliance node uploads the updated fine-tuning model parameters to the central node based on the TLS protocol.

[0136] S306: The central node aggregates the fine-tuning model parameters uploaded by all alliance nodes to obtain aggregated fine-tuning model parameters.

[0137] Optionally, the central node aggregates the fine-tuning model parameters uploaded by all alliance nodes to obtain aggregated fine-tuning model parameters, including:

[0138] The central node calculates the average value of the fine-tuning model parameters uploaded by all alliance nodes and determines the average value as the aggregated fine-tuning model parameters.

[0139] Among them, the central node of the embodiment of the present application can use the method of calculating the average value to realize the aggregation processing of the fine-tuning model parameters uploaded by the alliance nodes. This method can quickly realize the aggregation processing of the fine-tuning model parameters sent by multiple alliance nodes, thereby improving the efficiency of large model training.

[0140] Optionally, the central node aggregates the fine-tuning model parameters uploaded by all alliance nodes to obtain aggregated fine-tuning model parameters, including:

[0141] The central node obtains the pre-stored weight coefficient of each alliance node; the central node calculates a weighted average value based on the weight coefficient of each alliance node and the fine-tuning model parameters, and determines the weighted average value as the aggregated fine-tuning model parameters.

[0142] It is understandable that the weight coefficient here can be determined according to actual conditions, and is not specifically limited in the embodiments of the present application.

[0143] Optionally, the amount of training data corresponding to each alliance node may be obtained, and a weight coefficient may be determined for each alliance node based on the ratio of the amount of training data corresponding to each alliance node.

[0144] Among them, the central node of the embodiment of the present application can realize the aggregation processing of the fine-tuning model parameters uploaded by the alliance nodes by calculating the weighted average of the pre-stored weight coefficients of each alliance node. The weight coefficient can be assigned according to the amount of local data on the data terminal, that is, the size of the training data. The more data the data terminal has, the greater the weight of the corresponding alliance node. This can fully consider the impact of the size of the data on the training results, and further improve the accuracy of large model training.

[0145] Optionally, the Federated Averaging (FedAvg) algorithm is used for model aggregation. The central node collects the weight updates of the Adapter Modules of all alliance nodes, calculates the average of the weights, and distributes the updated weights back to each alliance node. This process is carried out in multiple rounds until convergence is reached. In order to ensure the accuracy of model aggregation, the FedAvg algorithm uses weighted averaging to perform model aggregation. Specifically, the model parameters uploaded by each alliance node will be assigned a weight, and then weighted averaged. The weight of the model parameters uploaded by the alliance node is assigned according to the amount of local data on the device. The device with more data has a greater weight.

[0146] S307: The central node updates the pre-trained large model according to the aggregated fine-tuning model parameters to obtain an updated large model.

[0147] S308: If the updated large model does not converge, the central node continues to use the updated large model as the pre-trained large model and restarts the step of "obtaining the adapter module corresponding to the large model and sending the adapter module to each alliance node" until the updated large model converges to complete the training of the large model.

[0148] This application proposes a large model training method based on blockchain. The entire process of large model training is carried out on the blockchain. The central node is used to send the Adaptor module corresponding to the large model to the alliance node. After receiving the adapter module corresponding to the large model, the alliance node adds it to the TEE. The training data is encrypted by the data terminal and uploaded to the alliance node. The alliance node performs decryption of the training data in the TEE and efficiently fine-tunes the parameters of the Adaptor module through the decrypted training data. After the training is completed, it is sent to the central node for aggregation processing to complete the training of the entire large model. Through training in the TEE environment, the training process of the model in a alliance node and the plaintext of the training data are invisible to other blockchain participants, which can ensure the security of the training data, model data and data during the training process. In addition, this application uses plaintext data for training, which is more efficient than model parameter aggregation based on multi-party secure computing protocols such as homomorphic encryption, thereby improving the efficiency of model training.

[0149] Optionally, the embodiment of the present application encrypts the training data. Accordingly, Figure 4 A flowchart of another large-scale model training method based on blockchain provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the method includes:

[0150] S401: The central node obtains the pre-trained large model.

[0151] S402: The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node.

[0152] Among them, the adapter module includes the fine-tuning model parameters of the pre-trained large model.

[0153] S403: Each alliance node adds the adapter module to the trusted execution environment.

[0154] The implementation of steps S401 to S403 is the same as that of steps S301 and S303, and will not be described in detail here.

[0155] S404: The data terminal obtains training data; the data terminal encrypts the training data to obtain encrypted training data.

[0156] Optionally, the training data is stored in a local storage location of the data terminal, and the data terminal can directly obtain the training data from the local storage location.

[0157] Optionally, the data terminal encrypts the training data to obtain encrypted training data, including:

[0158] The data terminal encrypts the training data using the pre-issued public key to obtain encrypted training data.

[0159] Among them, the data terminal of the embodiment of the present application can use the issued public key to encrypt the training data. The encryption method is simple, and the public key encryption scheme is safe and reliable, which can effectively ensure the data security of the training data.

[0160] Optionally, based on the above public key encryption method, each alliance node decrypts the encrypted training data in the trusted execution environment, including:

[0161] Each alliance node, in a trusted execution environment, uses a pre-issued private key to decrypt the encrypted training data to obtain the decrypted training data; the private key and public key belong to a public-private key pair.

[0162] Among them, the process of decrypting the encrypted training data by the alliance node in the embodiment of the present application is executed in TEE, which provides reliable protection for the training data. In order to decrypt the training data encrypted with the public key, the alliance node uses the private key belonging to the same public-private key pair as the public key to decrypt. The data security is high, the encryption and decryption methods are simple, and the training efficiency of large models is also improved.

[0163] Optionally, it also includes:

[0164] Each data terminal initiates a blockchain registration request to the central node through the corresponding alliance node; the central node generates a public-private key pair based on the registration request; the public key in the public-private key pair is sent to the corresponding data terminal, and the private key in the public-private key pair is sent to the corresponding alliance node.

[0165] Here, the data terminal of the embodiment of the present application can initiate registration with the central node of the blockchain through its corresponding alliance node. The central node acts as a CA. After receiving the registration request, it can generate a public-private key pair for the data terminal and the alliance node corresponding to the data terminal, and send the public key to the data terminal and the private key to the alliance node. The data terminal and the alliance node can achieve secure and reliable data transmission based on the public-private key pair. The training data encrypted by the public-private key pair is invisible to other devices and nodes, thereby improving the security of the training data.

[0166] S405: The data terminal sends the encrypted training data to the alliance node.

[0167] S406: The alliance node stores the encrypted training data on the blockchain.

[0168] Optionally, the alliance node stores the encrypted training data on the blockchain, including:

[0169] The alliance nodes store the encrypted training data on-chain in the InterPlanetary File System of the blockchain.

[0170] Among them, the embodiment of the present application can realize the storage of encrypted training data on the blockchain based on the IPFS distributed storage protocol. IPFS is completely decentralized. It does not require users to trust the identity and authority of any node, nor does it require any node to use specific encryption and authentication mechanisms. The user's data can only be controlled and accessed by the user himself, and will not be leaked by any node, thereby protecting the user's privacy and rights. The user's data is verified through hash values ​​and digital signatures and will not be tampered with or deleted by any node, ensuring that the user's data is reliable and recoverable. Therefore, the security of the training data stored on the blockchain is guaranteed.

[0171] The embodiment of the present application does not impose any specific restrictions on the order of steps S401-S403 and steps S404-S406. Steps S404-S406 may also be performed before steps S401-S403, or simultaneously, or crosswise.

[0172] Optionally, after the alliance node stores the encrypted training data on the blockchain's InterPlanetary File System, the following steps are also included:

[0173] The alliance node calculates the first information digest of the encrypted training data; after adding a timestamp to the first information digest, it is packaged into a block and stored in the block record part of the blockchain.

[0174] Here, in the embodiment of the present application, after batch hashing the training data, the hash value of the training data is obtained as the first information digest. The first information digest is timestamped and packaged into blocks and stored in the block record part of the blockchain, thereby recording the source of the training data and preventing the training data from being tampered with. Due to the irreversibility of the data after hashing, the traceability of the data is guaranteed, and the security of the data is also guaranteed.

[0175] S407: Each alliance node obtains encrypted training data on the blockchain.

[0176] Among them, the training data is obtained by the data terminal corresponding to each alliance node, encrypted and sent to the alliance node, and then stored on the blockchain by the alliance node.

[0177] S408: Each alliance node decrypts the encrypted training data in the trusted execution environment and uses the decrypted training data to train the fine-tuning model parameters of the adapter module to obtain updated fine-tuning model parameters; each alliance node uploads the updated fine-tuning model parameters to the central node.

[0178] Optionally, at each alliance node, in the trusted execution environment, after training the fine-tuning model parameters of the adapter module using the training data to obtain updated fine-tuning model parameters, the method further includes:

[0179] Each alliance node calculates the second information digest of the training data; after adding a timestamp to the second information digest and the adapter module's digest, it packages them into blocks and stores them in the block record part of the blockchain.

[0180] Here, during training in the embodiment of the present application, the correspondence between the hash of the training data used in this training and the summary of the adapter module is recorded in the block record part of the blockchain, and a timestamp is added, thereby achieving traceability of the data throughout the training process and preventing the data from being tampered with during the training process. Due to the irreversibility of the data after hash processing, data leakage is also prevented, further ensuring the data security of large model training.

[0181] S409: The central node aggregates the fine-tuning model parameters uploaded by all alliance nodes to obtain aggregated fine-tuning model parameters.

[0182] S410: The central node updates the pre-trained large model according to the aggregated fine-tuning model parameters to obtain an updated large model.

[0183] S411: If the updated large model does not converge, the central node continues to use the updated large model as the pre-trained large model and restarts the step of "obtaining the adapter module corresponding to the large model and sending the adapter module to each alliance node" until the updated large model converges to complete the training of the large model.

[0184] Among them, the implementation method of steps S407-S411 is the same as that of steps S304-S308, and will not be repeated here.

[0185] Here, the data terminal in the embodiment of the present application corresponds to an alliance node on the blockchain. The data terminal encrypts the training data and sends it to the corresponding alliance node on the blockchain, which is then stored on the blockchain by the alliance node. The alliance node can replace the data terminal to realize federated learning of the parameters of the large model. The data terminal cannot obtain the data of the large model, which ensures the security of the large model. Moreover, due to the encryption of the data, the alliance node on the blockchain cannot obtain the training data of the data terminals corresponding to other alliance nodes, and the data security is high. Furthermore, after receiving the training data, the alliance node in the embodiment of the present application stores the encrypted training data on the blockchain, which realizes the traceability of the data source and improves the security of the data.

[0186] In one possible implementation, the blockchain-based large-scale model training method works as follows:

[0187] Step 1: Alliance chain registration. Each alliance node and its corresponding data terminal register and join the alliance chain.

[0188] Step 2: Public key issuance. The trusted certificate authority, acting as the central node, generates a public-private key pair. The public key is issued to all alliance nodes on the alliance chain, which then sends it to the data terminal. The private key is then issued to alliance nodes on the alliance chain for storage.

[0189] Step 3: Data upload. Each alliance node will split the data used to train the federated model into batches, encrypt them with the public key, and upload them to the IPFS storage system on the alliance chain.

[0190] Step 4: Initialize the federated model. The training method used in this embodiment is a highly efficient fine-tuning training based on the traditional FedAvg algorithm. The fine-tuning method is to fine-tune the adapter FedAdaptor. The central node server first uploads a pre-trained large model and then freezes the parameters of the pre-trained large model except the Adaptor. The central node server sends the Adaptor to all federated node servers based on the TLS security protocol.

[0191] The server collects weight updates from all alliance nodes, calculates the average weight, and distributes the updated weights back to each alliance node. This process continues in multiple rounds until convergence is achieved. To ensure the accuracy of model aggregation, the FedAvg algorithm uses a weighted average approach for model aggregation. Specifically, the model parameters uploaded by each alliance node are assigned a weight, and then a weighted average is performed. The weights assigned to model parameters uploaded by alliance nodes are based on the amount of local data on the device, with devices with more data receiving higher weights.

[0192] Step 5: Alliance node model verification. In the previous step, all alliance nodes received the model sent by the central node and joined the TEE environment.

[0193] Step 6: Federated distributed model training. Each federated block node collects its own batch training data from the IPFS system and decrypts the data using the private key in the TEE, and then performs efficient fine-tuning of the Adaptor.

[0194] Step 7: Model aggregation and update. Each alliance node takes the model trained in the TEE in the previous step to the alliance chain node outside the TEE environment and uploads it to the central node based on the TLS protocol. After the central node receives the Adaptor model uploaded by each alliance node, it performs the model parameter aggregation and update in the FedAvg algorithm.

[0195] Step 8: Iterate the model until convergence. Repeat the above steps until the model converges.

[0196] Based on the above method, safe large model training is achieved.

[0197] Figure 5 A schematic diagram of the structure of a large model training device based on blockchain provided in an embodiment of the present application is shown as follows: Figure 5 As shown, the device of the embodiment of the present application includes: a first acquisition module 501, a first processing module 502, a second processing module 503, a second acquisition module 504, a third processing module 505, an upload module 506, an aggregation module 507, an update module 508 and a fourth processing module 509. The large-scale model training device based on blockchain here can be the processing device of the above-mentioned central node, alliance node or data terminal, the processor itself, or a chip or integrated circuit that implements the function of the processor. It should be noted here that the division of the first acquisition module 501, the first processing module 502, the second processing module 503, the second acquisition module 504, the third processing module 505, the upload module 506, the aggregation module 507, the update module 508 and the fourth processing module 509 is only a division of logical functions. Physically, the two can be integrated or independent.

[0198] Among them, the first acquisition module is used for the central node to obtain the pre-trained large model;

[0199] The first processing module is used for the central node to obtain the adapter module corresponding to the large model and send the adapter module to each alliance node; wherein the adapter module includes the fine-tuning model parameters of the pre-trained large model;

[0200] a second processing module, for each federation node, to add the adapter module to the trusted execution environment;

[0201] The second acquisition module is used by each alliance node to obtain encrypted training data on the blockchain. The training data is obtained by the data terminal corresponding to each alliance node, encrypted, and sent to the alliance node, which then stores it on the blockchain.

[0202] A third processing module is configured for each alliance node to decrypt the encrypted training data in the trusted execution environment and use the decrypted training data to train the fine-tuning model parameters of the adapter module to obtain updated fine-tuning model parameters.

[0203] The upload module is used by each alliance node to upload the updated fine-tuned model parameters to the central node;

[0204] Aggregation module, used by the central node to aggregate the fine-tuning model parameters uploaded by all alliance nodes to obtain aggregated fine-tuning model parameters;

[0205] The update module is used by the central node to update the pre-trained large model according to the aggregated fine-tuning model parameters to obtain an updated large model;

[0206] The fourth processing module is used to: if the updated large model does not converge, the central node continues to use the updated large model as the pre-trained large model and restarts the step of "obtaining the adapter module corresponding to the large model and sending the adapter module to each alliance node" until the updated large model converges to complete the training of the large model.

[0207] Optionally, before the second acquisition module is used by each alliance node to acquire the encrypted training data on the blockchain, it further includes:

[0208] The third acquisition module is used for the data terminal to acquire training data;

[0209] An encryption module is used for the data terminal to encrypt the training data to obtain encrypted training data;

[0210] The sending module is used for the data terminal to send the encrypted training data to the alliance node;

[0211] The first upload module is used by the alliance node to store the encrypted training data on the blockchain.

[0212] Optionally, the encryption module is specifically used to:

[0213] The data terminal encrypts the training data using the pre-issued public key to obtain encrypted training data.

[0214] Optionally, the third processing module is specifically configured to:

[0215] Each alliance node, in a trusted execution environment, uses a pre-issued private key to decrypt the encrypted training data to obtain the decrypted training data; the private key and public key belong to a public-private key pair.

[0216] Optionally, it also includes:

[0217] The registration module is used for each data terminal to initiate a blockchain registration request to the central node through the corresponding alliance node;

[0218] The public-private key pair distribution module is used by the central node to generate a public-private key pair based on a registration request; the public key in the public-private key pair is distributed to the corresponding data terminal, and the private key in the public-private key pair is distributed to the corresponding alliance node.

[0219] Optionally, the first uploading module is specifically configured to:

[0220] The alliance nodes store the encrypted training data on-chain in the InterPlanetary File System of the blockchain.

[0221] Optionally, after the first uploading module is used to store the encrypted training data on-chain in the InterPlanetary File System of the blockchain, the apparatus further includes a second uploading module, which is used to:

[0222] The alliance node calculates the first information digest of the encrypted training data; after adding a timestamp to the first information digest, it is packaged into a block and stored in the block record part of the blockchain.

[0223] Optionally, after the third processing module is used for each alliance node of the third processing module to train the fine-tuning model parameters of the adapter module using training data in a trusted execution environment to obtain updated fine-tuning model parameters, a third uploading module is further included, which is used to:

[0224] Each alliance node calculates the second information digest of the training data; after adding a timestamp to the second information digest and the adapter module's digest, it packages them into blocks and stores them in the block record part of the blockchain.

[0225] Optionally, the first processing module is specifically configured to:

[0226] The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node based on the transport layer security protocol.

[0227] Optionally, the aggregation module is specifically configured to:

[0228] The central node calculates the average value of the fine-tuning model parameters uploaded by all alliance nodes and determines the average value as the aggregated fine-tuning model parameters.

[0229] Optionally, the aggregation module is further configured to:

[0230] The central node obtains the pre-stored weight coefficient of each alliance node; the central node calculates a weighted average value based on the weight coefficient of each alliance node and the fine-tuning model parameters, and determines the weighted average value as the aggregated fine-tuning model parameters.

[0231] refer to Figure 6, which shows a schematic diagram of the structure of a blockchain-based large-model training device 600 suitable for implementing the embodiments of the present disclosure. The blockchain-based large-model training device 600 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The blockchain-based large model training device shown is only an example and should not bring any limitations to the functions and scope of use of the embodiments of the present disclosure.

[0232] like Figure 6 As shown, the large model training device 600 based on blockchain may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (Read Only Memory, referred to as ROM) 602 or the program loaded from the storage device 608 to the random access memory (Random Access Memory, referred to as RAM) 603. In RAM 603, various programs and data required for the operation of the large model training device 600 based on blockchain are also stored. The processing device 601, ROM 602 and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0233] Typically, the following devices can be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 609. The communication device 609 can allow the blockchain-based large model training device 600 to communicate with other devices wirelessly or wired to exchange data. Although Figure 6 The blockchain-based large model training device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0234] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0235] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0236] The above-mentioned computer-readable medium may be included in the above-mentioned blockchain-based large model training device; or it may exist independently without being assembled into the blockchain-based large model training device.

[0237] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the blockchain-based large model training device, the blockchain-based large model training device executes the method shown in the above-mentioned embodiment.

[0238] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0239] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0240] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0241] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0242] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0243] The blockchain-based large-model training device in the embodiment of the present application can be used to execute the technical solutions in the above-mentioned method embodiments of the present application. Its implementation principles and technical effects are similar and will not be repeated here.

[0244] An embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they are used to implement any of the above-mentioned blockchain-based large model training methods.

[0245] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement any of the above-mentioned blockchain-based large model training methods.

[0246] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0247] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0248] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0249] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A large model training method based on blockchain, characterized in that: The blockchain includes a central node and multiple alliance nodes; Each alliance node corresponds to a data terminal, and the method includes: The central node obtains a pre-trained large model; The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node; wherein the adapter module includes the fine-tuning model parameters of the pre-trained large model; Each alliance node adds the adapter module to the trusted execution environment; Each alliance node obtains encrypted training data on the blockchain; wherein the training data is obtained by the data terminal corresponding to each alliance node, encrypted, and sent to the alliance node, which is then uploaded and stored on the blockchain by the alliance node; Each alliance node decrypts the encrypted training data in the trusted execution environment, and uses the decrypted training data to train the fine-tuning model parameters of the adapter module to obtain updated fine-tuning model parameters; Each alliance node uploads the updated fine-tuning model parameters to the central node; The central node aggregates the fine-tuning model parameters uploaded by all alliance nodes to obtain aggregated fine-tuning model parameters; The central node updates the pre-trained large model according to the aggregated fine-tuning model parameters to obtain an updated large model; If the updated large model does not converge, the central node continues to use the updated large model as the pre-trained large model and restarts the step of "obtaining the adapter module corresponding to the large model and sending the adapter module to each alliance node" until the updated large model converges to complete the training of the large model.

2. The method according to claim 1, characterized in that Each alliance node, before obtaining the encrypted training data on the blockchain, further includes: The data terminal obtains training data; The data terminal encrypts the training data to obtain encrypted training data; The data terminal sends the encrypted training data to the alliance node; The alliance node stores the encrypted training data on the blockchain.

3. The method according to claim 2, characterized in that The data terminal encrypts the training data to obtain encrypted training data, including: The data terminal encrypts the training data using the pre-issued public key to obtain encrypted training data.

4. The method according to claim 3, characterized in that Each alliance node decrypts the encrypted training data in the trusted execution environment, including: Each alliance node, in a trusted execution environment, uses a pre-issued private key to decrypt the encrypted training data to obtain decrypted training data; wherein the private key and the public key form a public-private key pair.

5. The method according to claim 4, characterized in that Also includes: Each data terminal initiates a blockchain registration request to the central node through the corresponding alliance node; The central node generates a public-private key pair according to the registration request; The public key in the public-private key pair is sent to the corresponding data terminal, and the private key in the public-private key pair is sent to the corresponding alliance node.

6. The method according to claim 2, characterized in that The alliance node stores the encrypted training data on the blockchain, including: The alliance node stores the encrypted training data on-chain in the InterPlanetary File System of the blockchain.

7. The method according to claim 6, characterized in that After the alliance node stores the encrypted training data on-chain in the InterPlanetary File System of the blockchain, the process further includes: The alliance node calculates a first information digest of the encrypted training data; After adding a timestamp to the first information summary, it is packaged into a block and stored in the block record part of the blockchain.

8. The method according to claim 6, characterized in that After each alliance node, in the trusted execution environment, uses the training data to train the fine-tuning model parameters of the adapter module to obtain updated fine-tuning model parameters, the method further includes: Each alliance node calculates a second information summary of the training data; After adding a timestamp to the second information digest and the digest of the adapter module, they are packaged into blocks and stored in the block record part of the blockchain.

9. The method according to any one of claims 1 to 8, characterized in that The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node, including: The central node obtains the adapter module corresponding to the large model and sends the adapter module to each alliance node based on the transport layer security protocol.

10. The method according to any one of claims 1 to 8, characterized in that The central node aggregates the fine-tuning model parameters uploaded by all alliance nodes to obtain aggregated fine-tuning model parameters, including: The central node calculates an average value of the fine-tuning model parameters uploaded by all alliance nodes, and determines the average value as the aggregated fine-tuning model parameters.

11. The method according to any one of claims 1 to 8, characterized in that The central node aggregates the fine-tuning model parameters uploaded by all alliance nodes to obtain aggregated fine-tuning model parameters, including: The central node obtains the pre-stored weight coefficient of each alliance node; The central node calculates a weighted average value according to the weight coefficient of each alliance node and the fine-tuning model parameters, and determines the weighted average value as the aggregated fine-tuning model parameters.

12. A large model training device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 11 when executed by a processor.

Images