End side intelligent learning method and system based on Transform and block chain
By using a low-rank adapter and a forward gradient descent optimizer on edge devices, combined with a blockchain consensus mechanism, the computational complexity and parameter count issues of the Transformer model on resource-constrained devices are resolved. This achieves efficient and secure distributed learning, adapts to dynamic network environments, and improves the accuracy and robustness of model training.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-28
- Publication Date
- 2026-03-24
AI Technical Summary
Existing technologies for deploying Transformer models on edge devices suffer from high computational complexity, large number of model parameters, and limited resources. Furthermore, distributed collaborative learning lacks reliable assurance mechanisms, consumes a lot of energy, has slow model convergence, is difficult to cope with dynamic network environments, and has unreasonable node contribution evaluation.
By employing a low-rank adapter (LoRA) and a forward gradient descent optimizer to reduce the parameters of the Transformer model, and combining this with a blockchain consensus mechanism, efficient and accurate group learning is achieved through signature verification and gray model optimization of node contribution weights.
It enables efficient, secure, and accurate distributed model training on resource-constrained edge devices, adapts to dynamic network environments, and improves collaborative efficiency, accuracy, and robustness of model updates.
Smart Images
Figure CN121724104A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to edge-side intelligent learning methods and systems based on Transformer and blockchain, belonging to the field of group learning, and is particularly applicable to edge-side intelligent learning based on Transformer and blockchain. Background Technology
[0002] With the rapid development of edge computing and IoT technologies, collaborative learning among edge devices has become a research hotspot. While swarm learning, as a decentralized distributed learning paradigm, can achieve collaborative training among devices without relying on a central server, existing technical solutions still have significant shortcomings.
[0003] In terms of edge model training, traditional CNN and RNN models have limited ability to handle sequential data and non-local dependencies. Although the Transformer architecture has excellent performance, it is difficult to deploy directly to resource-constrained edge devices due to the high computational complexity of its self-attention mechanism and the large number of model parameters. Existing model compression methods often sacrifice performance and lack effective edge adaptation solutions.
[0004] In terms of distributed collaboration mechanisms, existing solutions have three main drawbacks: they lack reliable trust guarantee mechanisms and are vulnerable to malicious node attacks; traditional consensus algorithms are energy-intensive and unsuitable for resource-constrained edge scenarios; and simple parameter aggregation strategies lead to slow model convergence and are susceptible to abnormal node influences.
[0005] In terms of node contribution evaluation, existing methods are mostly based on static statistics of historical performance, lacking predictability and adaptability, and are difficult to cope with dynamically changing network environments. In particular, they cannot achieve reasonable weight allocation for newly joined nodes or nodes that participate intermittently.
[0006] In summary, current research largely focuses on optimizing single technologies, leaving a significant gap in the deep integration of advanced model architectures, distributed consensus, and intelligent evaluation mechanisms. The lack of effective integration among key technologies results in significant deficiencies in the practicality, efficiency, and security of existing systems, severely hindering the large-scale application of swarm learning technology in edge environments. Innovative solutions are urgently needed to comprehensively address these issues. Summary of the Invention
[0007] In view of this, the purpose of this invention is to provide an edge-side intelligent learning method and system based on Transformer and blockchain, solving problems such as security, collaborative efficiency, and traceability in existing technologies, and achieving efficient and accurate group learning under the blockchain consensus mechanism. Specifically, the parameters of the Transformer model of the verification nodes are reduced by using Low-Rank Adaptation (LoRA) technology and Forward Gradient Descent (FGD) optimizer to reduce storage space and communication overhead. At the same time, a grey model is introduced to intelligently optimize the weights of the verification nodes, improving the accuracy of model prediction.
[0008] For scenarios where N computer device nodes connect via a P2P protocol network for decentralized group learning, this invention provides an edge-side intelligent learning system based on Transformer and blockchain. The system is characterized by comprising a local training management module, a blockchain network management module, a smart contract and transaction management module, a consensus and global update module, and a security and privacy protection module.
[0009] The local training management module is responsible for local model training on the edge device and includes three sub-modules: the Transformer model adapter sub-module is used to load and configure the Transformer model and inject LoRA; the memory-optimized training sub-module implements memory-optimized training algorithms such as FGD; and the local data management sub-module manages the loading, preprocessing and privacy protection of the local dataset.
[0010] The blockchain network management module is responsible for node status monitoring, role allocation, and distributed ledger management. It includes three sub-modules: the node status monitoring sub-module monitors the resource status of its own node and exchanges status information with other nodes; the dynamic role allocation sub-module dynamically allocates node roles based on status information; and the distributed ledger sub-module maintains blockchain data and synchronizes ledger status.
[0011] The smart contract and transaction management module is responsible for processing model update transactions and includes three sub-modules: the model transaction generation sub-module encapsulates local model updates into transactions and signs them; the transaction verification sub-module verifies the validity of received transactions; and the transaction pool management sub-module maintains a valid transaction pool and arranges transactions in order.
[0012] The consensus and global update module is responsible for executing the consensus algorithm, analyzing node contributions, and updating the global model. It consists of three sub-modules: the improved consensus sub-module executes a lightweight consensus algorithm and competes for the right to package data; the model weight analysis sub-module predicts node contribution weights based on historical data; and the weighted aggregation sub-module aggregates model updates in transactions according to weights and updates the global model.
[0013] The security and privacy protection module is responsible for encryption and privacy protection, and includes two sub-modules: the encryption security sub-module manages key pairs and provides signing and verification functions; the differential privacy sub-module adds noise to the model update to achieve differential privacy.
[0014] The Transformer model adapter submodule is connected to the memory-optimized training submodule, which is connected to the local data management submodule and the differential privacy submodule, respectively. The local data management submodule is connected to the transaction generation submodule and the weighted aggregation submodule, respectively.
[0015] The node status monitoring submodule is connected to the dynamic role allocation submodule, the dynamic role allocation submodule is connected to the distributed ledger submodule, and the distributed ledger submodule is connected to the improved consensus submodule and the model weight analysis submodule, respectively.
[0016] The transaction generation submodule is connected to the transaction verification submodule and the encryption security submodule, respectively. The transaction verification submodule is connected to the transaction pool management submodule and the encryption security submodule, respectively. The transaction pool management submodule is connected to the improved consensus submodule.
[0017] The improved consensus submodule is connected to the model weight analysis submodule, and the model weight analysis submodule is connected to the weighted aggregation submodule.
[0018] Preferably, the model weight analysis submodule is implemented using a gray model.
[0019] An edge-side intelligent learning method based on Transformer and blockchain is characterized by comprising the following steps:
[0020] Step 1: Node Status Monitoring and Dynamic Role Allocation: Nodes in the network exchange and verify their status information with each other through a pre-defined status monitoring protocol; each node autonomously completes the dynamic allocation of the "computation" or "verification" role based on the received network-wide status information and the preset strategy; at the same time, each node performs blockchain data synchronization operations to ensure that all nodes in the network obtain a consistent initial blockchain state.
[0021] Step 2: Establishing Relationships and Initializing the Training Environment: Based on the role allocation results output in Step 1, each node assigns a corresponding verification node to each computing node through a node negotiation protocol, thereby establishing the relationship and temporary communication link between computing nodes and verification nodes; the computing nodes initialize their local training environment based on the preset local training parameters of the Transformer model.
[0022] Step 3: Local Model Training and Model Update Transaction Generation: The compute node uses the local dataset to train the Transformer model, updates the model parameters through the backpropagation algorithm, and calculates the difference between its local model parameters and the global model parameters as the parameter update amount; the compute node generates a model update transaction containing the parameter update amount, the node's digital signature, and local training metrics, encrypts the transaction, and outputs it to its associated verification node.
[0023] Step 4: Transaction Verification and Transaction Pool Maintenance: Verification nodes receive model update transactions from associated computing nodes, first verifying the validity of the transaction signature and the legitimacy of the sender's identity; they then broadcast verified transactions to other verification nodes in the network, while simultaneously receiving and collecting other verified broadcast transactions; each verification node sorts the collected valid transactions by reception time to form and maintain its local transaction pool.
[0024] Step 5: Consensus Execution and New Block Generation: Verifying nodes compete for the right to package new blocks based on the preset consensus mechanism and valid transactions in the current transaction pool. The validating node that wins the right to package the blocks selects valid transactions from the transaction pool, packages them according to the block capacity limit to generate a new block, and appends the new block to the blockchain to complete the on-chain evidence storage for this round of model update.
[0025] Step Six: Global Model Parameter Update: The validator nodes perform a weighted average of the parameter updates in the valid transactions of the new block and update the global model parameters.
[0026] Step 7: Training terminated: Repeat steps 1 through 6 until the global model update of the Transformer model is less than the set threshold, thus completing the training task.
[0027] Furthermore, the Transformer model described herein adds a low-rank adapter after each linear transformation of the attention layer in the standard Transformer encoder architecture.
[0028] Furthermore, the backpropagation algorithm described in step three is the forward gradient descent method.
[0029] Furthermore, compute node w uses the local dataset D w The parameter update amount during Transformer model training is: Δθ w =θ w -θ global The model updates the transaction: T w ={Δθ w signature w metrics w}, where the local training parameter is θ w The loss function is L(θ)w ), θ global For global model parameters, signature w For the digital signature of node w, metrics w The local training metric is composed of the loss function and accuracy.
[0030] Furthermore, step five specifically includes:
[0031] S501: Verification nodes compete for consensus on valid transactions in the current transaction pool based on a preset consensus mechanism, vying for the right to package the new block; assuming the current transaction pool has m model update transactions {T1, T2, ..., T...} m};
[0032] S502: The validator node that wins the right to package the transaction selects valid transactions from the transaction pool and packages them into a new block B according to the maximum block size. new ={blockheader,{T1, T2,…,T k The new block is then appended to the blockchain to complete the on-chain notarization of this round of model update; where k is the block capacity limit and blockheader is the block header.
[0033] Furthermore, the global model parameter θ mentioned in step six... global ←θ global +Δθ avg ,in ← indicates an assignment operation.
[0034] Preferably, the model updates the transaction: T w ={Δθ w , q(w), signature w metrics w}, among which, comprehensive quality indicators scalar norm of parameter update i =||Δθ w ||2,||·||2 represents the L2 norm, α+β+γ=1 represents the artificially set weighting coefficients, and the loss... w ,accurac(y w ) are metrics w The loss function and accuracy in the model.
[0035] Furthermore, the weighted average mentioned in step six involves predicting and correcting the weights of the computation node contributions using a grey model, specifically as follows:
[0036] S601: Arrange the comprehensive quality indicators corresponding to the new block in node order to construct the comprehensive quality indicator sequence Q. (0) =(q (0)(1), q (0) (2), ..., q (0) (k));
[0037] S602: Estimating parameters of grey differential equations using the least squares method in,
[0038] S603: Calculate the predicted value of the comprehensive quality index sequence of time response. in
[0039] S604: Calculate the weights based on the predicted values of the comprehensive quality index series.
[0040] S605: Update global model parameters θ global ←θ global +η·Δθ weight ;in,
[0041] The beneficial effects of this invention are as follows: by leveraging the decentralized architecture, signature verification, and immutability of blockchain, contributions are quantified and training traceability is ensured, promoting efficient and reliable training of distributed models in sensitive scenarios; dynamic role allocation and parameter configuration adapt to node state fluctuations and improve collaborative efficiency; flexible switching between dual consensus mechanisms is supported, balancing security and timeliness; and the parameters of the Transformer model are significantly reduced through low-rank adapter and forward gradient descent optimization techniques; combined with the gray model correction mechanism, the accuracy and robustness of global model updates are effectively improved, while adapting to dynamically changing network environments. Attached Figure Description
[0042] To illustrate the objectives and technical solutions of this invention, the following figures are provided:
[0043] Figure 1 This is an architecture diagram of the edge-side intelligent learning system based on Transformer and blockchain provided in Embodiment 1 of the present invention; wherein, the arrows indicate the direction of data flow, black indicates the internal connection relationship of the module, and orange indicates the connection relationship between modules;
[0044] Figure 2 A flowchart of the edge learning method based on Transformer and blockchain provided in Embodiment 2 of the present invention;
[0045] Figure 3 This is a diagram illustrating the specific implementation architecture of blockchain edge learning in Embodiment 2 of the present invention;
[0046] Figure 4A flowchart of the edge-side intelligent learning method based on Transformer and blockchain provided in Embodiment 3 of the present invention;
[0047] Figure 5 Experimental results of the edge-side intelligent learning method based on Transformer and blockchain provided in Embodiment 3 of this invention. Detailed Implementation
[0048] To make the technical solution, the purpose of the embodiments, and the advantages of the system architecture of the present invention clearer, the following will be described in conjunction with the appendix. Figure 1 The preferred embodiments of the present invention will be described in detail below.
[0049] Example 1: The MedMNIST dataset (https: / / medmnist.com / ) is a large-scale MNIST-like medical image dataset developed by researchers from Shanghai Jiao Tong University and other institutions. It contains 10 preprocessed open medical datasets, with data sources including different imaging modalities such as X-ray, OCT, ultrasound, and CT, covering multiple medical fields such as colorectal cancer, retinal diseases, breast diseases, and liver tumors. The images are standardized to 28×28 pixels for classification tasks, with data sizes ranging from 100 to 100,000. Task types include binary classification, multi-class classification, ordered regression, and multi-labeling. For scenarios where multiple computer device nodes connect via a P2P protocol network for decentralized group learning, to achieve accurate predictions for these tasks based on multi-party collaboration, it is necessary to train an efficient, accurate, and low-communication-overhead prediction model based on group learning. Therefore, this example provides an "edge-side intelligent learning system based on Transformer and blockchain".
[0050] Specifically, in combination Figure 1 The edge-side intelligent learning system based on Transformer and blockchain consists of a local training management module (1), a blockchain network management module (2), a smart contract and transaction management module (3), a consensus and global update module (4), a security and privacy protection module (5), and a network communication module (6).
[0051] The local training management module (1) is responsible for local model training on the edge device and includes three sub-modules: Transformer model adapter sub-module (11) for loading and configuring Transformer models and injecting LoRA; memory-optimized training sub-module (12) for implementing memory-optimized training algorithms such as FGD; and local data management sub-module (13) for managing the loading, preprocessing and privacy protection of local datasets.
[0052] The blockchain network management module (2) is responsible for node status monitoring, role allocation and distributed ledger management, and includes three sub-modules: node status monitoring sub-module (21) monitors the resource status of the node (computing, memory, network, power, etc.) and exchanges status information with other nodes; dynamic role allocation sub-module (22) dynamically allocates node roles (computing or verification) according to status information; and distributed ledger sub-module (23) maintains blockchain data and synchronizes ledger status.
[0053] Compute nodes are responsible for training the Transformer model using local data, generating model parameter updates, and creating transactions. Validator nodes are responsible for validating transactions, participating in consensus competition, packaging new blocks, and updating the global model through weighted aggregation based on analysis. Compute nodes focus on model training computation, while validator nodes focus on blockchain maintenance and consensus execution. Both achieve resource optimization and collaborative work through dynamic role allocation.
[0054] The smart contract and transaction management module (3) is responsible for processing model update transactions and includes three sub-modules: the model transaction generation sub-module (31) encapsulates the local model update into a transaction and signs it; the transaction verification sub-module (32) verifies the validity of the received transaction (signature, format, etc.); and the transaction pool management sub-module (33) maintains the valid transaction pool and arranges transactions in order.
[0055] The consensus and global update module (4) is responsible for executing the consensus algorithm, analyzing node contributions, and updating the global model. It includes three sub-modules: the improved consensus sub-module (41) executes the lightweight consensus algorithm and competes for the right to package; the model weight analysis sub-module (42) uses the model to predict the node contribution weight based on historical data; and the weighted aggregation sub-module (43) aggregates the model updates in the transactions according to the weights and updates the global model.
[0056] The security and privacy protection module (5) is responsible for encryption and privacy protection and includes two sub-modules: the encryption security sub-module (51) manages key pairs and provides signature and verification functions; the differential privacy sub-module (52) adds noise in model updates to achieve differential privacy.
[0057] The Transformer model adapter submodule (11) is connected to the memory-optimized training submodule (12), the memory-optimized training submodule (12) is connected to the local data management submodule (13) and the differential privacy submodule (52) respectively, and the local data management submodule (13) is connected to the transaction generation submodule (31) and the weighted aggregation submodule (43) respectively.
[0058] The node status monitoring submodule (21) is connected to the dynamic role allocation submodule (22), the dynamic role allocation submodule (22) is connected to the distributed ledger submodule (23), and the distributed ledger submodule (23) is connected to the improved consensus submodule (41) and the model weight analysis submodule (42) respectively.
[0059] The transaction generation submodule (31) is connected to the transaction verification submodule (32) and the encryption security submodule (51) respectively. The transaction verification submodule (32) is connected to the transaction pool management submodule (33) and the encryption security submodule (51) respectively. The transaction pool management submodule (33) is connected to the improved consensus submodule (41).
[0060] The improved consensus submodule (41) is connected to the model weight analysis submodule (42), and the model weight analysis submodule (42) is connected to the weighted aggregation submodule (43).
[0061] The network communication module (6) includes a network communication device with a P2P protocol, which is responsible for all network communication between nodes.
[0062] The specific functional descriptions of each module are as follows:
[0063] The local data management submodule (13) provides preprocessed data to the Transformer model adapter submodule (11); the Transformer model adapter submodule (11) provides the configured model to the memory-optimized training submodule (12); the memory-optimized training submodule (12) trains the model using local data and outputs the trained model parameters; the local training management module (1) obtains global model parameters from the weighted aggregation submodule (43) of the consensus and global update module (4); the local training management module (1) sends the trained model update (parameter difference) to the model transaction generation submodule (31) of the smart contract and transaction management module (3);
[0064] The node status monitoring submodule (21) provides status information to the dynamic role allocation submodule (22); the dynamic role allocation submodule (22) allocates roles according to the status information and notifies the distributed ledger submodule (23) of the role change of this node; the node status monitoring submodule (21) exchanges status information with other nodes through the network communication module (6); the dynamic role allocation submodule (22) notifies the smart contract and transaction management module (3) and the consensus and global update module (4) of the role allocation result so that they can perform corresponding operations according to the role;
[0065] The distributed ledger submodule (23) synchronizes the ledger with other nodes through the network communication module (6) and provides ledger query services to other modules; the model transaction generation submodule (31) sends the generated transactions to the transaction verification submodule (32) for self-verification, and then broadcasts them through the network communication module (6); the transaction verification submodule (32) sends the verified transactions to the transaction pool management submodule (33); the transaction pool management submodule (33) maintains the transaction pool and provides the transactions in the transaction pool to the improved consensus submodule (41) of the consensus and global update module (4); the model transaction generation submodule (31) obtains model updates from the local training management module (1); the transaction verification submodule (32) verifies the signature through the cryptographic security submodule (51) of the security and privacy protection module (5); the transaction pool management submodule (33) provides the transaction pool content to the consensus and global update module (4);
[0066] The improved consensus submodule (41) obtains transactions from the transaction pool management submodule (33), executes consensus, and generates a new block; the model weight analysis submodule (42) obtains historical transaction data from the distributed ledger submodule (23), calculates the node contribution weight, and provides it to the weighted aggregation submodule (43); the weighted aggregation submodule (43) updates the weighted aggregation model and generates a new global model; the improved consensus submodule (41) sends the new block to the distributed ledger submodule (23) of the blockchain network management module (2) for recording; the weighted aggregation submodule (43) sends the updated global model parameters to the local training management module (1) for the next round of training;
[0067] The security and privacy protection module (5) is responsible for encryption and privacy protection; it contains two sub-modules: the encryption security sub-module (51): manages key pairs and provides signature and verification functions; the differential privacy sub-module (52): adds noise in model updates to achieve differential privacy; the internal connection of the module: the two sub-modules are relatively independent and can be called by other modules respectively; the encryption security sub-module (51) provides signatures for the model transaction generation sub-module (31) of the smart contract and transaction management module (3) and provides verification for the transaction verification sub-module (32); the differential privacy sub-module (52) is called by the memory optimization training sub-module (12) of the local training management module (1) to add noise during training;
[0068] All modules that require network communication pass messages through the network communication module (6).
[0069] Example 2: In order to achieve accurate prediction of these tasks based on multi-party collaboration, based on the scenario and system architecture described in Example 1, it is necessary to train an efficient, accurate and low-communication-overhead prediction model based on group learning. To this end, this example provides an "edge-side learning method based on Transformer and blockchain".
[0070] Combination Figure 2 An edge-side learning method based on Transformer and blockchain is characterized by comprising the following steps:
[0071] Step 1: Node Status Monitoring and Dynamic Role Allocation: Nodes in the network exchange and verify their status information with each other through a pre-defined status monitoring protocol; each node autonomously completes the dynamic allocation of the "computation" or "verification" role based on the received network-wide status information and preset strategies; at the same time, each node performs blockchain data synchronization operations to ensure that all nodes in the network obtain a consistent initial blockchain state.
[0072] Step 2: Establishing Relationships and Initializing the Training Environment: Based on the role allocation results output in Step 1, each node assigns a corresponding verification node to each computing node through a node negotiation protocol, thereby establishing the relationship and temporary communication link between computing nodes and verification nodes; the computing nodes initialize their local training environment based on the preset local training parameters of the Transformer model.
[0073] The Transformer model described above adds a low-rank adapter after each linear transformation of the attention layer in the standard Transformer encoder architecture.
[0074] Step 3: Local Model Training and Model Update Transaction Generation: The compute node uses the local dataset to train the Transformer model, updates the model parameters through the backpropagation algorithm, and calculates the difference between its local model parameters and the global model parameters as the parameter update amount. The compute node generates a model update transaction containing the parameter update amount, the node's digital signature, and local training metrics, encrypts the transaction, and outputs it to its associated verification node.
[0075] The backpropagation algorithm described is the forward gradient descent method.
[0076] Any compute node w uses the local dataset D w The parameter update amount during Transformer model training is: Δθ w =θ w -θ global The model updates the transaction: T w ={Δθ w signature w metrics w}, where the local training parameter is θ w The loss function is L(θ) w ), θ global For global model parameters, signature wFor the digital signature of node w, metrics w The local training metric is composed of the loss function and accuracy.
[0077] Step 4: Transaction Verification and Transaction Pool Maintenance: Verification nodes receive model update transactions from associated computing nodes, first verifying the validity of the transaction signature and the legitimacy of the sender's identity; they then broadcast verified transactions to other verification nodes in the network, while simultaneously receiving and collecting other verified broadcast transactions; each verification node sorts the collected valid transactions by reception time to form and maintain its local transaction pool.
[0078] Step 5: Consensus Execution and New Block Generation: Validating nodes compete for the right to package a new block based on a preset consensus mechanism, selecting valid transactions from the current transaction pool. The validating node that wins the packaging right selects valid transactions from the transaction pool, packages them according to the block capacity limit, and generates a new block. This new block is then appended to the blockchain, completing the on-chain notarization of this round of model updates. Specifically:
[0079] S501: Verification nodes compete for consensus on valid transactions in the current transaction pool based on a preset consensus mechanism, vying for the right to package the new block; assuming the current transaction pool has m model update transactions {T1, T2, ..., T...} m};
[0080] S502: The validator node that wins the right to package the transaction selects valid transactions from the transaction pool and packages them into a new block B according to the maximum block size. new ={blockheader,{T1, T2,…,T k The new block is then appended to the blockchain to complete the on-chain notarization of this round of model update; where k is the block capacity limit and blockheader is the block header.
[0081] Step Six: Global Model Parameter Update: The validator nodes perform a weighted average of the parameter updates in the valid transactions of the new block and update the global model parameters.
[0082] The global model parameter θ global ←θ global +Δθ avg ,in ← indicates an assignment operation.
[0083] For a detailed implementation of edge learning in blockchain, please refer to the following: Figure 3 The process is shown below.
[0084] Step 7: Training terminated: Repeat steps 1 through 6 until the global model update of the Transformer model is less than the set threshold, thus completing the training task.
[0085] Example 3: In order to improve the accuracy and robustness of global model updates and adapt to dynamically changing network environments, this example provides an "edge-side intelligent learning method based on Transformer and blockchain" for the method described in Example 2.
[0086] The model weight analysis submodule (42) in the edge-side intelligent learning system based on Transformer and blockchain in Example 1 is implemented using a grey model.
[0087] Combination Figure 4 An edge-side intelligent learning method based on Transformer and blockchain is characterized by comprising the following steps:
[0088] Step 1: Node Status Monitoring and Dynamic Role Allocation: Nodes in the network exchange and verify their status information with each other through a pre-defined status monitoring protocol; each node autonomously completes the dynamic allocation of the "computation" or "verification" role based on the received network-wide status information and preset strategies; at the same time, each node performs blockchain data synchronization operations to ensure that all nodes in the network obtain a consistent initial blockchain state.
[0089] Step 2: Establishing Relationships and Initializing the Training Environment: Based on the role allocation results output in Step 1, each node assigns a corresponding verification node to each computing node through a node negotiation protocol, thereby establishing the relationship and temporary communication link between computing nodes and verification nodes; the computing nodes initialize their local training environment based on the preset local training parameters of the Transformer model.
[0090] The Transformer model described above adds a low-rank adapter after each linear transformation of the attention layer in the standard Transformer encoder architecture.
[0091] Step 3: Local Model Training and Model Update Transaction Generation: The compute node uses the local dataset to train the Transformer model, updates the model parameters through the backpropagation algorithm, and calculates the difference between its local model parameters and the global model parameters as the parameter update amount. The compute node generates a model update transaction containing the parameter update amount, the node's digital signature, and local training metrics, encrypts the transaction, and outputs it to its associated verification node.
[0092] The backpropagation algorithm described is the forward gradient descent method.
[0093] Any compute node w uses the local dataset D w The parameter update amount during Transformer model training is: Δθ w =θ w -θ globalThe model updates the transaction: T w ={Δθ w , q(w), signature w metrics w}, among which, comprehensive quality indicators scalar norm of parameter update i =||Δθ w ||2,||·||2 represents the L2 norm, α+β+γ=1 represents the artificially set weighting coefficients, and the loss... w ,accurac(y w ) are metrics w The loss function and accuracy are given, and the local training parameter is θ. w The loss function is L(θ) w ), θ global For global model parameters, signature w For the digital signature of node w, metrics w The local training metric is composed of the loss function and accuracy.
[0094] Step 4: Transaction Verification and Transaction Pool Maintenance: Verification nodes receive model update transactions from associated computing nodes, first verifying the validity of the transaction signature and the legitimacy of the sender's identity; they then broadcast verified transactions to other verification nodes in the network, while simultaneously receiving and collecting other verified broadcast transactions; each verification node sorts the collected valid transactions by reception time to form and maintain its local transaction pool.
[0095] Step 5: Consensus Execution and New Block Generation: Validating nodes compete for the right to package a new block based on a preset consensus mechanism, selecting valid transactions from the current transaction pool. The validating node that wins the packaging right selects valid transactions from the transaction pool, packages them according to the block capacity limit, and generates a new block. This new block is then appended to the blockchain, completing the on-chain notarization of this round of model updates. Specifically:
[0096] S501: Verification nodes compete for consensus on valid transactions in the current transaction pool based on a preset consensus mechanism, vying for the right to package the new block; assuming the current transaction pool has m model update transactions {T1, T2, ..., T...} m};
[0097] S502: The validator node that wins the right to package the transaction selects valid transactions from the transaction pool and packages them into a new block B according to the maximum block size. new ={blockheader,{T1, T2,…,T k The new block is then appended to the blockchain to complete the on-chain notarization of this round of model update; where k is the block capacity limit and blockheader is the block header.
[0098] Step Six: Global Model Parameter Update: The validator nodes perform a weighted average of the parameter updates in the valid transactions of the new block and update the global model parameters.
[0099] The weighted average is calculated by using a grey model to predict and correct the weights of the computation node contributions, specifically:
[0100] S601: Arrange the comprehensive quality indicators corresponding to the new block in node order to construct the comprehensive quality indicator sequence Q. (0) =(q (0) (1), q (0) (2), ..., q (0) (k));
[0101] S602: Estimating parameters of grey differential equations using the least squares method in,
[0102] S603: Calculate the predicted value of the comprehensive quality index sequence of time response. in
[0103] S604: Calculate the weights based on the predicted values of the comprehensive quality index series.
[0104] S605: Update global model parameters θ global ←θ global +η·Δθ weight ;in,
[0105] Step 7: Training terminated: Repeat steps 1 through 6 until the global model update of the Transformer model is less than the set threshold, thus completing the training task.
[0106] To better demonstrate the beneficial effects of this invention, the method of this embodiment was implemented on GCC and NTL software. Each computing node only has a local sharded dataset. The global model is aggregated through blockchain consensus (PoW / PoS), and the global training of the Transformer model is completed through 10 rounds of communication training. The model is then validated using the global dataset, resulting in the following... Figure 5 The results are shown.
[0107] As can be seen, the global training model of the present invention gradually increases in testing accuracy as the number of nodes, i.e., the dataset, increases. When there are more than 6 nodes, i.e., after 6 rounds of training, the accuracy is significantly improved, eventually approaching 90%, which is much higher than the accuracy of the training model with a single node (i.e., the accuracy of the model after 1 round of training). At the same time, the blockchain consensus mechanism effectively filters malicious updates, and the system has strong stability.
[0108] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.
Claims
1. An edge-side intelligent learning system based on Transformer and blockchain, characterized in that, It consists of a local training management module (1), a blockchain network management module (2), a smart contract and transaction management module (3), a consensus and global update module (4), and a security and privacy protection module (5); The local training management module (1) is responsible for local model training on the edge device, and includes a Transformer model adapter submodule (11), a memory-optimized training submodule (12), and a local data management submodule (13). The blockchain network management module (2) is responsible for node status monitoring, role allocation and distributed ledger management, including node status monitoring submodule (21), dynamic role allocation submodule (22) and distributed ledger submodule (23); The smart contract and transaction management module (3) is responsible for processing model update transactions, including a model transaction generation submodule (31), a transaction verification submodule (32), and a transaction pool management submodule (33); The consensus and global update module (4) is responsible for executing the consensus algorithm, analyzing node contributions and updating the global model, and includes an improved consensus submodule (41), a model weight analysis submodule (42), and a weighted aggregation submodule (43). The security and privacy protection module (5) is responsible for encryption and privacy protection, including the encryption security submodule (51) which manages key pairs and the differential privacy submodule (52); The Transformer model adapter submodule (11) is connected to the memory-optimized training submodule (12), the memory-optimized training submodule (12) is connected to the local data management submodule (13) and the differential privacy submodule (52) respectively, and the local data management submodule (13) is connected to the transaction generation submodule (31) and the weighted aggregation submodule (43) respectively. The node status monitoring submodule (21) is connected to the dynamic role allocation submodule (22), the dynamic role allocation submodule (22) is connected to the distributed ledger submodule (23), and the distributed ledger submodule (23) is connected to the improved consensus submodule (41) and the model weight analysis submodule (42) respectively. The transaction generation submodule (31) is connected to the transaction verification submodule (32) and the encryption security submodule (51) respectively. The transaction verification submodule (32) is connected to the transaction pool management submodule (33) and the encryption security submodule (51) respectively. The transaction pool management submodule (33) is connected to the improved consensus submodule (41). The improved consensus submodule (41) is connected to the model weight analysis submodule (42), and the model weight analysis submodule (42) is connected to the weighted aggregation submodule (43).
2. The edge-side intelligent learning system based on Transformer and blockchain according to claim 1, characterized in that, The model weight analysis submodule (42) is implemented using a grey model.
3. An edge-side intelligent learning method based on Transformer and blockchain, characterized in that; Includes the following steps: Step 1: Node Status Monitoring and Dynamic Role Assignment: Nodes in the network exchange and verify their status information with each other through a pre-defined status monitoring protocol; Each node autonomously completes the dynamic allocation of "computation" or "verification" roles based on the received network status information and preset strategies; at the same time, each node performs blockchain data synchronization operations to ensure that all nodes in the network obtain a consistent initial blockchain state. Step 2: Establishing Relationships and Initializing the Training Environment: Based on the role allocation results output in Step 1, each node assigns a corresponding verification node to each computing node through an inter-node negotiation protocol, thereby establishing the relationship and temporary communication link between computing nodes and verification nodes. The compute nodes initialize their local training environment based on the preset local training parameters of the Transformer model; Step 3: Local Model Training and Model Update Transaction Generation: The compute node uses the local dataset to train the Transformer model, updates the model parameters through the backpropagation algorithm, and calculates the difference between its local model parameters and the global model parameters as the parameter update amount; the compute node generates a model update transaction containing the parameter update amount, the node's digital signature, and local training metrics, encrypts the transaction, and outputs it to its associated verification node. Step 4: Transaction Verification and Transaction Pool Maintenance: Verification nodes receive model update transactions from associated computing nodes, first verifying the validity of the transaction signature and the legitimacy of the sender's identity; they then broadcast verified transactions to other verification nodes in the network, while simultaneously receiving and collecting other verified broadcast transactions; each verification node sorts the collected valid transactions by reception time to form and maintain its local transaction pool. Step 5: Consensus Execution and New Block Generation: Verifying nodes compete for the right to package new blocks based on the preset consensus mechanism and valid transactions in the current transaction pool. The validating node that wins the right to package the blocks selects valid transactions from the transaction pool, packages them according to the block capacity limit to generate a new block, and appends the new block to the blockchain to complete the on-chain evidence storage for this round of model update. Step Six: Global Model Parameter Update: The validator nodes perform a weighted average of the parameter updates in the valid transactions of the new block and update the global model parameters. Step 7: Training terminated: Repeat steps 1 through 6 until the global model update of the Transformer model is less than the set threshold, thus completing the training task.
4. The edge-side intelligent learning method based on Transformer and blockchain according to claim 3, characterized in that, The Transformer model described above adds a low-rank adapter after each linear transformation of the attention layer in the standard Transformer encoder architecture.
5. The edge-side intelligent learning method based on Transformer and blockchain according to claim 3, characterized in that, The backpropagation algorithm described in step three is the forward gradient descent method.
6. The edge-side intelligent learning method based on Transformer and blockchain according to claim 3, characterized in that, Compute node w uses local dataset D w The parameter update amount during Transformer model training is: Δθ w =θ w -θ global The model updates the transaction: T w ={Δθ w signature w metrics w }, where the local training parameter is θ w The loss function is L(θ) w ), θ global For global model parameters, signature w For the digital signature of node w, metrics w The local training metric is composed of the loss function and accuracy.
7. The edge-side intelligent learning method based on Transformer and blockchain according to claim 3, characterized in that, Step five specifically refers to: S501: Verification nodes compete for consensus on valid transactions in the current transaction pool based on a preset consensus mechanism, vying for the right to package the new block; assuming the current transaction pool has m model update transactions {T1, T2, ..., T...} m }; S502: The validator node that wins the right to package the transaction selects valid transactions from the transaction pool and packages them into a new block B according to the maximum block size. new ={blockheader,{T1,T2,…,T k The new block is then appended to the blockchain to complete the on-chain notarization of this round of model update; where k is the block capacity limit and blockheader is the block header.
8. The edge-side intelligent learning method based on Transformer and blockchain according to claim 6, characterized in that, The global model parameter θ mentioned in step six global ←θ global +Δθ avg ,in ← indicates an assignment operation.
9. The edge-side intelligent learning method based on Transformer and blockchain according to claim 3, characterized in that, Any compute node w uses the local dataset D w The parameter update amount during Transformer model training is: Δθ w =θ w -θ global The model updates the transaction: T w ={Δθ w , q(w), signature w metrics w }, among which, comprehensive quality indicators scalar norm of parameter update i =||Δθ w ||2,||·||2 represents the L2 norm, α+β+γ=1 represents the artificially set weighting coefficients, and the loss... w ,accurac(y w ) are metrics w The loss function and accuracy are given, and the local training parameter is θ. w The loss function is L(θ) w ), θ global For global model parameters, signature w For the digital signature of node w, metrics w The local training metric is composed of the loss function and accuracy.
10. The edge-side intelligent learning method based on Transformer and blockchain according to claim 9, characterized in that, The weighted average mentioned in step six involves predicting and correcting the weights of the computation node contributions using a grey model, specifically as follows: S601: Arrange the comprehensive quality indicators corresponding to the new block in node order to construct a comprehensive quality indicator sequence. Q (0) =(q (0) (1),q (0) (2),…,q (0) (k)); S602: Estimating parameters of grey differential equations using the least squares method in, S603: Calculate the predicted value of the comprehensive quality index sequence of time response. in S604: Calculate the weights based on the predicted values of the comprehensive quality index series. S605: Update global model parameters θ global ←θ global +η·Δθ weight ;in,