Model training method and device based on block chain, equipment and medium

Abstract

The embodiment of the invention provides a block chain-based model training method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring training data comprising a plurality of data fragments; each data fragment corresponds to a block on a block chain; based on the block chain, obtaining a target number of data fragments from the training data; distributing the target number of data fragments to a plurality of processing nodes; each processing node trains a to-be-trained model based on the distributed data fragments to obtain a corresponding training sub-result; aggregating the training sub-results corresponding to the processing nodes to obtain an aggregation result; and adjusting parameters of the to-be-trained model based on the aggregation result to obtain a target training model. According to the method, a block chain mechanism is utilized, a credible basis is provided for decentralized model training, and the safety and tamper resistance of the training process are effectively improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of blockchain, and in particular, to a model training method and device based on blockchain, an electronic device, a storage medium and a computer program product. BACKGROUND

[0002] In a cross-organization or cross-chain decentralized AI collaborative training scenario, multiple participants (such as research institutions, enterprises or individuals) are usually unwilling and unable to store original data centrally due to data privacy, compliance or ownership issues. In the training of a decentralized cross-organization large model, traditional data transmission methods rely on centralized servers or network-wide broadcasting, which not only makes it difficult to verify whether each node has truly and completely obtained the specified training slice, but also is prone to security risks such as "fake pulling" or data loss in a high-concurrency network environment. In addition, the bandwidth, online status and storage capacity of each participating node fluctuate continuously, and if a static sharding or centralized scheduling strategy is used, some nodes are likely to become bottlenecks of the system, causing resource waste and making it difficult to achieve efficient and reliable distribution of massive training data in a complex multi-chain and multi-organization environment. Therefore, there is an urgent need in the art for a model training method that meets the needs of a decentralized model training scenario. SUMMARY

[0003] Embodiments of the present disclosure provide a model training method and device based on blockchain, an electronic device, a storage medium and a computer program product.

[0004] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0005] According to one aspect of the present disclosure, a model training method based on blockchain is provided, the method comprising: obtaining training data comprising a plurality of data slices; each data slice corresponding to a block on a blockchain; based on the blockchain, obtaining a target number of data slices from the training data; assigning the target number of data slices to a plurality of processing nodes; each processing node training a to-be-trained model based on the assigned data slices to obtain a corresponding training sub-result; aggregating the training sub-result corresponding to each processing node to obtain an aggregated result; adjusting the parameters of the to-be-trained model based on the aggregated result to obtain a target training model.

[0006] In an exemplary embodiment, the obtaining of the training data comprising a plurality of data slices comprises: dividing the training data into a plurality of data slices; performing a hash calculation on each data slice to obtain a corresponding hash value; based on the hash value, registering on the blockchain to determine the block corresponding to the data slice.

[0007] In an example embodiment, the block stores at least identification information and a source address of the corresponding data shard; the identification information is used to identify the corresponding data shard; and the source address is used to represent a data storage address of the corresponding data shard.

[0008] In an example embodiment, the obtaining, based on the block chain, the target number of data shards from the training data includes: obtaining identification information of a previous block on the block chain; generating a selection random value based on the identification information of the previous block through a random function; determining the target number of shard indexes based on the selection random value; and obtaining the target number of data shards from the training data according to the target number of shard indexes.

[0009] In an example embodiment, the obtaining, according to the target number of shard indexes, the target number of data shards from the training data includes: refreshing index information of each data shard in the training data; and obtaining the target number of data shards from the training data according to the target number of shard indexes.

[0010] In an example embodiment, the allocating the target number of data shards to the plurality of processing nodes includes: determining a priority of each processing node according to attribute information of each processing node; and obtaining an allocated data shard from the target number of data shards according to the priority of each processing node.

[0011] In an example embodiment, the attribute information includes at least one of reward and punishment information, bandwidth information, state information, and processing capability information.

[0012] In an example embodiment, the training, by each processing node, a to-be-trained model based on an allocated data shard to obtain a corresponding training sub-result includes: obtaining first identification information and a first source address corresponding to a first data shard based on a first block corresponding to the allocated data shard; obtaining the first data shard according to the first source address; and training the to-be-trained model based on the first data shard to obtain a first training sub-result.

[0013] In an example embodiment, the training, by each processing node, a to-be-trained model based on an allocated data shard to obtain a corresponding training sub-result further includes: performing a hash calculation on the first data shard to obtain first verification information; verifying the first verification information against the first identification information; and in response to a verification pass, training the to-be-trained model based on the first data shard.

[0014] In the example embodiment, the aggregating the training sub-results corresponding to each of the processing nodes to obtain an aggregated result comprises: receiving the training sub-results uploaded by each of the processing nodes; the training sub-results are subjected to signature processing; performing signature verification on the training sub-results of each of the processing nodes by using pre-stored signature verification information; in response to the signature verification of the training sub-results of each of the processing nodes being passed, aggregating each of the training sub-results to obtain the aggregated result.

[0015] In the example embodiment, the aggregating the training sub-results corresponding to each of the processing nodes to obtain an aggregated result further comprises: receiving the training sub-results uploaded by each of the processing nodes; the training sub-results are subjected to signature processing; obtaining a corresponding signature segment according to each of the training sub-results; in response to obtaining a preset number of signature segments, aggregating the preset number of signature segments to obtain an aggregated signature; performing signature verification on the aggregated signature by using pre-stored signature verification information; in response to the signature verification of the aggregated signature being passed, aggregating each of the training sub-results to obtain the aggregated result.

[0016] In the example embodiment, the adjusting the parameters of the to-be-trained model based on the aggregated result to obtain a target training model comprises: receiving the training sub-results uploaded by each of the processing nodes; calculating the average gradient of each of the training sub-results to obtain the aggregated result; adjusting the parameters of the to-be-trained model based on the aggregated result; in response to the parameters of the to-be-trained model satisfying a preset convergence condition, obtaining the target training model.

[0017] In the example embodiment, the method further comprises: performing quality evaluation on the training sub-results uploaded by each of the processing nodes to obtain a quality evaluation result; determining the reward and punishment information corresponding to each of the processing nodes according to the quality evaluation result corresponding to each of the processing nodes.

[0018] According to a further aspect of the present disclosure, a blockchain-based model training apparatus is provided, comprising: a data shard on-chain module configured to obtain training data comprising a plurality of data shards, each of the data shards corresponding to a block on a blockchain; a data shard obtaining module configured to obtain a target number of the data shards from the training data based on the blockchain; a data shard distribution module configured to distribute the target number of data shards to a plurality of processing nodes; a processing node training module configured to train, by each of the processing nodes, a to-be-trained model based on the distributed data shards to obtain a corresponding training sub-result; a result aggregation module configured to aggregate the training sub-results corresponding to each of the processing nodes to obtain an aggregated result; and a parameter adjustment module configured to adjust parameters of the to-be-trained model based on the aggregated result to obtain a target training model.

[0019] According to a further aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; and a storage apparatus configured to store one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the blockchain-based model training method as described in the above embodiments.

[0020] According to a further aspect of the present disclosure, a computer-readable storage medium is provided, which stores a computer program, which, when executed by a processor, implements the blockchain-based model training method as described in the above embodiments.

[0021] According to a further aspect of the present disclosure, a computer program product is provided, comprising a computer program / signaling, which, when executed by a processor, implements the blockchain-based model training method as described in the above embodiments.

[0022] The blockchain-based model training method provided by the embodiments of the present disclosure comprises: obtaining training data comprising a plurality of data shards, each of the data shards corresponding to a block on a blockchain; obtaining a target number of the data shards from the training data based on the blockchain; distributing the target number of data shards to a plurality of processing nodes; training, by each of the processing nodes, a to-be-trained model based on the distributed data shards to obtain a corresponding training sub-result; aggregating the training sub-results corresponding to each of the processing nodes to obtain an aggregated result; and adjusting parameters of the to-be-trained model based on the aggregated result to obtain a target training model. The method utilizes the blockchain mechanism to provide a credible basis for decentralized model training, and effectively improves the security and tamper resistance of the training process.

[0023] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings, which are incorporated in and form a part of the specification, illustrate one or more embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure. The drawings are only for the purpose of illustrating one or more embodiments of the disclosure and are not to be construed as limiting the disclosure thereto.

[0025] Figure 1 A schematic diagram illustrating an exemplary system architecture of an embodiment of the present disclosure is shown; Figure 2 A flowchart illustrating a blockchain-based model training method of an embodiment of the present disclosure is shown; Figure 3 A flowchart illustrating a data sharding and chaining method of an embodiment of the present disclosure is shown; Figure 4 A flowchart illustrating a data sharding obtaining method of an embodiment of the present disclosure is shown; Figure 5 A flowchart illustrating a data sharding distribution method of an embodiment of the present disclosure is shown; Figure 6 A flowchart illustrating a processing node training method of an embodiment of the present disclosure is shown; Figure 7 A flowchart illustrating a result aggregation method of an embodiment of the present disclosure is shown Figure 1 ; Figure 8 A flowchart illustrating a result aggregation method of an embodiment of the present disclosure is shown Figure 2 ; Figure 9 A flowchart illustrating a parameter adjustment method of an embodiment of the present disclosure is shown; Figure 10 A structural schematic diagram of a blockchain-based model training apparatus of an embodiment of the present disclosure is shown; Figure 11 A structural schematic diagram of an electronic device suitable for implementing an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0026] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the implementations set forth herein; rather, these implementations are provided as non-limiting examples so that this disclosure will fully convey its scope to those skilled in the art. Features described in the description, examples, or claims can be combined in any suitable manner in one or more embodiments.

[0027] Further, the accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification. The drawings are not necessarily to scale, the dimensions of certain features being exaggerated from others for illustrative purposes. Same reference numerals in different drawings represent same or similar elements unless otherwise specified.

[0028] It should be noted that the ordinal numbers such as "first", "second", etc. mentioned in the embodiments of the present disclosure are used only to distinguish different objects, and do not represent the order, time sequence, priority or importance of the objects, and the description of "first", "second" does not necessarily mean that the objects are different.

[0029] Figure 1 A schematic diagram of an exemplary system architecture of the embodiments of the present disclosure is shown.

[0030] As Figure 1 shown, the system architecture can include a server 101, a network 102, a terminal device 103, a terminal device 104 and a terminal device 105. The network 102 is a medium for providing a communication link between the terminal device 103, the terminal device 104 or the terminal device 105 and the server 101. The network 102 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0031] The server 101 can be a server that provides various services, such as a background management server that provides support for the operation of the user using the terminal device 103, the terminal device 104 or the terminal device 105. The background management server can analyze and process the received request data, etc., and feed back the processing result to the terminal device 103, the terminal device 104 or the terminal device 105.

[0032] The terminal device 103, the terminal device 104 and the terminal device 105 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a wearable smart device, a virtual reality device, an augmented reality device, etc., but are not limited thereto.

[0033] It should be understood that Figure 1 the number of the terminal device 103, the terminal device 104, the terminal device 105, the network 102 and the server 101 in the above description is only illustrative, the server 101 can be a server of one entity, and can also be a server cluster composed of multiple servers, and can also be a cloud server. According to actual needs, there can be any number of terminal devices, networks and servers.

[0034] The steps of the method in the example embodiments of the present disclosure will be described in more detail below in conjunction with the accompanying drawings and examples.

[0035] Figure 2 A flowchart of a model training method based on a blockchain is shown. As shown in the flowchart, the model training method based on a blockchain can include the following steps. Figure 2

[0036] In step S210, training data including a plurality of data shards is obtained; each of the data shards corresponds to a block on a blockchain.

[0037] In the example embodiments of the present disclosure, in order to meet the needs of a decentralized model training scenario, the training data obtained by each participant is preprocessed. Instead of uploading raw data collectively, each participant divides their own training data into a plurality of data shards locally. The data shards are a plurality of training data units divided according to a fixed or variable size. The training data can be raw data collected, model weight slices, vector database blocks, or intermediate caches.

[0038] In the example embodiments of the present disclosure, in order to ensure that the uniqueness and integrity of these data shards can be verified in subsequent processes, the blockchain technology is used to ensure traceability and verification. Each participant needs to submit the data shards obtained after preprocessing to the chain for registration. The registered data shards correspond to a certain block on the blockchain respectively. The data shards uploaded by each participant are effectively integrated into a globally identifiable training data set through a unified blockchain, which not only ensures that the data is safely included in the model training process without leaving the local, but also provides a basis for subsequent random selection, integrity verification, and source tracing.

[0039] In step S220, a target number of data shards are obtained from the training data based on the blockchain.

[0040] In the example embodiments of the present disclosure, at the beginning of each round of training, the smart contract based on the blockchain obtains a target number of data shards on the blockchain for model training by each processing node. This process selects a target number of data shards from the training data pool through a randomization process on the blockchain, which is fair and secure. This process does not require the introduction of a centralized scheduler or a trusted third party, which meets the trust requirements of decentralized training.

[0041] ​In an example embodiment, the blockchain-based smart contract obtains a hash value of a previous block as a random seed. The previous block can be the last block obtained in the last round of training. The hash value is obtained by hashing the data shard corresponding to the block. The hash value has the characteristics of global consensus, openness and transparency, and cannot be predicted in advance, which is the basis for the credibility of the blockchain. With the hash value as the initial input, a verifiable random function (VRF) is called to generate an output value with randomness and verifiability, and then a set of data shard indexes to be processed is derived based on the output value. This design ensures that the random number generation process cannot be manipulated by any single node (including the contract itself), and the result can be verified by all participants afterwards, thereby laying the foundation for fair slicing.

[0042] In step S230, the target number of data shards are allocated to the plurality of processing nodes.

[0043] In the embodiments of the present disclosure, the model training process is synchronized by a plurality of decentralized processing nodes. Therefore, the target number of data shards selected in the foregoing steps need to be randomly allocated to the plurality of processing nodes for model training, while ensuring the randomness and fairness of the allocation process.

[0044] In an example embodiment, the blockchain-based smart contract obtains attribute information of the plurality of processing nodes. The attribute information can be any information about the processing nodes, such as at least one of reward and punishment information, bandwidth information, state information, and processing capability information. Through a preset weighting algorithm, the contract calculates a comprehensive priority score for each node, thereby constructing a dynamic task queue reflecting the instant reliability and processing capability of the nodes. Based on the calculated priority order, the smart contract allocates the target number of data shards to the plurality of processing nodes. This dynamic scheduling strategy based on multi-dimensional attributes achieves a balance between the fairness and efficiency of task allocation. It should be noted that the selection of attribute information of the processing nodes and the design of the weighting algorithm can be adjusted according to actual needs, and the present disclosure is not limited thereto.

[0045] In step S240, each of the processing nodes trains the to-be-trained model based on the allocated data shard to obtain a corresponding training sub-result.

[0046] In the embodiments of the present disclosure, when each processing node receives the task allocated by the smart contract, it obtains the allocated data shard from the blockchain. Each processing node trains the to-be-trained model based on the obtained data shard. The training process of the related model on the processing node is known in the prior art, and will not be described here. Each processing node independently trains the model locally to obtain a corresponding training sub-result. The training sub-result can be a gradient or an updated parameter generated by training calculation.

[0047] In an exemplary embodiment, each processing node does not receive raw data directly from the contract. Instead, it pulls allocated data shards from a blockchain-based storage network (such as IPFS, Arweave, or designated private storage) based on the data shard source addresses recorded on the blockchain. Simultaneously, while pulling data shards, the processing nodes can also obtain and verify the credibility of the training data based on the security mechanisms provided by the blockchain itself. For example, the processing nodes use pre-registered hash values ​​(such as Merkle roots) on the blockchain to verify the data shard. This process ensures the integrity and authenticity of the downloaded data; only shards that fully pass verification are recognized as valid input by the processing nodes.

[0048] In step S250, the training sub-results corresponding to each processing node are aggregated to obtain an aggregated result.

[0049] In this embodiment, each processing node independently trains its model locally based on its allocated data shards, obtaining corresponding training sub-results. Therefore, it is necessary to aggregate the training sub-results corresponding to each processing node to obtain an aggregated result. This aggregated result reflects the sum of training results of each processing node during this round of training. It should be noted that there are many ways to aggregate the training sub-results, and this disclosure does not specifically limit the aggregation method.

[0050] In an exemplary embodiment, the training sub-results obtained from training each processing node are the calculated gradient parameters. The aggregated result is obtained by calculating the average gradient of each training sub-result.

[0051] In step S260, the parameters of the model to be trained are adjusted based on the aggregation result to obtain the target training model.

[0052] In this embodiment of the disclosure, the parameters of the model to be trained are updated and adjusted based on the aggregation results obtained above to achieve one round of training of the model to be trained. It is then checked whether the parameters of the model to be trained after parameter adjustment meet a preset convergence condition. If the parameters of the model to be trained meet the preset convergence condition, then this model is used as the target training model. If the parameters of the model to be trained do not meet the preset convergence condition, then the next round of model training is performed based on this model.

[0053] The model training method based on a blockchain provided by the embodiments of the present disclosure comprises the following steps: obtaining training data comprising a plurality of data shards; each data shard corresponds to a block on a blockchain; obtaining a target number of data shards from the training data based on the blockchain; assigning the target number of data shards to a plurality of processing nodes; each processing node trains a to-be-trained model based on the assigned data shard to obtain a corresponding training sub-result; aggregating the training sub-result corresponding to each processing node to obtain an aggregation result; and adjusting the parameters of the to-be-trained model based on the aggregation result to obtain a target training model. The method uses a blockchain mechanism to provide a credible basis for decentralized model training, effectively improving the security and tamper resistance of the training process.

[0054] Figure 3 A flowchart of the data shard chaining method of the embodiments of the present disclosure is shown. In the embodiments of the present disclosure, in the Figure 2 Based on the model training method based on a blockchain shown in the above, the step S210 can comprise the following steps.

[0055] In step S310, the training data is divided into a plurality of data shards.

[0056] In the embodiments of the present disclosure, the core task is to structurally segment the collected training data, laying the foundation for decentralized parallel processing. Each participating organization or node performs this operation locally, and they do not need to upload the original data set, but rather divide the local training data into several data shards of moderate size that are easy to process independently. Each data shard is considered as an independent training sample set. By this preprocessing, the massive data is decomposed into smaller task units, thereby providing a data foundation for distributed model training. Not only is the dispersibility and privacy of the original data protected, but also subsequent fine-grained task scheduling, fault tolerance processing, and flexible node participation are enabled.

[0057] In step S320, a hash value corresponding to each data shard is calculated.

[0058] In the embodiments of the present disclosure, the system generates a unique and verifiable hash value for each data shard to ensure its integrity throughout its life cycle. By performing hash calculation on each data shard, a corresponding hash value can be obtained.

[0059] Specifically, a Merkle tree algorithm can be used for the calculation. Each data shard is used as a leaf node of the Merkle tree, and a fixed, short-length root hash value is finally generated for the shard through recursive hash operations. The calculated Merkle root hash value serves as the unique representation of the data shard in cryptography. It has two key features: one is collision resistance, that is, it is almost impossible to find two different shards that produce the same hash value, ensuring the uniqueness of the representation; the other is anonymity, that is, the original data content cannot be inferred from the hash value itself, protecting data privacy.

[0060] In an example embodiment, the hash calculation of the data shard can be performed by the following formula:

[0061] wherein, is the Merkle tree calculation; is the data shard; is the Merkle root hash value.

[0062] In step S330, registration is performed on the blockchain based on the hash value, and the block corresponding to the data shard is determined.

[0063] In the embodiments of the present disclosure, the hash value corresponding to the calculated data shard is used to register the data shard on the blockchain. The smart contract receives a transaction containing the unique identifier (hash value) of the data shard, the storage source address, and the Merkle root hash. After the transaction is agreed by the blockchain network, it is packaged and recorded in a newly generated block. After registration, all key information of the data shard is fixed on the blockchain: the unique identifier is used for global retrieval, the source address indicates the specific location in the off-chain storage network such as IPFS, and the Merkle root hash serves as the verification of data integrity. Through the above registration process, the block establishes a corresponding relationship with the data shard. By registering each data shard on the blockchain, the data shard becomes trusted, traceable, and verifiable data.

[0064] In an example embodiment, at least the identification information and the source address of the corresponding data shard are stored in the block; the identification information is used to identify the corresponding data shard; and the source address is used to represent the data storage address of the corresponding data shard.

[0065] Figure 4 A flowchart of a data shard acquisition method according to an embodiment of the present disclosure is shown. In the embodiments of the present disclosure, based on the blockchain-based model training method shown in Figure 2 The step S220 can include the following steps based on the blockchain-based model training method shown in

[0066] In step S410, identification information of a previous block on the blockchain is obtained.

[0067] In the embodiments of the present disclosure, at the beginning of each round of training, the smart contract based on the blockchain obtains the identification information of the previous block on the blockchain. The identification information usually refers to the hash value of the data shard stored in the block. The previous block can be the last block obtained in the last round of training.

[0068] In the embodiments of the present disclosure, the identification information of the previous block is used as a random seed. On the one hand, the identification information is the result of global consensus in the network, and has the characteristics of openness, transparency and global consistency; on the other hand, the identification information has the time sequence and continuity of the generation of the blockchain block.

[0069] In step S420, a shard selection random value is generated by a random function based on the identification information of the previous block.

[0070] In the embodiments of the present disclosure, the identification information of the previous block is used as a random seed. The smart contract will input the identification information of the previous block to call a verifiable random function (VRF) to generate a shard selection random value. VRF is a random number function that can not only generate a highly random output value, but also the generated random value can be verified by calculation.

[0071] In an exemplary embodiment, the shard selection random value can be generated by the following formula:

[0072] wherein, VRF is a verifiable random function; is a shard selection random value; is the identification information of the previous block.

[0073] In step S430, the target number of shard indexes are determined based on the shard selection random value.

[0074] In the embodiments of the present disclosure, the system converts the generated shard selection random value into the shard indexes of the specific data shards to be processed in the current round. Since the shard selection random value of each round depends on the latest block hash and is completely different, the generated set of shard indexes will be completely refreshed in each round of training. This mechanism effectively prevents malicious nodes from performing attack behaviors such as long-term data pollution or early preparation of false calculation results on specific shards.

[0075] Specifically, the algorithm takes the shard selection random value and the total number of data shards in the work pool as input. Usually, this will map the random value to the index range of [1, M] through a series of modulo operations or other pseudo-random mapping functions, so as to pseudo-randomly select k (target number) non-repeating shard indexes.

[0076] In an example embodiment, the shard index can be determined by the following formula:

[0077] wherein, is a verifiable random function; is a shard random value.

[0078] In step S440, the target number of data shards are obtained from the training data according to the target number of shard indexes.

[0079] In the embodiments of the present disclosure, the smart contract or the dispatcher parses and locates the corresponding block on the blockchain according to the target number of shard indexes determined above. Then, the corresponding data shard is obtained based on the correspondence between the block and the data shard. In this way, the system realizes the decoupling of data index and data itself, avoids the blockchain becoming a bottleneck for data distribution, while ensuring the traceability and credibility of data.

[0080] Specifically, the source address corresponding to the data shard is obtained according to the block. Based on the source address, the data content of the corresponding data shard is pulled from the storage network (such as IPFS, Arweave or designated private storage) on the blockchain.

[0081] In an example embodiment, the step S440 can further include the following steps.

[0082] Refreshing the index information of each data shard in the training data; Obtaining the target number of data shards from the training data according to the target number of shard indexes.

[0083] In the embodiments of the present disclosure, the smart contract or the scheduling service dynamically refreshes the index mapping relationship of all data shards in the training data pool in each round of data acquisition, ensuring that the shard index of the current round strictly corresponds to the latest data state registered on the chain. This mechanism effectively prevents nodes from predicting or manipulating task allocation by caching old indexes. Through the dynamic maintenance of indexes and the on-chain and off-chain collaborative addressing, the safe and accurate scheduling of training data is realized, providing a trusted input for subsequent distributed training.

[0084] Figure 5 A flowchart of a data shard allocation method according to an embodiment of the present disclosure is shown. In the embodiments of the present disclosure, in Figure 2 Based on the blockchain-based model training method shown in FIG. 2, the step S230 can include the following steps.

[0085] In step S510, priorities of the processing nodes are determined according to the attribute information of the processing nodes.

[0086] In the embodiments of the present disclosure, the smart contract executes a set of dynamic processing node evaluation algorithms to determine the priority order of task allocation. The input of the algorithm is the attribute information of the processing nodes, which constitutes a multi-dimensional portrait of the node credibility and capability.

[0087] In exemplary embodiments, the attribute information includes at least one of reward and punishment information, bandwidth information, state information, and processing capability information. The reward and punishment information represents the reputation quantization value accumulated by the processing node due to the behavior result in the historical task, reflecting its long-term credibility and contribution to cooperation. The bandwidth information represents the current real-time network data transmission capability of the processing node, which directly determines its efficiency of obtaining data shards and uploading training results. The state information represents the current running health status of the processing node, such as whether it is online, idle or full, which is the basis for its immediate response to task scheduling. The processing capability information represents the hardware computing performance (such as GPU / CPU computing power) of the processing node, which is used to evaluate the speed and potential of the processing node in executing local model training tasks.

[0088] In exemplary embodiments, taking the attribute information including reward and punishment information and bandwidth information as an example, the priority of each processing node can be determined by the following formula:

[0089] wherein, is the priority; is the reward and punishment information of the processing node; is the bandwidth information of the processing node; is the maximum bandwidth; , is the weight coefficient of different attributes.

[0090] In step S520, the allocated data shards are obtained from the target number of data shards according to the priorities of the processing nodes.

[0091] In the embodiments of the present disclosure, the smart contract assigns each data shard to the corresponding processing node in order from high to low priority. In this processing, the selection of data shards and the allocation of processing nodes are decoupled, thereby effectively preventing nodes from plotting cheating or launching attacks on specific data. The allocation of nodes can be dynamically scheduled independently according to real-time network status and node performance, so that high-performance and high-bandwidth nodes can obtain tasks in priority. At the same time, this method can be more flexible in the increase or decrease of processing nodes and the increase or decrease of data shards, thereby adapting to changes in scale and complex environments.

[0092] In some example embodiments, the blockchain-based model training method can further include the following steps.

[0093] performing quality evaluation on the training sub-results uploaded by each processing node to obtain a quality evaluation result; determining the reward and punishment information corresponding to each processing node according to the quality evaluation result corresponding to each processing node.

[0094] In the embodiments of the present disclosure, the quality and timeliness of the training results submitted by each processing node are evaluated, aiming to objectively measure the true value of the training sub-results submitted by each processing node. The smart contract receives all the training sub-results uploaded by the nodes and automatically analyzes them through a set of preset evaluation algorithms. The evaluation criteria are usually multi-dimensional: first, timeliness, that is, whether the result is submitted within the specified timeout threshold; second, effectiveness and consistency, which can be achieved by comparing the trend of the results of a single node with the aggregated overall results, or by using the mutual verification of the results of the nodes in the threshold signature mechanism. Through this series of calculations, the system outputs a quantitative quality evaluation result for the performance of each node in this task.

[0095] According to the above quality evaluation result, the system automatically performs incentive settlement and reputation update on the chain to determine the reward and punishment information corresponding to each processing node. For example, for a node that submits on time and is verified as a high-quality result, the smart contract will correspondingly improve its "reputation value" on the chain, so that the node can obtain a higher priority in future task allocation. Conversely, for a node that submits late, the result is invalid, or is found to have fraudulent behavior, the contract will perform punitive measures to reduce its reputation value. Through this mechanism, the individual interests of the nodes are effectively aligned with the overall goal of the network, thereby promoting benign competition among the nodes and ensuring the long-term stability and healthy development of the training network.

[0096] In example embodiments, the reward and punishment information corresponding to each processing node can be determined by the following formula:

[0097] wherein, is the change value of the reward and punishment information; is the task weight of the processing node in this round; is the current reward and punishment information of the processing node; , , is the corresponding coefficient.

[0098] Figure 6 A flowchart of a processing node training method according to an embodiment of the present disclosure is shown. In the embodiments of the present disclosure, in the training process of each processing node, the processing node first uploads the training sub-result to the smart contract, and the smart contract performs quality evaluation on the training sub-result uploaded by each processing node to obtain a quality evaluation result. Figure 2Based on the illustrated blockchain-based model training method, the step S240 can include the following steps.

[0099] In step S610, based on the first block corresponding to the allocated data shard, the first identification information and the first source address corresponding to the first data shard are obtained.

[0100] In the embodiments of the present disclosure, the first block corresponding to the allocated data shard is determined on the blockchain. The first block can be any block on the blockchain. The first identification information and the first source address corresponding to the first data shard stored in the first block are obtained. The first identification information is usually the unique ID of the data shard or its positioning information in the Merkle tree. The first source address is a pointer to the actual storage location of the data shard, such as an IPFS content identifier (CID) or a specific network address.

[0101] In step S620, the first data shard is obtained according to the first source address.

[0102] In the embodiments of the present disclosure, the processing node pulls the actual data shard content from the corresponding network address according to the first source address. The node connects to, for example, IPFS, Arweave, or a designated private storage network, and requests to download the corresponding data content by submitting the source address (such as CID).

[0103] In step S630, the to-be-trained model is trained based on the first data shard to obtain a first training sub-result.

[0104] In the embodiments of the present disclosure, based on the data content of the obtained first data shard, the model training task is performed locally. The processing node loads the current to-be-trained model (the to-be-trained model after the last round of training), feeds the samples in the data shard into the model for forward propagation calculation, and calculates the gradient or update amount of the model parameters according to the preset loss function through the back propagation algorithm. This calculation process is completely completed locally on the node, and the generated gradient or parameter increment constitutes the first training sub-result of the processing node.

[0105] This step is the core calculation link of distributed training, which successfully decomposes the global training task into numerous parallelizable micro-tasks. The training sub-result output by each processing node can be regarded as the local knowledge learned by the global model on the specific data subset. This design not only fully utilizes the distributed computing power and greatly improves the training efficiency, but also ensures that the original training data does not leave its storage location, thereby providing data security assurance for data providers.

[0106] In the exemplary embodiments, the step S630 can further include the following steps.

[0107] performing a hash calculation on the first data shard to obtain first verification information; verifying the first verification information against the first identification information; in response to a verification pass, training the to-be-trained model based on the first data shard.

[0108] In the embodiments of the present disclosure, after the processing node successfully pulls the assigned data shard from the network, the data integrity can also be verified by the blockchain. The processing node will use the same algorithm as in the data registration stage (such as constructing a Merkle tree and calculating its root hash) on the pulled first data shard to obtain first verification information as the verification information of the data. The processing node compares and verifies the first verification information calculated locally with the first identification information (the hash value registered in the registration stage) obtained from the first block of the blockchain. If they are completely consistent, it proves that the data shard has not been tampered with by any party since registration, and is intact in the transmission process. In response to the verification pass, the subsequent training step of the to-be-trained model is based on the pulled first data shard. Otherwise, the pulled first data shard is discarded. The verification process effectively prevents malicious nodes from providing contaminated data or introducing erroneous data in the network transmission to harm the model training, and fundamentally guarantees the security of the data used to update the global model.

[0109] Figure 7 The flow of the result aggregation method of the embodiments of the present disclosure is shown Figure 1 In the embodiments of the present disclosure, based on the model training method based on the blockchain shown in Figure 2 The step S250 can include the following steps based on the model training method based on the blockchain shown in

[0110] In step S710, the training sub-results uploaded by each processing node are received; the training sub-results are subjected to signature processing.

[0111] In the embodiments of the present disclosure, after completing the local model training, each processing node will generate its training sub-result (such as gradient or model parameter update amount). Before the result is sent, the training sub-result will be subjected to signature processing. Specifically, each node uses its own private key to generate a digital signature for the training sub-result, and then uploads the training sub-result-signature pair to the blockchain network.

[0112] In step S720, the training sub-results of each processing node are subjected to signature verification by pre-stored signature verification information.

[0113] In the embodiments of the present disclosure, the training sub-results of each processing node are signature-verified by the signature verification information pre-stored in the smart contract. The verification process includes: using the public key of the node to decrypt the signature and comparing it with the value obtained by hashing the training sub-result itself. If the two match, it proves that the result is indeed sent by the holder of the corresponding private key (i.e. the node) and the content is complete and correct. Signature verification can immediately identify and reject training sub-results with unknown sources, fake identities, or tampered contents during transmission.

[0114] In step S730, in response to the signature verification of the training sub-results of each processing node passing, the training sub-results are aggregated to obtain the aggregated result.

[0115] In the embodiments of the present disclosure, in response to the signature verification of the training sub-results of each processing node passing, the smart contract aggregates each training sub-result to obtain an aggregated result. The aggregated result can reflect the total training result of each processing node in this round of training process.

[0116] In the exemplary embodiments, the training sub-result obtained by each processing node is a calculated gradient parameter. The aggregated result is obtained by calculating the average gradient of each training sub-result, i.e. performing element-wise arithmetic average on all qualified gradient vectors from each data shard.

[0117] Figure 8 The result aggregation method of the embodiments of the present disclosure is shown Figure 2 In the embodiments of the present disclosure, in Figure 9 Based on the blockchain-based model training method shown in

[0118] In step S810, the training sub-results uploaded by each processing node are received; the training sub-results are signature-processed.

[0119] In the embodiments of the present disclosure, after completing the local model training, each processing node generates its training sub-result (e.g. gradient or model parameter update amount). Before the result is sent, the training sub-result is signature-processed. Specifically, each node uses its own private key to generate a digital signature for the training sub-result, and then uploads the training sub-result-signature pair to the blockchain network.

[0120] In an embodiment of the present disclosure, in a round of training, each data shard forms a data shard group. After each processing node in the group completes local training, it not only generates a training sub-result, but also generates a BLS signature fragment of the result using its personal private key. This signature fragment is part of a threshold signature scheme. Threshold signature is a cryptographic scheme that allows any group of members to cooperate to generate a unified and valid digital signature without the participation of all members, and a single member's signature fragment cannot represent the whole group.

[0121] In step S820, a corresponding signature fragment is obtained according to each training sub-result.

[0122] In an embodiment of the present disclosure, the BLS signature fragment corresponding to each training sub-result is parsed and extracted. At this stage, the contract may temporarily cache these signature fragments, or perform preliminary format checking on the training sub-results, but the focus of its verification is not the correctness of the individual results, but the format validity of the signature fragments themselves.

[0123] In step S830, in response to obtaining a preset number of signature fragments, the preset number of signature fragments are aggregated to obtain an aggregated signature.

[0124] In an embodiment of the present disclosure, the smart contract continues to monitor until the number of valid signature fragments collected reaches a preset threshold value (for example, two-thirds of the nodes in the group). Once this condition is met, the contract immediately triggers an aggregation algorithm to combine these independent signature fragments into a single, complete aggregated signature. This aggregated signature represents the collective confirmation of the majority of nodes in the processing group to the training task result. Only when a sufficient number of processing nodes actually complete and agree to submit the result will the smart contract be executed. At the same time, in this scheme, only a signature verification information corresponding to the aggregated signature needs to be pre-stored on the blockchain.

[0125] In step S840, the aggregated signature is verified by the pre-stored signature verification information.

[0126] In an embodiment of the present disclosure, the contract no longer verifies each node's individual signature one by one, but uses the aggregated public key (i.e., the "signature verification information") pre-deployed on the chain to verify this single aggregated signature. Verification success proves that a sufficient number of authorized nodes participated in the training and reached a consensus. This greatly improves the verification efficiency and fundamentally prevents "empty signature" attacks, because malicious nodes cannot fake a valid signature that can pass the aggregated public key verification.

[0127] In step S850, in response to passing the signature verification of the aggregated signature, the training sub-results are aggregated to obtain the aggregated result.

[0128] In the embodiments of the present disclosure, in response to the signature verification passing of the aggregated signature, the smart contract aggregates the training sub-results of each processing node to obtain an aggregated result. The aggregated result can reflect the summation training result of each processing node in the training process of the round.

[0129] In the example embodiments, the training sub-result obtained by each processing node is a gradient parameter generated by calculation. The aggregated result is obtained by calculating the average gradient of each training sub-result, that is, performing element-wise arithmetic average on all qualified gradient vectors from each data shard.

[0130] Figure 2 A flowchart of the parameter adjustment method of the embodiments of the present disclosure is shown. In the embodiments of the present disclosure, in the step S260 based on the blockchain-based model training method shown in the above, the step S260 can include the following steps. Figure 10

[0131] In step S910, the training sub-results uploaded by each processing node are received.

[0132] In the embodiments of the present disclosure, the system enters the data preparation phase of global model updating. The smart contract receives the training sub-results calculated and uploaded by each processing node for each data shard in the round. These sub-results are usually the gradient or update amount of the model parameters, which represents the characteristics learned by each processing node on its local data shard. This step is the process of collecting information from distributed nodes to the consensus center.

[0133] In step S920, the average gradient of each training sub-result is calculated to obtain the aggregated result.

[0134] In the embodiments of the present disclosure, the aggregated result is obtained by calculating the average gradient of each training sub-result. By taking the average, the system can smooth out the deviation caused by data anomalies or calculation errors of individual nodes, so that the final update direction can better represent the overall trend of the entire training data set, thereby ensuring the robustness and generalization ability of model optimization.

[0135] In the example embodiments, the average gradient calculation can be performed by the following formula:

[0136] wherein, is the aggregated result; is the training sub-result of each processing node.

[0137] In step S930, the parameters of the model to be trained are adjusted based on the aggregated result.

[0138] ​In the embodiments of the present disclosure, the smart contract obtains the parameters of the current to-be-trained model (the to-be-trained model after the last round of training), and then applies the average gradient to the parameters according to the rules of the optimization algorithm (such as the stochastic gradient descent method SGD), so as to complete the parameter adjustment of the to-be-trained model in this round. It should be pointed out that the average gradient can also be adjusted based on the pre-set learning rate, and the adjusted average gradient is applied to the relevant parameters.

[0139] In step S940, in response to the parameters of the to-be-trained model satisfying the preset convergence condition, the target training model is obtained.

[0140] In the embodiments of the present disclosure, it is evaluated whether the parameters of the to-be-trained model after the above parameter adjustment satisfy the preset convergence condition. These convergence conditions usually include that the training iteration round reaches the upper limit, the loss function value drops below the threshold, or the amplitude of model parameter update becomes extremely small, etc. When the parameters of the to-be-trained model satisfy the preset convergence condition, the model is taken as the target training model. When the parameters of the to-be-trained model do not satisfy the preset convergence condition, the to-be-trained model is taken as the basis for the next round of model training.

[0141] Based on the same inventive concept, the embodiments of the present disclosure provide a model training device based on a block chain, as described in the following embodiments. Since the principle of solving problems of the device embodiments is similar to that of the above-mentioned method embodiments, the repeated parts will not be described again.

[0142] Figure 10 The structure schematic diagram of a model training device based on a block chain in the embodiments of the present disclosure is shown. As shown in Figure 11 The model training device based on a block chain 1000 can include a data shard on-chain module 1010, a data shard obtaining module 1020, a data shard distribution module 1030, a processing node training module 1040, a result aggregation module 1050 and a parameter adjustment module 1060.

[0143] The data shard on-chain module 1010 is configured to obtain training data including a plurality of data shards, and each data shard corresponds to a block on a block chain. The data shard obtaining module 1020 is configured to obtain a target number of data shards from the training data based on the block chain. The data shard distribution module 1030 is configured to distribute the target number of data shards to a plurality of processing nodes. The processing node training module 1040 is configured to train a to-be-trained model based on the distributed data shards by each processing node, and obtain a corresponding training sub-result. The result aggregation module 1050 is configured to aggregate the training sub-results corresponding to the processing nodes to obtain an aggregated result. The parameter adjustment module 1060 is configured to adjust the parameters of the to-be-trained model based on the aggregated result to obtain a target training model.

[0144] In an example embodiment, the obtaining the training data including a plurality of data shards includes: dividing the training data into a plurality of data shards; performing hash calculation on each data shard to obtain a corresponding hash value; and registering the hash value on the blockchain to determine the block corresponding to the data shard.

[0145] In an example embodiment, the block stores at least identification information and a source address of the corresponding data shard; the identification information is used to identify the corresponding data shard; and the source address is used to represent a data storage address of the corresponding data shard.

[0146] In an example embodiment, the obtaining the target number of data shards from the training data based on the blockchain includes: obtaining identification information of a previous block on the blockchain; generating a shard selection random value based on the identification information of the previous block through a random function; determining the target number of shard indexes based on the shard selection random value; and obtaining the target number of data shards from the training data according to the target number of shard indexes.

[0147] In an example embodiment, the obtaining the target number of data shards from the training data according to the target number of shard indexes includes: refreshing index information of each data shard in the training data; and obtaining the target number of data shards from the training data according to the target number of shard indexes.

[0148] In an example embodiment, the allocating the target number of data shards to the plurality of processing nodes includes: determining a priority of each processing node according to attribute information of each processing node; and obtaining an allocated data shard from the target number of data shards according to the priority of each processing node.

[0149] In an example embodiment, the attribute information includes at least one of reward and punishment information, bandwidth information, state information, and processing capability information.

[0150] In the example embodiment, the respective processing nodes train the to-be-trained model based on the allocated data shards to obtain corresponding training sub-results, including: based on a first block corresponding to the allocated first data shard, obtaining first identification information and a first source address corresponding to the first data shard; obtaining the first data shard according to the first source address; and training the to-be-trained model based on the first data shard to obtain a first training sub-result.

[0151] In the example embodiment, the respective processing nodes train the to-be-trained model based on the allocated data shards to obtain corresponding training sub-results, including: based on a first block corresponding to the allocated first data shard, obtaining first identification information and a first source address corresponding to the first data shard; obtaining the first data shard according to the first source address; and training the to-be-trained model based on the first data shard to obtain a first training sub-result.

[0152] In the example embodiment, the respective processing nodes train the to-be-trained model based on the allocated data shards to obtain corresponding training sub-results, including: based on a first block corresponding to the allocated first data shard, obtaining first identification information and a first source address corresponding to the first data shard; obtaining the first data shard according to the first source address; and training the to-be-trained model based on the first data shard to obtain a first training sub-result.

[0153] In the example embodiment, the respective processing nodes train the to-be-trained model based on the allocated data shards to obtain corresponding training sub-results, including: based on a first block corresponding to the allocated first data shard, obtaining first identification information and a first source address corresponding to the first data shard; obtaining the first data shard according to the first source address; and training the to-be-trained model based on the first data shard to obtain a first training sub-result.

[0154] In the example embodiment, the respective processing nodes train the to-be-trained model based on the allocated data shards to obtain corresponding training sub-results, including: based on a first block corresponding to the allocated first data shard, obtaining first identification information and a first source address corresponding to the first data shard; obtaining the first data shard according to the first source address; and training the to-be-trained model based on the first data shard to obtain a first training sub-result.

[0155] In the example embodiment, the training sub-results uploaded by each processing node are quality evaluated to obtain quality evaluation results; and the reward and punishment information corresponding to each processing node is determined according to the quality evaluation results corresponding to the processing node.

[0156] Figure 11 A structural schematic diagram of an electronic device suitable for implementing the example embodiment of the present disclosure is shown. The electronic device 1100 according to this implementation of the present disclosure is described below with reference to Figure 11 Figure 11 The electronic device 1100 shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0157] As shown in ​ , the electronic device 1100 is in the form of a general computing device. The components of the electronic device 1100 can include, but are not limited to, the at least one processing unit 1110 described above, the at least one storage unit 1120 described above, a bus 1130 connecting different system components (including the storage unit 1120 and the processing unit 1110), and a display unit 1140.

[0158] The storage unit 1120 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 11201 and / or a cache memory unit 11202, and can further include a read-only memory (ROM) 11203.

[0159] The storage unit 1120 can further include a program / utility 11204 having a set of program modules 11205, including but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof can include implementation of a network environment.

[0160] The bus 1130 can represent one or more of several types of bus structures, including a storage unit bus or storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit bus, or a local bus using any of a variety of bus architectures.

[0161] ​The electronic device 1100 can also communicate with one or more external devices 1170 such as a keyboard or pointing device, a Bluetooth device, or a device that enables a user to interact with the electronic device 1100. Additionally, the electronic device 1100 can further communicate with one or more devices that enable the electronic device 1100 to interact with one or more other computing devices. Such communication can occur via an input / output (I / O) interface 1150. Still yet, the electronic device 1100 can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or the Internet) through a network adapter 1160. As depicted, the network adapter 1160 communicates with the other components of the electronic device 1100 via the bus 1130. It should be appreciated that the electronic device 1100 can be a part of one or more components of a larger computer system, which includes one or more servers, one or more routers, one or more other electronic devices, and / or one or more external devices, and can be accessed via one or more network adapters 1160.

[0162] In the exemplary embodiments of the present disclosure, a computer readable storage medium is also provided, on which a program product capable of implementing the above-mentioned method of the present specification is stored.

[0163] In some possible implementation manners, various aspects of the present application can also be implemented in the form of a program product, which includes program codes for causing an end device to perform the steps according to various exemplary embodiments of the present application described in the above-mentioned “Exemplary Method” section of the present specification when the program product is run on the end device.

[0164] The program product for implementing the above-mentioned method according to the embodiments of the present application can be in the form of a portable compact disc read-only memory (CD-ROM) and includes program codes, and can be run on an end device such as a personal computer. However, the program product of the present application is not limited to this, and in the present document, the readable storage medium can be any tangible medium containing or storing a program, which can be used by or in conjunction with a signaling execution system, device or apparatus.

[0165] The program product can take any combination of one or more computer-readable media. The computer-readable media can be a computer-readable storage medium or a computer-readable signal medium. The computer-readable storage medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include the following: an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0166] The computer-readable signal medium can include a computer-readable storage medium that is configured to store and deliver a computer-readable program code. The computer-readable program code can be propagated as a computer-readable signal medium.

[0167] The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the foregoing.

[0168] The program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, C++, etc., or conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0169] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, such a division is not mandatory. Indeed, according to embodiments of the present disclosure, features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, features and functions of one module or unit described above can be further divided into a plurality of modules or units.

[0170] Moreover, although individual steps of the methods in the present disclosure are described in a particular order in the drawings, this is not required or implied as to the order of the steps, nor is it required that all of the steps be performed to achieve the desired result. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, one step can be broken into multiple steps, etc.

[0171] From the above description of the embodiments, those skilled in the art will readily perceive that the example embodiments described herein can be implemented by software and / or by hardware coupled with software. Accordingly, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.) or network, and includes a number of signals to enable a computing device (which can be a personal computer, server, mobile terminal, or network device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0172] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the present disclosure cover any and all variations of the present disclosure that come within the scope of the following claims and their equivalents. It is intended that the specification and examples be considered exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0173] It should be understood that the present disclosure is not limited to the precise structures described and shown in the drawings, and that various modifications and changes can be made without departing from the scope of the present disclosure. The scope of the present disclosure is limited only by the claims that follow.

Claims

1. A blockchain-based model training method, characterized in that, The method includes: Acquire training data comprising multiple data shards; each of the data shards corresponds to a block on the blockchain; Based on the blockchain, a target number of data shards are obtained from the training data; The target number of data fragments are distributed to multiple processing nodes; Each of the processing nodes trains the model to be trained based on the allocated data shards, and obtains the corresponding training sub-results; The training sub-results corresponding to each of the processing nodes are aggregated to obtain an aggregated result; Based on the aggregation results, the parameters of the model to be trained are adjusted to obtain the target training model.

2. The method according to claim 1, characterized in that, The acquisition of training data comprising multiple data slices includes: The training data is divided into multiple data slices; Perform hash calculations on each of the data fragments to obtain the corresponding hash values; Based on the hash value, the block corresponding to the data shard is determined by registering it on the blockchain.

3. The method according to claim 2, characterized in that, The block stores at least the identification information and source address of the corresponding data fragment; the identification information is used to identify the corresponding data fragment; the source address is used to represent the data storage address of the corresponding data fragment.

4. The method according to claim 1, characterized in that, The step of obtaining a target number of data shards from the training data based on the blockchain includes: Obtain the identifier information of the previous block on the blockchain; Based on the identification information of the previous block, a random value for selecting a segment is generated using a random function; Based on the selected random value, determine the target number of shard indices; Based on the target number of shard indices, obtain the target number of data shards from the training data.

5. The method according to claim 4, characterized in that, The step of obtaining the target number of data shards from the training data according to the target number of shard indices includes: Refresh the index information of each data segment in the training data; Based on the target number of shard indices, obtain the target number of data shards from the training data.

6. The method according to claim 1, characterized in that, The step of allocating the target number of data shards to multiple processing nodes includes: The priority of each processing node is determined based on its attribute information. Based on the priority of each processing node, the allocated data shards are obtained from the target number of data shards.

7. The method according to claim 6, characterized in that, The attribute information includes at least one of the following: reward and punishment information, bandwidth information, status information, and processing capacity information.

8. The method according to claim 1, characterized in that, Each of the processing nodes trains the model to be trained based on the allocated data shards, obtaining corresponding training sub-results, including: Based on the first block corresponding to the allocated data fragment, obtain the first identifier information and the first source address corresponding to the first data fragment; Based on the first source address, obtain the first data fragment; The model to be trained is trained based on the first data slice to obtain the first training sub-result.

9. The method according to claim 8, characterized in that, Each of the processing nodes trains the model to be trained based on the allocated data shards, obtaining corresponding training sub-results, and further includes: Perform hash calculation on the first data fragment to obtain the first verification information; Verify the first verification information with the first identification information; Upon successful verification, the model to be trained is trained based on the first data segment.

10. The method according to claim 1, characterized in that, The step of aggregating the training sub-results corresponding to each of the processing nodes to obtain the aggregation result includes: Receive the training sub-results uploaded by each of the processing nodes; the training sub-results are signed. The training sub-results of each processing node are verified by pre-stored signature verification information. In response to the successful signature verification of the training sub-results at each of the processing nodes, the training sub-results are aggregated to obtain the aggregated result.

11. The method according to claim 1, characterized in that, The step of aggregating the training sub-results corresponding to each of the processing nodes to obtain the aggregation result further includes: Receive the training sub-results uploaded by each of the processing nodes; the training sub-results are signed. Based on the results of each training sub-project, obtain the corresponding signature fragment; In response to obtaining a preset number of signature fragments, the preset number of signature fragments are aggregated to obtain an aggregated signature; The aggregated signature is verified using pre-stored signature verification information. In response to the successful signature verification of the aggregated signature, the individual training sub-results are aggregated to obtain the aggregated result.

12. The method according to claim 1, characterized in that, The step of adjusting the parameters of the model to be trained based on the aggregation results to obtain the target training model includes: Receive the training sub-results uploaded by each of the processing nodes; Calculate the average gradient of each of the training sub-results to obtain the aggregated result; The parameters of the model to be trained are adjusted based on the aggregation results; The target training model is obtained when the parameters of the model to be trained satisfy a preset convergence condition.

13. The method according to claim 1, characterized in that, The method further includes: The quality of the training sub-results uploaded by each of the processing nodes is evaluated to obtain the quality evaluation results; Based on the quality assessment results corresponding to each processing node, the reward and punishment information corresponding to the processing node is determined.

14. A blockchain-based model training device, characterized in that, include: The data sharding and on-chain module is configured to acquire training data comprising multiple data shards; each of the data shards corresponds to a block on the blockchain. The data shard acquisition module is configured to acquire a target number of data shards from the training data based on the blockchain; The data sharding allocation module is configured to allocate the target number of data shards to multiple processing nodes; The processing node training module is configured such that each processing node trains the model to be trained based on the allocated data shards, and obtains the corresponding training sub-results; The result aggregation module is configured to aggregate the training sub-results corresponding to each of the processing nodes to obtain an aggregation result. The parameter adjustment module is configured to adjust the parameters of the model to be trained based on the aggregation results to obtain the target training model.

15. An electronic device, characterized in that, include: One or more processors; A storage device configured to store one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 13.

16. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 13.

17. A computer program product, comprising a computer program / signaling, characterized in that, When the computer program / signaling is executed by the processor, it implements the method as described in any one of claims 1 to 13.

Images