A high-performance computing data processing method and system based on blockchain

Abstract

This application discloses a high-performance computing power data processing method and system based on blockchain. The method includes: sending a query request to a computing power pool; in response to the query request, multiple computing power nodes in the computing power pool generate computing power data respectively; classifying and fusing the computing power data to obtain multiple types of fused computing power data; comparing the multiple types of fused computing power data to distinguish between valid and invalid computing power data; storing the valid computing power data and discarding the invalid computing power data. By classifying and verifying the calculation results, this method improves the utilization of computing power resources, ensuring security while improving efficiency, thus making it better suited for demanding scenarios such as finance and healthcare.

Description

Technical Field

[0001] This application relates to the field of blockchain technology, and in particular to a high-performance computing data processing method and system based on blockchain. Background Technology

[0002] Computer technology is advancing rapidly, profoundly impacting people's daily work and lives. With continuous innovation in algorithms and computing power, blockchain technology also faces new challenges. Blockchain's characteristics include decentralization, immutability, transparency, security, and programmability, leading to its widespread application in finance, supply chain, healthcare, real estate, and other fields. However, as its application deepens, some shortcomings of existing blockchain technology have been exposed, such as resource consumption issues, security risks, privacy risks, and data accuracy concerns. Especially considering the characteristics of its application scenarios, the requirements for security, privacy, and resource consumption are constantly increasing. Summary of the Invention

[0003] This application provides a high-performance computing power data processing method and system based on blockchain to solve the problems of wasted computing power resources and security in existing blockchain technologies.

[0004] To address the aforementioned technical problems, this application adopts the following technical solution: providing a high-performance computing power data processing method based on blockchain, comprising:

[0005] S10: Send a query request to the computing power pool;

[0006] S20: In response to the query request, the multiple computing nodes in the computing power pool generate computing power data respectively;

[0007] S30: Classify and fuse the computing power data to obtain multi-class fused computing power data;

[0008] S40: Compare the multiple types of fused computing power data and distinguish between effective computing power data and invalid computing power data;

[0009] S50: Store the valid computing power data and discard the invalid computing power data.

[0010] In one possible implementation, prior to the step of issuing a query request to the computing power pool, the method further includes:

[0011] S11: Verify the qualifications of computing power nodes and include the verified computing power nodes into the computing power pool.

[0012] In one possible implementation, after the steps of storing the valid computing power data and discarding the invalid computing power data, the method further includes:

[0013] S60: Monitor the load of each computing node in the computing power pool in real time and dynamically adjust task allocation.

[0014] In one possible implementation, after the steps of storing the valid computing power data and discarding the invalid computing power data, the method further includes:

[0015] S70: Based on the effective computing power data and response speed, improve reward allocation and rating; based on the ineffective computing power data, reduce rewards and lower ratings.

[0016] In one possible implementation, the step of generating computing power data by multiple computing power nodes in the computing power pool in response to the query request includes:

[0017] S21: Divide the large task into several sub-tasks and distribute them to multiple computing power nodes;

[0018] S22: The computing power node processes each of the subtasks using a specified algorithm version and input parameters;

[0019] S23: The computing nodes periodically hash the intermediate calculation results onto the blockchain until the calculation is complete.

[0020] In one possible implementation, the step of comparing the multiple types of fused computing power data and distinguishing between effective computing power data and invalid computing power data includes:

[0021] S41: Each computing node submits a zk-SNARK proof to verify its validity.

[0022] In one possible implementation, the step of comparing the multiple types of fused computing power data and distinguishing between effective computing power data and invalid computing power data includes:

[0023] S42: Randomly sample and verify the multi-type fusion computing power data to confirm its effectiveness.

[0024] In one possible implementation, after the steps of storing the valid computing power data and discarding the invalid computing power data, the method further includes:

[0025] S51: Record the hash and reason for the invalid computing power data to the blockchain for auditing and verification;

[0026] S52: Release the storage space occupied by the invalid computing power data to reallocate other tasks.

[0027] To address the aforementioned technical problems, another technical solution adopted in this application is: providing a blockchain-based high-performance computing data processing system, applicable to the aforementioned blockchain-based high-performance computing data processing method, comprising:

[0028] The request module is used to send a computing power data query request to the computing power pool;

[0029] The receiving module is configured to respond to the query request by receiving the computing power data generated by multiple computing power nodes in the computing power pool.

[0030] The fusion module is used to classify and fuse the received computing power data to generate the multi-class fused computing power data;

[0031] The processing module is used to compare multiple types of fused computing power data, determine the valid computing power data and the invalid computing power data, and store the valid computing power data and discard the invalid computing power data.

[0032] To address the aforementioned technical problems, another technical solution adopted in this application is: providing a computer-readable storage medium for high-performance computing data processing based on blockchain, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the steps of the aforementioned high-performance computing data processing method based on blockchain.

[0033] The beneficial effects of this application are as follows: Unlike the existing technology, this application discloses a high-performance computing power data processing method and system based on blockchain. By dividing and verifying the calculation results, it improves the utilization of computing power resources, and ensures security while improving efficiency, thus making it more suitable for demanding scenarios such as finance and healthcare. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0035] Figure 1 This is a flowchart of the steps of a high-performance computing data processing method based on blockchain in one embodiment of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0037] The terms "first," "second," and "third" used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0038] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0039] Please see Figure 1 This application includes a blockchain-based high-performance computing data processing method, comprising the following steps:

[0040] S10: Send a query request to the computing power pool;

[0041] S20: In response to the query request, multiple computing nodes in the computing power pool generate computing power data respectively;

[0042] S30: Classify and fuse computing power data to obtain multi-class fused computing power data;

[0043] S40: Compare multiple types of fused computing power data to distinguish between effective and ineffective computing power data;

[0044] S50: Stores valid computing power data and discards invalid computing power data.

[0045] Specifically, the query request in step S10 can be designed as a standardized request format based on JSON or Protobuf, including information such as task type, data size, deadline, and resource requirements. The task type can be AI training, cryptographic computation, etc., and resource requirements can include CPU, GPU, storage, etc., to organize and match tasks with computing power nodes. Furthermore, smart contracts, such as Solidity contracts, can be set up to automatically generate query requests, ensuring the transparency and immutability of the requests.

[0046] The query request can be sent in either a broadcast mode, where the request is broadcast directly to each node in the computing power pool via Kafka or the blockchain network, or a subscription mode, where computing power nodes subscribe to corresponding tasks according to their own capabilities and resources.

[0047] In step S20, each computing node performs calculations on the obtained tasks based on the received query requests and obtains computing power data.

[0048] In step S30, the classification strategy for computing power data can be based on task type or resource type. Task types include computational and storage-based data. Computational data includes AI model training, cryptographic computation, etc., while storage-based data includes distributed file storage, database index construction, etc. Resource types include GPU computing power data, FPGA accelerated data, and lightweight edge node data, etc. The classification strategy should be selected based on the specific task or scenario.

[0049] Data fusion methods can be consensus-driven aggregation or federated learning fusion. The former uses PBFT or Raft protocols to perform multi-node cross-validation on similar data and generate the final fusion result, while the latter is for distributed AI tasks, which aggregates model parameters through secure multi-party computation to avoid leakage of original data.

[0050] In step S40, the validity of computing power data can be determined based on consensus compliance and resource utility. Valid computing power data needs to meet technical validity, economic validity, and security validity. Technical validity means that the task was completed according to the protocol. Economic validity means that the input-output ratio is reasonable, that is, avoiding situations where costs cannot be covered. Security validity means that there is no malicious behavior, such as attack behavior or theft of computing power resources.

[0051] Methods for classifying the validity of computing power data include having nodes submit zk-SHARK proofs and randomly selecting some nodes for repeated verification. For technical validity, methods such as data format verification, hash matching, and TEE verification can be used. For economic validity, methods such as input-output threshold checks can be used. For security validity, methods such as smart contract detection can be used.

[0052] Furthermore, machine learning and other models can be introduced to process invalid computing power data, analyze its characteristics, and then dynamically adjust the detection threshold and downgrade the rating of nodes that frequently submit invalid computing power data, so as to improve the detection of the validity of the detection data.

[0053] In step S50, the storage scheme for valid computing power data can be a hierarchical storage architecture or a data index architecture. The processing of invalid computing power data can include resource reclamation and data erasure after recording traces on the chain. The content of the on-chain record includes hash and judgment reason. After data erasure, random data overwriting can also be performed to ensure trace removal.

[0054] In one embodiment, before the step of issuing a query request to the computing power pool, the method further includes:

[0055] S11: Verify the qualifications of computing power nodes and include verified computing power nodes into the computing power pool.

[0056] Specifically, computing power nodes need to pass KYC or AML verification and can verify trustworthiness through staking, with collateral being tokens or hardware resources. Nodes can verify their identity using digital signatures (such as ECDSA and BLS) to prevent Sybil attacks, and use secure channels to encrypt request content to ensure privacy. Smart contracts can also be used for node qualification verification, task assignment, and reward distribution.

[0057] In one embodiment, after the steps of storing valid computing power data and discarding invalid computing power data, the method further includes:

[0058] S60: Monitors the load of each computing node in the computing power pool in real time and dynamically adjusts task allocation.

[0059] Specifically, the load of each computing node in the computing power pool is monitored in real time. The monitoring content includes, but is not limited to, CPU load, GPU load, remaining storage space, response latency, etc., and the task allocation is dynamically adjusted based on the above parameters.

[0060] In one embodiment, after the steps of storing valid computing power data and discarding invalid computing power data, the method further includes:

[0061] S70: Improve reward allocation and rating based on effective computing power data and response speed; reduce rewards and lower ratings based on ineffective computing power data.

[0062] Specifically, each computing node can be rated and rewarded based on indicators such as the effectiveness and response speed of computing power data. Reward allocation can be based on the rating or set in parallel with the rating. Furthermore, a rating decay mechanism can be introduced to prevent early data nodes from gaining an excessive first-mover advantage over later data nodes, crowding out rating priority and affecting the introduction of subsequent data nodes.

[0063] In one embodiment, the step of multiple computing nodes in the computing power pool generating computing power data respectively in response to a query request includes:

[0064] S21: Break down large tasks into several sub-tasks and distribute them to multiple computing nodes;

[0065] S22: The computing nodes process each subtask using a specified algorithm version and input parameters;

[0066] S23: The computing nodes periodically hash the intermediate calculation results onto the blockchain until the calculation is complete.

[0067] Specifically, each computing node is required to use a specified algorithm version and input parameters to ensure the reproducibility of the computation results. Standardization requirements are imposed on the format of computing data, including but not limited to task codes, timestamps, resource usage logs, and result hash values, to ensure data standardization.

[0068] Intermediate calculation results are periodically uploaded to the blockchain to facilitate subsequent auditing and traceability. The time interval can be set based on requirements to ensure that the calculation is performed at least once in most calculation processes, avoiding the completion of the entire calculation between two uploads, which would render this step meaningless.

[0069] In one embodiment, the step of comparing multiple types of fused computing power data and distinguishing between effective computing power data and invalid computing power data includes:

[0070] S41: Each computing node submits a zk-SNARK proof to verify its validity.

[0071] zk-SNARK is a zero-knowledge proof method that can verify the correctness of a calculation without revealing the specific data content, thus ensuring that the data has not been tampered with and improving security and stability.

[0072] In one embodiment, the step of comparing multiple types of fused computing power data and distinguishing between effective computing power data and invalid computing power data includes:

[0073] S42: Randomly sample and verify the validity of multi-type fusion computing power data.

[0074] Specifically, it can involve randomly sampling and verifying multiple types of fused computing power data, or directly randomly sampling computing power nodes and verifying the data, comparing the verification results with the data, and judging the validity of the computing power data.

[0075] In one embodiment, after the steps of storing valid computing power data and discarding invalid computing power data, the method further includes:

[0076] S51: Record the hash and reason for invalid computing power data to the blockchain for auditing and verification purposes;

[0077] S52: Release the storage space occupied by invalid computing power data to be reallocated to other tasks.

[0078] Specifically, before discarding invalid computing power data, the hash and the reason for invalidity are saved for subsequent verification. The verification results are then used to correct the judgment method and process, ensuring that the judgment of data validity is dynamic and accurate, thereby improving stability and reliability.

[0079] This application also includes a blockchain-based high-performance computing data processing system, applicable to the aforementioned blockchain-based high-performance computing data processing method, comprising:

[0080] The request module is used to send computing power data query requests to the computing power pool;

[0081] The receiving module is used to respond to query requests and receive computing power data generated by multiple computing power nodes in the computing power pool.

[0082] The fusion module is used to classify and fuse the received computing power data to generate multiple types of fused computing power data.

[0083] The processing module is used to compare multiple types of fused computing power data, determine the valid computing power data and the invalid computing power data, and store the valid computing power data and discard the invalid computing power data.

[0084] This application embodiment also includes a computer-readable storage medium for high-performance computing data processing based on blockchain. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps of the aforementioned high-performance computing data processing method based on blockchain.

[0085] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A high-performance computing data processing method based on blockchain, characterized in that, include: S10: Send a query request to the computing power pool. The query request is designed with a standardized request format based on JSON or Protobuf, including task type and resource requirements. The task type includes AI training, cryptographic computation, distributed file storage and database index construction. The resource requirements include CPU, GPU and storage. S20: In response to the query request, the multiple computing nodes in the computing power pool generate computing power data respectively; S30: Classify and fuse the computing power data, and use PBFT or Raft protocol to perform multi-node cross-validation on the same type of computing power data to obtain multi-class fused computing power data; S40: Compare the multi-type fused computing power data and classify effective computing power data into invalid computing power data based on technical effectiveness, economic effectiveness, and security effectiveness; wherein, the technical effectiveness is determined by data format verification and / or hash matching and / or TEE verification, the economic effectiveness is determined by input-output threshold screening, and the security effectiveness is determined by smart contract detection; S50: Store the valid computing power data and discard the invalid computing power data; S70: Based on the effective computing power data and response speed, increase reward allocation and rating; based on the ineffective computing power data, reduce rewards and lower ratings.

2. The high-performance computing data processing method based on blockchain according to claim 1, characterized in that, Before the step of sending a query request to the computing power pool, the method further includes: S11: Verify the qualifications of computing power nodes and include the verified computing power nodes into the computing power pool.

3. The high-performance computing data processing method based on blockchain according to claim 1, characterized in that, After the steps of storing the valid computing power data and discarding the invalid computing power data, the method further includes: S60: Monitor the load of each computing node in the computing power pool in real time and dynamically adjust task allocation.

4. The high-performance computing data processing method based on blockchain according to claim 1, characterized in that, The step of generating computing power data by multiple computing power nodes in the computing power pool in response to the query request includes: S21: Divide the large task into several sub-tasks and distribute them to multiple computing power nodes; S22: The computing power node processes each of the subtasks using a specified algorithm version and input parameters; S23: The computing nodes periodically hash the intermediate calculation results onto the blockchain until the calculation is complete.

5. The high-performance computing data processing method based on blockchain according to claim 1, characterized in that, The step of comparing the multiple types of fused computing power data and distinguishing between effective computing power data and invalid computing power data includes: S41: Each computing node submits a zk-SNARK proof to verify its validity.

6. The high-performance computing data processing method based on blockchain according to claim 1, characterized in that, The step of comparing the multiple types of fused computing power data and distinguishing between effective computing power data and invalid computing power data includes: S42: Randomly sample and verify the multi-type fusion computing power data to confirm its effectiveness.

7. The high-performance computing data processing method based on blockchain according to claim 1, characterized in that, After the steps of storing the valid computing power data and discarding the invalid computing power data, the method further includes: S51: Record the hash and reason for the invalid computing power data to the blockchain for auditing and verification; S52: Release the storage space occupied by the invalid computing power data to reallocate other tasks.

8. A blockchain-based high-performance computing data processing system, applicable to the blockchain-based high-performance computing data processing method as described in any one of claims 1 to 7, characterized in that, include: The request module is used to send a computing power data query request to the computing power pool; The receiving module is configured to respond to the query request by receiving the computing power data generated by multiple computing power nodes in the computing power pool. The fusion module is used to classify and fuse the received computing power data to generate the multi-class fused computing power data; The processing module is used to compare multiple types of fused computing power data, determine the valid computing power data and the invalid computing power data, and store the valid computing power data and discard the invalid computing power data.

9. A high-performance computing data processing computer-readable storage medium based on blockchain, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the blockchain-based high-performance computing data processing method as described in any one of claims 1 to 7.

Images