Data co-processing method based on block chain

Abstract

The invention relates to the field of data co-processing, in particular to a block chain-based data co-processing method. The method comprises the following steps: firstly, carrying out data fragmentation processing on an obtained original data set, carrying out data processing on an obtained data fragment set by using a right weighted dynamic priority algorithm according to an obtained processing priority and running a built-in data processing logic of the smart contract, and carrying out verification by using a built-in data verification algorithm of the smart contract; the integrity verification score is obtained and compared with a verification threshold value, if the integrity verification score passes the verification threshold value, a data reassembly algorithm is used for processing, and an assembled data set is obtained. The problems of centralized consumption of computing resources and single-point bottleneck are solved; a dynamic adjustment mechanism is lacked; the integrity and credibility of the data are difficult to guarantee; large-scale data or extreme data is easily influenced by extreme values, and the processing result is distorted; the technical problems of insufficient robustness and inconsistent and unreliable verification results in the case of complex and unstable data fragments in a block chain environment are solved.

Description

Technical Field

[0001] The present invention relates to the field of collaborative data processing, and in particular to a collaborative data processing method based on blockchain. Background Art

[0002] With the rapid development of information technology, the amount of data worldwide is growing at an unprecedented rate and has driven the vigorous development of the digital economy. In the modern digital economy era, data has become the most important asset. Whether it is business decision-making, financial transactions or scientific research exploration, the accuracy, integrity and security of data are crucial. However, data management and processing face many challenges, such as data silos, data privacy protection, and data tampering. Traditional data processing methods have limitations. Blockchain technology, as an emerging distributed ledger technology, provides a blockchain-based data collaborative processing method due to its decentralized, tamper-proof and transparent characteristics. Through blockchain technology, data sharing and collaborative processing can be achieved across organizations, ensuring the authenticity and consistency of data, and effectively solving the security and privacy issues in traditional data collaborative processing. With the continuous evolution of blockchain technology, blockchain-based data collaborative processing methods will be widely used, providing reliable and secure technical support for the development of the digital economy.

[0003] Existing data collaborative processing methods have at least the following technical problems: single-point processing methods result in concentrated consumption of computing resources and single-point bottleneck problems; data is processed according to a simple order or fixed priority rule, lacking a dynamic adjustment mechanism, resulting in low data processing efficiency, especially when processing complex data fragments, which is prone to unreasonable resource allocation problems. Due to the centralization of data processing nodes and the untraceability of the data processing process, the integrity and credibility of the data are difficult to be fully guaranteed; when faced with large-scale data or extreme data, it is easily affected by extreme values, resulting in distorted processing results; in the blockchain environment, traditional verification methods show insufficient robustness when faced with complex and unstable data fragments, resulting in inconsistency and unreliability in verification results. Summary of the invention

[0004] The present invention provides a data collaborative processing method based on blockchain to solve the centralized consumption of computing resources and single-point bottlenecks caused by the existing single-point processing mode; processing is performed according to a simple order of data or a fixed priority rule, lacking a dynamic adjustment mechanism, resulting in low data processing efficiency, especially when processing complex data fragments, which is prone to unreasonable resource allocation; due to the centralization of data processing nodes and the untraceability of the data processing process, the integrity and credibility of the data are difficult to be fully guaranteed; when faced with large-scale data or extreme data, it is easily affected by extreme values, resulting in distorted processing results; in the blockchain environment, traditional verification methods show insufficient robustness when faced with complex and unstable data fragments, resulting in technical problems such as inconsistency and unreliability of verification results.

[0005] The present invention provides a data collaborative processing method based on blockchain, which specifically includes the following technical solutions:

[0006] A data collaborative processing method based on blockchain, comprising the following steps:

[0007] S1: Obtain the original data set, perform data sharding on the original data set to obtain different data fragments, and integrate them into a data fragment set. Based on the data of different data fragments, use the equity-weighted dynamic priority algorithm to calculate the processing priority of the data fragments;

[0008] S2: According to the processing priority of the data fragments, the data fragments are sent to different nodes in the blockchain network. The nodes run the built-in data processing logic of the smart contract to process the data and obtain the processing results of the data in the data fragments;

[0009] S3: Based on the processing result of the data in the data segment, use the built-in data verification algorithm of the smart contract to verify and obtain the data integrity verification score in the data segment. Compare the data integrity verification score in the data segment with the verification threshold to determine whether the processing result of the data in the data segment passes the verification;

[0010] S4: Based on the processing results of the data in the verified data fragments, use the data reassembly algorithm to obtain the assembled data set.

[0011] Preferably, the S1 specifically includes:

[0012] Through data sharding, the original data set is divided into data fragments; different data fragments are stored and processed in different nodes of the blockchain network.

[0013] Preferably, the S1 specifically includes:

[0014] The equity-weighted dynamic priority algorithm is introduced to optimize the blockchain data processing order; the equity-weighted dynamic priority algorithm calculates the processing priority of each data fragment by combining the calculation of data fragment characteristics, the influence of node equity weights and the adjustment of data fragment entropy values.

[0015] Preferably, the S1 specifically includes:

[0016] Data fragment characteristic calculation: for each data in the data fragment, the position-related weight term is introduced and corrected, the processing results of each data are summed and divided by the number of data in the data fragment, and a logarithmic function is introduced to compress the complexity of the data fragment; the influence of node equity weight, the node equity of the equity proof mechanism in the blockchain is used as the weight; the entropy value of the data fragment is adjusted by performing square root processing on the entropy value of the data in the data fragment to smooth the influence of the entropy value.

[0017] Preferably, the S2 specifically includes:

[0018] The built-in data processing logic of the smart contract calculates the processing results of the data in the data segment through logarithmic transformation, hyperbolic tangent transformation, nonlinear weight adjustment and exponential decay smoothing.

[0019] Preferably, the S3 specifically includes:

[0020] In the process of verification using the built-in data verification algorithm of the smart contract, the verification is performed in parallel on each node in the blockchain network. By introducing cubic roots, logarithmic functions, exponential functions, and performing square operations and arccosine calculations on the data in the data fragment, the data integrity verification score in the data fragment is obtained.

[0021] Preferably, the S4 specifically includes:

[0022] The data reassembly algorithm utilizes the decentralization and immutability of blockchain technology, and assembles the processing results of data in data fragments by introducing weighted mechanisms and non-linear mapping.

[0023] Preferably, the S4 specifically includes:

[0024] The weighted mechanism dynamically adjusts the weight value of the data processing result in each data fragment by combining the processing result of the data in each data fragment with the dynamic adjustment parameter as the weight adjustment factor; the nonlinear mapping distinguishes the influence of the processing results of data fragments at different positions in the assembly process by processing the sequential numbering of the data fragments on the blockchain.

[0025] The beneficial effects of the technical solution of the present invention are:

[0026] 1. By sharding the original data set, each data fragment is independently stored and processed on different nodes of the blockchain, which realizes the decentralization and parallel processing of data, speeds up data processing, makes full use of the computing resources of each node in the blockchain network, and effectively avoids single-point overload;

[0027] 2. By using the equity-weighted dynamic priority algorithm, combined with the calculation of data fragment characteristics, the influence of node equity weights and the adjustment of data fragment entropy values, the processing priority of each data fragment is dynamically determined, the order of blockchain data processing is optimized, and important and complex data fragments are guaranteed to obtain processing resources first, thereby optimizing data processing efficiency and data security under limited resources, and ensuring the fairness and efficiency of data processing;

[0028] 3. The built-in data processing logic of smart contracts not only optimizes the processing efficiency of data in data fragments, but also ensures the integrity and credibility of the data;

[0029] 4. Use the built-in data verification algorithm of the smart contract to perform square operations, cubic root operations, and arccosine calculations on the data to smooth the extreme effects of data changes, reduce the impact of data noise, improve the robustness and stability of the verification process, ensure the accuracy and reliability of data processing results, and process in parallel on each node of the blockchain network, further improving the transparency and traceability of data collaborative processing;

[0030] 5. After the processing results of the data in the data fragments are verified, the data reassembly algorithm is used to assemble the processing results of the data in the verified data fragments, ensuring that the assembled data set is reasonable and complete, making full use of the decentralization and immutability of blockchain technology, and further improving the effectiveness and reliability of data collaborative processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of a blockchain-based data collaborative processing method described in the present invention. DETAILED DESCRIPTION

[0032] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0034] The following is a detailed description of a specific scheme of a blockchain-based data collaborative processing method provided by the present invention in conjunction with the accompanying drawings.

[0035] Refer to the attached Figure 1 , which shows a flowchart of a data collaborative processing method based on blockchain provided by an embodiment of the present invention, the method comprising the following steps:

[0036] S1. Obtain the original data set, perform data sharding on the original data set to obtain different data fragments, and integrate them into a data fragment set. Based on the data of different data fragments, use the equity-weighted dynamic priority algorithm to calculate the processing priority of the data fragments;

[0037] Get the original data set D = {d 1 , d 2 ,...d n}, and perform data sharding on the original data set to obtain different data fragments, which are then integrated into a data fragment set S = {S 1 , S 2 , ..., S k}, where k≤n; n represents the number of elements in the original data set; k represents the total number of data fragments;

[0038] The data sharding process is to divide the original data set into multiple independent data fragments to achieve parallel data processing, which can not only speed up data processing, but also make full use of the computing resources of each node in the blockchain network to avoid single point overload;

[0039] The data sharding processing formula is as follows:

[0040] S i ={d i，j |j∈[1,m i ]}

[0041] Among them, S i represents the i-th data fragment; d i，j represents the jth data of the i-th data segment; m i represents the number of data in the i-th data segment and satisfies

[0042] Different data fragments are independently stored and processed on different nodes of the blockchain network, ensuring the decentralization and parallel processing of data, and improving the efficiency of data collaborative processing;

[0043] Based on the data of different data fragments, the processing priority of the data fragments is calculated using the equity-weighted dynamic priority algorithm;

[0044] The equity-weighted dynamic priority algorithm optimizes the blockchain data processing order and dynamically determines the processing priority of each data fragment by combining the calculation of data fragment characteristics, the influence of node equity weights, and the adjustment of data fragment entropy values;

[0045] The data segment characteristic calculation, for each data in the data segment, by introducing a position-related weight item processed by a sine function, emphasizes the difference in contribution of data at different positions, and corrects it through nonlinear transformation, mapping the data to between (0, 1); sums the processing results of each data and divides it by the number of data in the data segment to represent the overall complexity and importance of the data segment; further, compresses the complexity of the data segment through a logarithmic function to prevent imbalance caused by extremely large priority values; the impact of the node equity weight uses the node equity of the equity proof mechanism in the blockchain as the weight;

[0046] The adjustment of the entropy value of the data segment is to reflect the complexity and uncertainty of the data segment through the entropy value of the data, and the entropy value is processed in the form of square root to smooth the influence of the entropy value;

[0047] The calculation formula for the processing priority of data fragments is:

[0048]

[0049] Among them, P i Indicates the processing priority of the i-th data fragment; log 2 Represents a logarithmic function with base 2, which is used to compress the complexity of data fragments and prevent imbalance caused by extremely large priority values; Represents the summation symbol, which is used to sum all the data in the i-th data segment; d i，j represents the jth data of the i-th data segment; sin(j) represents the sine function, which is used to introduce periodic changes to prevent linear distribution of data; Represents the value of the jth data of the i-th data segment to normalize and ensure that the range is between 0 and 1; m i represents the number of data in the i-th data segment; W i represents the equity weight of the node where the i-th data fragment is located, that is, the influence of the node in the equity proof mechanism; H(S i ) represents the entropy value of the ith data segment, which is used to measure the complexity and uncertainty of the data segment. The method for calculating the entropy value is a technical means well known to those skilled in the art and will not be described in detail here; Represents the square root of the entropy value of the i-th data segment, which is used to smooth the impact of the entropy value;

[0050] S2. According to the processing priority of the data fragments, the data fragments are sent to different nodes in the blockchain network. The nodes run the built-in data processing logic of the smart contract to process the data and obtain the processing results of the data in the data fragments;

[0051] According to the processing priority of the data fragments, the data fragments are sent to different nodes in the blockchain network. The nodes run the pre-deployed smart contract built-in data processing logic to process the data and obtain the processing results of the data in the data fragments; the built-in data processing logic of the smart contract optimizes the data processing in the blockchain network, and enhances the flexibility and effectiveness of data processing by independently calculating and processing each data in the data fragment. The accuracy and robustness of data processing are ensured through the synergistic effect of logarithmic transformation, hyperbolic tangent transformation, nonlinear weight adjustment and exponential decay smoothing processing; logarithmic transformation is used to compress the range of data and reduce the impact of extreme values; hyperbolic tangent transformation can enhance the discrimination of data; nonlinear weight adjustment is further realized by calculating the sum of squares of each data as the denominator. The calculation of the sum of squares can emphasize the nonlinear characteristics of the data and balance the contribution of each data to the data processing results, thereby making the data processing results smooth and stable; the exponential decay function is applied to smooth the data processing results to ensure the continuity and stability of the data processing process;

[0052] The calculation formula for the processing result of the data in the data segment is:

[0053]

[0054] Among them, r i，j represents the processing result of the ,th data of the ith data segment; d i，j represents the jth data of the i-th data segment; log 2 (1+d i，j ) represents the logarithmic transformation of the ,th data of the ith data segment, which is used to scale the data and smooth the extreme values; tanh represents the hyperbolic tangent function, which is used to enhance the discrimination of the data; Expressed as the denominator, it provides nonlinear adjustment; represents the negative power of the exponential function, with the base e and the exponent Used to reduce the impact of data;

[0055] It realizes the distributed processing of data, fully utilizes the computing power of each node in the blockchain network, further improves the efficiency of data processing, ensures the integrity and credibility of data processing results, and improves the reliability and security of data collaborative processing;

[0056] S3. Based on the processing result of the data in the data segment, use the built-in data verification algorithm of the smart contract to verify and obtain the data integrity verification score in the data segment. Compare the data integrity verification score in the data segment with the verification threshold to determine whether the processing result of the data in the data segment passes the verification;

[0057] Based on the processing results of the data in the data fragment, the built-in data verification algorithm of the smart contract is used for verification. The verification is performed in parallel on each node in the blockchain network to obtain the data integrity verification score in the data fragment. The verification results, the processing results of the data in the data fragment and the data integrity verification score in the data fragment are recorded on the blockchain to ensure the transparency and traceability of the verification process.

[0058] The built-in data verification algorithm of the smart contract emphasizes the influence of extremely large values ​​in the data by performing a square operation on the data in the data fragment. The introduction of the cubic root operation is to smooth the extreme influence of data changes, making the verification process robust and stable; the arc cosine calculation is performed on the data in the data fragment to further smooth the data, reduce the influence of data noise, and make the verification process reliable; the logarithmic function is introduced to compress the range of processing results of the data in the data fragment, avoid the influence of extreme values, and ensure the stability of the verification process; the exponential function is introduced to attenuate the weight of the processing results of the data in the data fragment, balance the influence of data in different data fragments, and maintain the consistency and reliability of the data integrity verification score in the data fragment;

[0059] The calculation formula for the data integrity verification score in a data segment is:

[0060]

[0061] Among them, C i，j represents the jth data integrity verification score of the i-th data segment; m i Indicates the amount of data in the i-th data segment; d i，j Represents the jth data of the i-th data segment; express The cube root of is used to increase the nonlinear characteristics to enhance the complexity and robustness of the calibration process and reduce the impact of extreme values; represents the arccosine function, which can handle values ​​in the range [-1, 1], thereby further complicating the contribution of each data in the data segment; r i，j represents the processing result of the jth data of the i-th data segment; ln(1+r i，j ) indicates that the processing result of the data in the data segment is logarithmically transformed, which can effectively compress the value range and avoid the influence of extreme values; represents an exponential function used to adjust the weight of the data integrity verification score in the data segment to ensure that the verification process can adapt to the size of the data segment and the absolute value of the result;

[0062] In the smart contract of the blockchain, each data integrity verification score in the data fragment is compared with the predetermined verification threshold Q to determine whether the verification passes; if C i，j ≥Q, the processing result of the data in the data segment passes the verification and enters the data segment assembly process; if C i，j <Q, the processing result of the data in the data segment fails the verification, and triggers the reprocessing and verification process until the verification passes; the verification threshold can be set according to the specific implementation scenario and is not limited here;

[0063] By performing parallel verification on each node of the blockchain network, the transparency and traceability of data are guaranteed, which is a reflection of the security of the blockchain.

[0064] S4, based on the processing results of the data in the verified data fragments, use the data reassembly algorithm to obtain an assembled data set;

[0065] Based on the processing results of the data in the verified data fragments, the assembled data set is obtained by using the data reassembly algorithm;

[0066] The data reassembly algorithm utilizes the decentralization and immutability of blockchain technology, and realizes the assembly of the processing results of data in data fragments by introducing a weighting mechanism and non-linear mapping;

[0067] The weighting mechanism combines the processing result of the data in each data segment with the dynamic adjustment parameter as a weight adjustment factor, and dynamically adjusts the weight value of the processing result of the data in each data segment;

[0068] The nonlinear mapping takes the position of the data fragments into consideration. The sequential numbering of each data fragment on the blockchain not only indicates the order of appearance in the processing process, but also affects the relative influence in the assembly process. By processing the numbers, the processing results of the data in the data fragments at different positions can be reasonably distinguished in terms of influence, ensuring that the assembly of the processing results of the data in the data fragments is reasonable.

[0069] The assembled dataset formula is as follows:

[0070]

[0071] Where D′ represents the assembled dataset; Represents the summation symbol, which accumulates all data fragments; represents the summation symbol, indicating the summation of all data in the i-th data segment; ri，j Indicates the processing result of the jth data of the i-th data segment; represents the weight adjustment factor, which adjusts the weight of the data in the data segment during the assembly process; α represents the dynamic adjustment parameter, which is used to adjust the confidence of the weighted item, reflecting the credibility and importance of the data processing results in the data segment; Represents an exponential function, which enhances the weight adjustment effect of data in the data segment; Represents the cube root of the data fragment sequence number, introducing a nonlinear attenuation factor to balance the relative influence of data fragments in the assembly process;

[0072] It provides a dynamic assembly strategy based on the processing results of the data in the data fragments, which recombine the processing results of the data in the verified data fragments into a complete data set to ensure the security of data collaborative processing.

[0073] In summary, a blockchain-based data collaborative processing method has been completed.

[0074] The order of the embodiments of the invention is for description only and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0075] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0076] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A data collaborative processing method based on blockchain, characterized in that: The following steps are involved: S1: Obtain the original data set, perform data sharding on the original data set to obtain different data fragments, and integrate them into a data fragment set. Based on the data of different data fragments, use the equity-weighted dynamic priority algorithm to calculate the processing priority of the data fragments; S2: According to the processing priority of the data fragments, the data fragments are sent to different nodes in the blockchain network. The nodes run the built-in data processing logic of the smart contract to process the data and obtain the processing results of the data in the data fragments; S3: Based on the processing result of the data in the data segment, use the built-in data verification algorithm of the smart contract to verify and obtain the data integrity verification score in the data segment. Compare the data integrity verification score in the data segment with the verification threshold to determine whether the processing result of the data in the data segment passes the verification; S4: Based on the processing results of the data in the verified data fragments, use the data reassembly algorithm to obtain the assembled data set.

2. According to a blockchain-based data collaborative processing method according to claim 1, it is characterized in that: In said S1, it specifically includes: Through data sharding, the original data set is divided into data fragments; different data fragments are stored and processed in different nodes of the blockchain network.

3. According to a blockchain-based data collaborative processing method according to claim 1, it is characterized in that: In said S1, it specifically includes: The equity-weighted dynamic priority algorithm is introduced to optimize the blockchain data processing order; the equity-weighted dynamic priority algorithm calculates the processing priority of each data fragment by combining the calculation of data fragment characteristics, the influence of node equity weights and the adjustment of data fragment entropy values.

4. A data collaborative processing method based on blockchain according to claim 3, characterized in that: In said S1, it specifically includes: Data fragment characteristic calculation: for each data in the data fragment, the position-related weight term is introduced and corrected, the processing results of each data are summed and divided by the number of data in the data fragment, and a logarithmic function is introduced to compress the complexity of the data fragment; the influence of node equity weight, the node equity of the equity proof mechanism in the blockchain is used as the weight; the entropy value of the data fragment is adjusted by performing square root processing on the entropy value of the data in the data fragment to smooth the influence of the entropy value.

5. According to a blockchain-based data collaborative processing method according to claim 1, it is characterized in that: In said S2, it specifically includes: The built-in data processing logic of the smart contract calculates the processing results of the data in the data segment through logarithmic transformation, hyperbolic tangent transformation, nonlinear weight adjustment and exponential decay smoothing.

6. A data collaborative processing method based on blockchain according to claim 1, characterized in that: In said S3, it specifically includes: In the process of verification using the built-in data verification algorithm of the smart contract, the verification is performed in parallel on each node in the blockchain network. By introducing cubic roots, logarithmic functions, exponential functions, and performing square operations and arccosine calculations on the data in the data fragment, the data integrity verification score in the data fragment is obtained.

7. A data collaborative processing method based on blockchain according to claim 1, characterized in that: In said S4, it specifically includes: The data reassembly algorithm utilizes the decentralization and immutability of blockchain technology, and assembles the processing results of data in data fragments by introducing weighted mechanisms and non-linear mapping.

8. A blockchain-based data collaborative processing method according to claim 7, characterized in that: In said S4, it specifically includes: The weighted mechanism dynamically adjusts the weight value of the data processing result in each data fragment by combining the processing result of the data in each data fragment with the dynamic adjustment parameter as the weight adjustment factor; the nonlinear mapping distinguishes the influence of the processing results of data fragments at different positions in the assembly process by processing the sequential numbering of the data fragments on the blockchain.

Images