Block chain-based analysis processing system

Abstract

The invention discloses an analysis processing system based on a block chain. An operation method of the system comprises the steps of performing data set generation and sample management of big data information; performing management analysis on sample data query of the big data; performing optimization analysis management on query of the big data information; and carrying out record authentication processing on the query result data information of the big data. The data set generation and sample management of the big data information comprises the following steps: after identifying and retrieving the big data information, generating a random data set; by constructing a sample tree model, big data sample information is managed and controlled. The step of managing and analyzing the sample data query of the big data comprises the step of managing and analyzing the access query of the sample data information after the construction of the sample tree model is completed. The method has the characteristics of high accuracy and intelligent and efficient analysis.

Description

Technical Field

[0001] This invention relates to the field of analysis and processing technology, specifically to a blockchain-based analysis and processing system. Background Technology

[0002] With social development and technological advancements, data plays an increasingly important role in contemporary society. Data analysis can lead to new scientific discoveries and more accurate weather forecasts. The larger the amount of information and the more sophisticated the analytical tools, the more accurate the results, and the closer the actions taken based on those results are to success. However, the analysis and processing of big data presents numerous challenges due to its sheer volume. Traditional data analysis often involves scanning the entire dataset, calculating the user's query results, and then returning them. However, in many cases, users don't require absolutely accurate results; they may only need a general outline or a rough trend of the data. While fuzzy queries can be used for big data analysis, reducing user waiting time, this can lead to a significant decrease in data accuracy, failing to meet user needs and severely reducing user satisfaction. Therefore, designing a blockchain-based analysis and processing system that offers high accuracy and intelligent, efficient analysis is essential. Summary of the Invention

[0003] The purpose of this invention is to provide a blockchain-based analysis and processing system to solve the problems mentioned in the background section.

[0004] To address the aforementioned technical problems, this invention provides the following technical solution: a blockchain-based analysis and processing method, comprising: Generate and manage datasets and samples for big data information; Manage and analyze sample data queries from big data; Optimize, analyze, and manage the querying of big data information; Record and authenticate the query results of big data.

[0005] According to the above technical solution, the generation and sample management of big data information datasets includes: After identifying and retrieving big data information, a random dataset is generated. By constructing a sample tree model, we can manage and control big data sample information.

[0006] According to the above technical solution, the management and analysis of sample data queries for big data includes: After the sample tree model is constructed, the access and query of sample data information are managed and analyzed.

[0007] According to the above technical solution, the optimization analysis and management of big data information queries includes: Based on the query requirements, the splitting of leaf nodes in the big data sample management tree is controlled and managed. After splitting the leaf nodes of the sample tree, perform hierarchical transformation of the sample tree according to the next sample data query requirements. Optimize the management of the storage of leaf node data information in the sample tree.

[0008] According to the above technical solution, the recording and authentication processing of query results data information for big data includes: The query results are processed through blockchain consensus authentication to effectively prevent data from being tampered with or lost. When the requirement is to extract all query result data, user authentication must be completed before the query result data can be extracted and downloaded.

[0009] According to the above technical solution, a blockchain-based analysis and processing system includes: The data management module is used to manage and analyze sample data queries from large datasets. The analysis and processing module is used for optimizing and managing the query of big data information; The record authentication module is used to record and authenticate query results of big data.

[0010] According to the above technical solution, the data management module includes: The dataset generation module is used for generating, analyzing, and processing datasets. The sample management module is used to manage and control sample data information; The data query module is used for querying and processing data information.

[0011] According to the above technical solution, the analysis and processing module includes: The node splitting module is used to optimize and manage node splitting. The hierarchy transformation module is used to perform hierarchy transformation of the sample tree; The storage management module is used for storing and managing data information.

[0012] According to the above technical solution, the record authentication module includes: The consensus recording module is used for consensus recording and management of data information; The authentication extraction module is used to perform authentication control on the extraction of data information.

[0013] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: By setting up a data management module, an analysis and processing module, and a record authentication module, this invention can achieve random sampling through sequential scanning, ensuring that the collected big data samples are randomly distributed. This results in a larger number of samples, smaller confidence intervals, and higher confidence levels, effectively improving the efficiency and accuracy of big data information analysis and processing, effectively meeting users' big data query needs, reducing users' waiting time, and effectively improving the accuracy of big data query results. Attached Figure Description

[0014] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a blockchain-based analysis and processing method provided in Embodiment 1 of the present invention; Figure 2 This is a module configuration diagram of a blockchain-based analysis and processing system provided in Embodiment 2 of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Example 1: Figure 1 This is a flowchart illustrating a blockchain-based analysis and processing method provided in Embodiment 1 of the present invention. This embodiment can be applied to a data analysis and processing system. The method can be executed by a blockchain-based analysis and processing system provided in this embodiment, which consists of multiple software and hardware modules, such as... Figure 1 As shown, the method specifically includes the following steps: S101. Generate and manage datasets and samples for big data information; For example, in this embodiment of the invention, after identifying and retrieving big data information, a random dataset is generated. First, the big data file information is scanned sequentially. When scanning each tuple in the big data file, the tuple is placed in a randomly selected position in the random dataset to obtain a random dataset. When multiple big data information tables are involved in the generation process, one table is selected as the outer table and another as the inner table. The outer table is randomized in a single-table manner. Then, the random dataset corresponding to the outer table is scanned sequentially. For each tuple scanned from the random dataset of the outer table, the index of the inner table is searched, and the corresponding tuple is joined to obtain the random dataset after joining the two tables. Through this process, random sampling can be achieved through sequential scanning, ensuring that the collected big data samples are randomly distributed, resulting in a larger number of samples, smaller confidence intervals, and higher confidence levels, effectively improving the efficiency and accuracy of big data information analysis and processing.

[0017] By constructing a sample tree model, big data sample information is managed and controlled. In this step, the initial number of nodes in the sample tree is set to 1, so that the query operation of sample data information can utilize the results generated by previous queries. A dataset is divided into multiple subsets according to the set dimension values, and the constructed sample tree saves the online aggregated samples and intermediate results. As the query proceeds, more and more samples and intermediate results can be obtained, and the sample tree model will also process the corresponding split nodes, making subsequent sample data information queries more efficient.

[0018] S102. Manage and analyze sample data queries for big data; For example, in this embodiment of the invention, after the sample tree model is constructed, the access query of sample data information is managed and analyzed. In this step, when performing a query operation in the sample management tree, the focus node is initially the root node, and it will change with the query. During the query process, if the query node is the focus node, the query can be processed in the subtree with the focus node as the root node. When the query node is not the focus node, the statistical estimates of the corresponding dimension are re-merged according to the node queue information obtained from the previous query to obtain the initial aggregation result. If the initial aggregation result meets the confidence level and confidence interval set by the user, the query result is returned. Otherwise, the data is scanned in the data source until the user's needs are met.

[0019] S103. Optimize and manage the query of big data information; For example, in this embodiment of the invention, the leaf node splitting of the big data sample management tree is controlled and managed according to the query demand information. In this step, based on the number of real-time user query demand information, all samples stored in the leaf node are scanned, and then the values ​​corresponding to the splitting dimensions of the samples are checked. A child node is created for each value, and the sample is stored in the corresponding child node. After adding the node to be split to a queue, a depth-first traversal is performed on the queue to retrieve the first node in the queue. If the node is a leaf node, it is split according to the set dimensions. If the node has already been split, it is added to the result queue. Otherwise, all child nodes of the node are added to the leaf node to complete the splitting. After the splitting is completed, all sample information stored in the node is sinking to the child nodes. Through this process, the user's big data query needs can be effectively met, the user's waiting time can be reduced, and the accuracy of big data query results can be effectively improved.

[0020] After splitting the leaf nodes of the sample tree, the hierarchical transformation of the sample tree is performed according to the next sample data query requirement. In this step, when a non-leaf node is split according to a discrete dimension, upon receiving the next big data information query request, a corresponding statistical estimate is created for each value that can be obtained from the split dimension information. Then, the corresponding child node for each value is located for each node in the queue, and the statistical estimate of that node is obtained. The above steps are repeated to complete the value acquisition and statistical estimation of all child nodes in the queue. After these statistical estimates are put into the queue, the hierarchical transformation of the sample tree is completed, and it awaits the next query request. Through this step, the processing of big data query requests can be made more efficient and accurate, reducing the occurrence of processing errors and greatly improving user satisfaction.

[0021] The storage of leaf node data information in the sample tree is optimized and managed. In this step, after constructing storage space for the sample cache, the reference to this storage space is stored in the leaf node. When the leaf node needs to access the sample it stores, the node will locate the storage space of the sample through this reference. Through this process, when all the samples in a leaf node A need to be stored in another leaf node B, it is only necessary to set the reference pointing to the sample storage in leaf node B to the reference in leaf node A, and then clear the corresponding reference in leaf node A, which effectively saves the time required for data storage and processing.

[0022] S104. Record and authenticate the query results of big data. For example, in this embodiment of the invention, the query result data information is processed by blockchain consensus authentication to effectively prevent the data from being tampered with or lost. When retrieving query results, user authentication is required before the results can be downloaded. This step significantly enhances the security of the results and effectively prevents the leakage of private information.

[0023] Example 2: Example 2 of the present invention provides a blockchain-based analysis and processing system. Figure 2 This is a schematic diagram of the module structure of a blockchain-based analysis and processing system provided in Embodiment 2, as shown below. Figure 2 As shown, the system includes: The data management module is used to manage and analyze sample data queries from large datasets. The analysis and processing module is used for optimizing and managing the query of big data information; The record authentication module is used to record and authenticate query results of big data.

[0024] In some embodiments of the present invention, the data management module includes: The dataset generation module is used for generating, analyzing, and processing datasets. The sample management module is used to manage and control sample data information; The data query module is used for querying and processing data information.

[0025] In some embodiments of the present invention, the analysis and processing module includes: The node splitting module is used to optimize and manage node splitting. The hierarchy transformation module is used to perform hierarchy transformation of the sample tree; The storage management module is used for storing and managing data information.

[0026] In some embodiments of the present invention, the authentication recording module includes: The consensus recording module is used for consensus recording and management of data information; The authentication extraction module is used to perform authentication control on the extraction of data information.

[0027] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0028] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A blockchain-based analysis and processing method, characterized in that: include: Generate and manage datasets and samples for big data information; Manage and analyze sample data queries from big data; Optimize, analyze, and manage the querying of big data information; Record and authenticate the query results of big data.

2. The blockchain-based analysis and processing method according to claim 1, characterized in that: The generation and sample management of big data information datasets includes: After identifying and retrieving big data information, a random dataset is generated. By constructing a sample tree model, we can manage and control big data sample information.

3. The blockchain-based analysis and processing method according to claim 1, characterized in that: The management and analysis of sample data queries for big data includes: After the sample tree model is constructed, the access and query of sample data information are managed and analyzed.

4. The blockchain-based analysis and processing method according to claim 1, characterized in that: The optimization, analysis, and management of big data information queries include: Based on the query requirements, the splitting of leaf nodes in the big data sample management tree is controlled and managed. After splitting the leaf nodes of the sample tree, perform hierarchical transformation of the sample tree according to the next sample data query requirements. Optimize the management of the storage of leaf node data information in the sample tree.

5. The blockchain-based analysis and processing method according to claim 1, characterized in that: The recording and authentication processing of query results data from big data includes: The query results are processed through blockchain consensus authentication to effectively prevent data from being tampered with or lost. When the requirement is to extract all query result data, user authentication must be completed before the query result data can be extracted and downloaded.

6. A blockchain-based analysis and processing system, characterized in that: include: The data management module is used to manage and analyze sample data queries from large datasets. The analysis and processing module is used for optimizing and managing the query of big data information; The record authentication module is used to record and authenticate query results of big data.

7. The blockchain-based analysis and processing system according to claim 6, characterized in that: The data management module includes: The dataset generation module is used for generating, analyzing, and processing datasets. The sample management module is used to manage and control sample data information; The data query module is used for querying and processing data information.

8. The blockchain-based analysis and processing system according to claim 6, characterized in that: The analysis and processing module includes: The node splitting module is used to optimize and manage node splitting. The hierarchy transformation module is used to perform hierarchy transformation of the sample tree; The storage management module is used for storing and managing data information.

9. A blockchain-based analysis and processing system according to claim 6, characterized in that: The record authentication module includes: The consensus recording module is used for consensus recording and management of data information; The authentication extraction module is used to perform authentication control on the extraction of data information.

Images