Visual block chain data storage method and system

By deploying lightweight proxy nodes and multi-threaded acquisition, designing a hierarchical storage architecture and abstract data model in the blockchain, the storage and visualization problems caused by the increase in blockchain data volume are solved, and efficient data processing and multi-dimensional data are achieved Hierarchical visual display improves user experience and data analysis capabilities.

CN119046382AActive Publication Date: 2024-11-29SHENZHEN YIHANG NETWORK INFORMATION TECH CO LTD

Patent Information

Application Number
CN202411523658.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-30
Publication Date
2024-11-29
Estimated Expiration
2044-10-30

AI Technical Summary

Technical Problem

The immutable and long-term storage characteristics of blockchain data lead to an increase in data volume, leading to an increase in storage costs, a decrease in the degree of node decentralization, a decrease in data access and processing efficiency, an increase in network transmission burden, and the multi-blockchain data format and structure are different. , it is difficult to achieve unified visualization.

Method used

Deploy lightweight proxy nodes in blockchain, use multi-threaded data to collect data in parallel, design a hierarchical storage architecture and abstract data model, map data from different blockchains to a unified framework, perform aggregation processing and three-dimensional graph construction, and provide Multi-level visual interface.

Benefits of technology

It improves data acquisition and storage efficiency, reduces storage costs, enhances system compatibility and scalability, simplifies data display, improves user experience and data analysis accuracy, and reduces information overload.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119046382A_ABST
    Figure CN119046382A_ABST
Patent Text Reader

Abstract

The invention discloses a visual block chain data storage method and system, and relates to the field of data storage processing, and the method and system are characterized in that a data collection module is used for deploying a lightweight proxy node in each block chain, and collecting original data on different nodes in parallel by using multiple threads; the data storage module uses an automatic storage capacity expansion mechanism to design a hierarchical storage architecture, establishes an abstract data model, maps original data of different block chains into a unified framework, and obtains original data with a unified format; the data integration processing module performs aggregation processing on the stored original data to obtain aggregation groups, analyzes relevance among different aggregation groups, and constructs a data relation three-dimensional map; and the visual display module is used for enabling a user to interact with the data stored in the block chain, taking the data relationship three-dimensional graph as a bottom-layer architecture, increasing multi-level visualization of the data by the user, providing a user-friendly interface, and realizing visual storage of the block chain data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data storage processing, and more specifically, to a visual blockchain data storage method and system. Background Art

[0002] The data storage of blockchain is tamper-proof and long-term. As the blockchain network continues to grow, the amount of blockchain data will continue to increase, further increasing the complexity of data on the chain and storage requirements. Especially for multi-blockchain data, each blockchain has a different design and implementation method, and the data format, structure and protocol between blockchains may be different, which increases the difficulty of data processing. In addition, blockchain data contains a variety of complex structured information. The correlation between these data is complex, and direct presentation may confuse users. The design complexity of visualization tools increases.

[0003] As the amount of blockchain data increases, especially for full nodes, the storage cost will increase significantly, resulting in a decrease in the degree of node decentralization. In the visualization process, efficient data query and retrieval technology is required to process large-scale data. As the amount of data increases, the efficiency of data access and processing gradually decreases, resulting in slower response speed and affecting user experience. Blockchain data needs to be constantly synchronized between network nodes. The substantial increase in data volume will increase the burden on network transmission, create bandwidth bottlenecks, and increase latency. The visualization of massive data requires consideration of how to simplify the display to avoid information overload.

[0004] Therefore, based on the above problems, it is necessary to design visual blockchain data storage. Summary of the invention

[0005] In view of the problems existing in the prior art, the purpose of the present invention is to provide a visual blockchain data storage method and system to realize the visual storage of blockchain data.

[0006] To achieve the above object, the present invention provides the following technical solution: the visual blockchain data storage system includes: Data collection module: used to deploy lightweight proxy nodes in each blockchain and use multi-threading to collect raw data on different nodes in parallel; Data storage module: Use the automated storage expansion mechanism to design a layered storage architecture, establish an abstract data model, map the raw data of different blockchains into a unified framework, and obtain the raw data in a unified format; Data integration processing module: aggregates the stored raw data, obtains aggregate groups, analyzes the correlation between different aggregate groups, and constructs a three-dimensional map of data relationships; Visualization display module: It is used to enable users to interact with the data stored in the blockchain. It uses a three-dimensional map of data relationships as the underlying architecture, increases users' multi-level visualization of data, and provides a user-friendly interface.

[0007] Preferably, the types of the original data include transaction records, smart contracts, block header information, and node status; Methods for using multithreading to collect raw data on different nodes in parallel include: Select a lightweight proxy node and deploy it to the blockchain; Select multi-threading technology, where each thread is responsible for fetching the latest raw data from a node; Use a short-term buffering mechanism for continuous acquisition of raw data.

[0008] Preferably, the tiered storage architecture includes a hot storage layer, a cold storage layer and an archive storage layer, the hot storage layer is used to store active data, the cold storage layer is used to store data with low access frequency, and the archive storage layer is used to store data that is no longer accessed for a long time.

[0009] Preferably, the method for designing a hierarchical storage architecture using an automated storage expansion mechanism includes: Use monitoring tools to consolidate the health status of each storage layer , Capacity utilization , Throughput and delay ; Calculate the evaluation index of each storage layer , taking the current time point t as the time base point, calculate the evaluation index at each time point in turn, and obtain the absolute difference between the evaluation index of each time point and the current time point, until the absolute difference of the evaluation index is greater than the absolute threshold, collect the evaluation index of all interval time points between this time point and the time base point, and combine the evaluation index of the time base point to form a p-order data group, where p represents the number of time points extracted from the time base point forward; in, and represents the weight coefficient; Select d time points forward from the current time t as the time base points, obtain d p-order data groups, and train the fitting model , set the stopping condition of the fitting model to be that the absolute difference between the output values ​​of two consecutive iterations in o consecutive iterations is less than the fitting threshold; in, Indicates the last time point The random error term at Indicates The fitting coefficients, Indicates time point The evaluation index at represents the adjustment factor; The evaluation index at the current time point t Input into the trained fitting model to obtain the Value, when When the value is greater than Q1, use horizontal expansion to increase the storage nodes of the hot storage layer. When the value is greater than Q2, use automated data lifecycle management tools to migrate data in the cold storage layer to the hot storage layer or archive storage layer. When the value is greater than Q3, use cloud services to add storage nodes to the archive storage layer; Among them, Q1, Q2 and Q3 are preset constants.

[0010] Preferably, the abstract data model is established to map the original data of different blockchains into a unified framework. The specific method includes: Set the properties of the block including block hash H_b, parent block hash H_pb, block height Q_h, timestamp t_b and transaction record T_b, and use smart contract Z_h, block header information Q_t, and node status J_z as abstract definitions, where the node status is arranged in the order of links in the block; Defining an abstract data model includes two layers of mapping, namely attribute mapping and abstract definition mapping. The attribute mapping layer is , the abstract definition mapping layer is ; Map the raw data of different blockchains into a unified framework formed by an abstract data model, so that the raw data has a unified standard format.

[0011] Preferably, the method for performing aggregation processing on the original data includes: Step A1: Store the collected raw data in a hierarchical storage architecture, then remove irrelevant characters and unify the writing format of the raw data, build a vocabulary based on the set of unique words in the raw data, calculate the TF-IDF value of each word, use each TF-IDF value as a feature of the word, build a blank vector, correspond each blank element in the blank vector to the TF-IDF value of the word in the vocabulary, and then convert each data type in the raw data at the same time point into a feature vector representation; Step A2: Collect the feature vectors at each node on the blockchain, and for each data type, extract the maximum feature vector and minimum , the feature vector of each data type is divided into K_q blocks to obtain the magnetic center ,in, Indicates The magnetic center of the block, Indicates the index of the block; Step A3: Calculate the distance between each eigenvector and the magnetic center ,in, Represents the feature vector With magnetic center The distance Indicates feature vectors, represents the number of eigenvectors; Step A4: Assign each feature vector to the block to which the nearest magnetic center belongs, extract the number of feature vectors in each block, and if the number is less than the evaluation threshold, dissolve the block, and refill the dissolve feature vectors into the blocks to which other magnetic centers belong, thereby adaptively adjusting the preset K_q blocks; Step A5: After completing the distribution of feature vectors, update each magnetic center to the mean of all feature vectors in its block, repeat steps A3 to A4 until the preset number of distributions is reached, and stop, treating each block as an aggregation group.

[0012] Preferably, the method for constructing the three-dimensional graph of data relationships includes: Construct a blank 3D layer, label the blockchain nodes horizontally, the data types vertically, and the time points vertically; Extract the nodes, data types and time points of each aggregation group, and simulate the aggregation group into solid points to fill in the blank three-dimensional layer to form point filling; Define the solid points to be connected by directed edges, set attribute values ​​on the directed edges, and set the attribute values ​​according to the formula ,in, and Used to represent the weight coefficient, Indicates the aggregation group at the same time point and Aggregation Group The number of times it appears at the same time, Indicates the aggregation group at a time point The number of occurrences, Indicates the aggregation group at a time point The number of occurrences, Indicates the aggregation group and Aggregation Group Overlapping content, Indicates the aggregation group and Aggregation Group All combined contents; Use directed edges to connect any two aggregation groups, calculate the attribute values ​​of the directed edges, and add the attribute values ​​to the directed edges to form a complete three-dimensional graph of data relationships.

[0013] Preferably, the method for the user to interact with the data stored in the blockchain includes: Use React.js or Vue.js front-end frameworks to build responsive user interfaces and use visualization libraries to create interactive displays; Integrated dashboard tools to customize data display and monitoring.

[0014] Preferably, the three-dimensional graph of data relationships is used as the underlying architecture to increase the multi-level visualization of data for users, and the specific method includes: Set up multi-level data display. The macro view is used to display the overall status of the blockchain network; the meso view is used to display detailed information of a block or a node; and the micro view is used to display specific details of a smart contract. With the data relationship three-dimensional graph as the underlying architecture, the Neo4j graph database and graph visualization tools are used to display the correlation between the primary color data on each node in the data relationship three-dimensional graph. By clicking on a block or node, you can view the relevant information layer by layer, and use the breadcrumb navigation to quickly switch between different levels.

[0015] A visual blockchain data storage method includes: Step G1: Use multi-threading to collect raw data on different nodes in parallel; Step G2: Build a tiered storage architecture, including a hot storage layer, a cold storage layer, and an archive storage layer, and use an automated storage expansion mechanism to design the tiered storage architecture; Step G3: Establish an abstract data model, map the raw data of different blockchains into a unified framework, and obtain the raw data in a unified format; Step G4: Aggregate the stored raw data to obtain aggregation groups, build a blank three-dimensional layer, and analyze the correlation between different aggregation groups to obtain a three-dimensional map of data relationships; Step G5: View the storage of blockchain data in real time through visual display and user-friendly interface.

[0016] Compared with the prior art, the present invention has the following beneficial effects: By deploying lightweight proxy nodes in the blockchain and using multi-threading technology to collect raw data from different nodes in parallel, the speed and efficiency of data collection can be greatly improved, especially when multiple nodes generate a large amount of data at the same time, solving the problem of insufficient performance of traditional single-threaded data collection methods. Supporting short-term caching mechanism ensures the continuity and stability of data collection, further improving the reliability of the system.

[0017] The layered architecture of hot storage, cold storage and archive storage can allocate data to different storage layers according to the activity and access frequency of the data. The automatic storage expansion mechanism dynamically adjusts the allocation of storage resources by monitoring the health status, capacity utilization, throughput and latency of the storage layer to ensure the efficiency and stability of system operation.

[0018] By building an abstract data model, the original data of different blockchains are mapped into a unified standard framework, so that data from different blockchains can be processed in a unified format. This abstract processing method makes the system have better compatibility and scalability, and can adapt to the original data types of different blockchains. Unified processing through abstract models reduces the complexity of processing heterogeneous blockchain data.

[0019] By aggregating the raw data, the system can extract valuable aggregation groups from a large amount of raw data and analyze the correlation between different aggregation groups. This aggregation process can not only reduce the amount of data, but also improve the accuracy of data analysis. The calculation and adaptive adjustment mechanism of the magnetic center introduced in the aggregation process can effectively optimize the allocation and adjustment of the aggregation group and ensure the accuracy and rationality of the aggregation results.

[0020] The constructed three-dimensional graph of data relationships allows users to intuitively view the correlation between different nodes, data types and time points. The construction of the three-dimensional graph helps users understand complex data relationships, especially in the display of on-chain data correlation. Using directed edges to connect different aggregation groups and displaying the relationship between different aggregation groups based on calculated attribute values ​​makes data analysis more intuitive. At the same time, the display of the three-dimensional graph can also help discover potential data correlations and patterns.

[0021] The system uses React.js or Vue.js front-end frameworks to provide users with a good interactive experience. A multi-level visual display method is set up, and users can go into different levels to view detailed data as needed, achieving a comprehensive display from the overall link status to the details of a single node or contract. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A structural schematic diagram of a visual blockchain data storage system is proposed for the present invention; Figure 2 Schematic diagram of the method applied to the visual blockchain data storage system in the present invention. DETAILED DESCRIPTION

[0023] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0024] Embodiment 1 Reference Figure 1 and Figure 2 , Example 1 further illustrates a visual blockchain data storage system proposed in the present invention.

[0025] There are some challenges and problems in the visual storage of blockchain data, which are analyzed as follows: 1. Huge amount of data The data storage of blockchain is tamper-proof and long-term. As the blockchain network continues to grow, the amount of data on the blockchain will continue to increase, especially on public chains (such as Bitcoin and Ethereum). The total amount of data on the blockchain is very large and difficult to fully present or store. The design characteristics of the blockchain determine that its data volume will continue to grow over time. The specific reasons include the decentralization and redundancy of the blockchain, the continued growth of on-chain transactions, the popularity of smart contracts (blockchains such as Ethereum allow developers to deploy smart contracts, which further increases the complexity and storage requirements of on-chain data. Information such as state changes and event logs after the execution of smart contracts needs to be recorded) and the growth in the number of users and nodes (with the development of the blockchain ecosystem, the number of users and nodes continues to increase, and the transaction volume and data synchronization requirements are also expanding, resulting in a massive increase in data volume).

[0026] As the amount of blockchain data increases, especially for full nodes, the storage cost will increase significantly, resulting in a decrease in the degree of node decentralization. In the visualization process, efficient data query and retrieval technology is required to process large-scale data. As the amount of data increases, the efficiency of data access and processing gradually decreases, resulting in slower response speed and affecting user experience. Blockchain data needs to be constantly synchronized between network nodes. The substantial increase in data volume will increase the burden on network transmission, create bandwidth bottlenecks, and increase latency. The visualization of massive data requires consideration of how to simplify the display and avoid information overload, while also requiring reasonable abstraction and screening of large-scale data.

[0027] 2. Data Complexity Blockchain data contains a variety of complex structured information, such as transaction records, smart contracts, block header information, node status, etc. The correlation between these data is complex, and direct presentation may confuse users. Each block in the blockchain contains multiple transactions, and each transaction contains multiple parts such as transaction input, output, signature, etc., forming a multi-level and multi-dimensional data structure.

[0028] The design complexity of visualization tools has increased: In order to present the multi-level and multi-dimensional data of the blockchain and the complexity of the on-chain data structure, visualization tools need to design complex interactive interfaces and data processing logic, which increases the difficulty of development and system complexity. In addition, displaying too much complex information may cause users to be unable to understand key data or even cause confusion. Therefore, how to extract the most valuable information from complex data and present it concisely has become an important issue.

[0029] 3. Cross-chain data integration In a multi-chain environment, each blockchain is designed and implemented differently, and the data format, structure, and protocol between blockchains may be different. For example, the data structures of Bitcoin and Ethereum are very different. Therefore, how to integrate data from different blockchains in a unified visual interface is very complicated.

[0030] Cross-chain data integration requires handling the data structures of different blockchains and ensuring that visualization tools can seamlessly support multi-chain data, allowing users to gain a global perspective rather than being limited to the view of a single chain.

[0031] Therefore, how to effectively process and store large-scale blockchain data is a challenge. Visualization tools need to be able to process large amounts of historical data and ensure that performance is not affected when presenting visualizations. Based on the above problems, a visual blockchain data storage method and system are designed. The specific design contents are as follows; The visual blockchain data storage system includes: Data collection module: used to deploy lightweight proxy nodes in each blockchain and use multi-threading to collect raw data on different nodes in parallel; The types of raw data include transaction records, smart contracts, block header information, and node status; Transaction records refer to transaction data on each node, including the transaction initiator, recipient, transaction amount, transaction time, handling fee, etc. Smart contracts include contract creation, execution, call records, etc., and can extract key information such as contract code, call parameters, and return results; The block header information refers to the metadata of the block, including the block number, timestamp, Merkle tree root, previous block hash, difficulty value, random number (nonce), etc. Node status refers to the operation status, synchronization status, block progress, activity, etc. of nodes in the network; Methods for using multithreading to collect raw data on different nodes in parallel include: Select a lightweight proxy node and deploy it to the blockchain; select the appropriate light node (light client) according to the different blockchain networks; for example, for Ethereum, use geth--syncmode "light" to run the light node; for Bitcoin, a similar SPV mode can be used. Deploy lightweight nodes in a reliable server or cloud environment (such as AWS EC2, Alibaba Cloud, DigitalOcean, etc.) to ensure that the node can continue to run and maintain a connection with the blockchain network, monitor the health of the node, and ensure that the node status can be obtained in real time.

[0032] Choose multi-threading technology, where each thread is responsible for fetching the latest raw data from a node. For example, in Python, you can use concurrent.futures.ThreadPoolExecutor to manage multiple threads. Use a short-term buffering mechanism for continuous acquisition of raw data.

[0033] In order to optimize the efficiency of data collection and avoid repeatedly grabbing the same block or transaction data, a short-term cache mechanism can be used. When a block or transaction data is successfully collected, it can be temporarily cached for a period of time to quickly respond to subsequent requests.

[0034] For example, Redis can be used to implement short-term caching. Redis is a high-performance in-memory database that is suitable for short-term caching of blockchain data. The captured transaction records, smart contracts, block header information, and node status can be stored in Redis, and an expiration time (TTL) can be set to ensure that the cached data is only retained for a period of time.

[0035] Design a caching strategy: To ensure the uniqueness of the cache, set a unique cache key for each data in Redis. For example, block data can use block:<block_height> As a Redis key; transaction data can use tx:<transaction_hash> As a Redis key.

[0036] Set an expiration time TTL (such as 10 or 1 minute) for each cache item to ensure that the data in the cache does not occupy memory space for a long time and that old data can be automatically eliminated when the blockchain data is updated.

[0037] During data collection, if the data of the block already exists in Redis, the cached data is directly returned to avoid repeated crawling; if the data of the block does not exist in the cache, the data is pulled from the proxy node and cached in Redis.

[0038] A short-term cache mechanism is used to avoid repeated fetching of the same data and reduce network burden.

[0039] Data storage module: Use the automated storage expansion mechanism to design a layered storage architecture, establish an abstract data model, map the raw data of different blockchains into a unified framework, and obtain the raw data in a unified format; The tiered storage architecture includes a hot storage layer, a cold storage layer, and an archive storage layer. The hot storage layer is used to store active data with a high access frequency to ensure fast response. The cold storage layer is used to store data with a low access frequency, such as historical transactions and block data, which have a low access frequency but still require fast query. The archive storage layer is used to store data that is no longer accessed for a long time, such as block history data that exceeds a certain time.

[0040] Methods for designing a tiered storage architecture using an automated storage expansion mechanism include: Use monitoring tools (such as Prometheus or Zabbix) to integrate the health status of each storage layer , Capacity utilization , Throughput and delay ; Calculate the evaluation index of each storage layer , taking the current time point t as the time base point, calculate the evaluation index at each time point in turn, and obtain the absolute difference between the evaluation index of each time point and the current time point, until the absolute difference of the evaluation index is greater than the absolute threshold, collect the evaluation index of all interval time points between this time point and the time base point, and combine the evaluation index of the time base point to form a p-order data group, where p represents the number of time points extracted from the time base point forward; in, and Represents the weight coefficient; select d time points forward from the current time t as the time base point, obtain d p-order data groups, and train the fitting model , set the stopping condition of the fitting model to be that the absolute difference between the output values ​​of two consecutive iterations in o consecutive iterations is less than the fitting threshold; The absolute threshold and fitting threshold can be set based on experimental data analysis or experience.

[0041] in, Indicates the last time point The random error term at Indicates The fitting coefficients, Indicates time point The evaluation index at represents the adjustment factor; The evaluation index at the current time point t Input into the trained fitting model to obtain the Value, when When the value is greater than Q1, use horizontal expansion to increase the storage nodes of the hot storage layer. When the value is greater than Q2, use automated data lifecycle management tools (such as AWS S3 storage class conversion rules) to migrate data in the cold storage layer to the hot storage layer or archive storage layer. When the value is greater than Q3, use cloud services (such as AWS Glacier, Azure Archive Storage, etc.) to increase the storage nodes of the archive storage layer; Among them, Q1, Q2 and Q3 are preset constants.

[0042] When adding nodes, you can use 1 node as the growth step and add one node each time, or you can set the number of nodes to be added by yourself.

[0043] In order to build an abstract data model and map the raw data of different blockchains into a unified framework, it is necessary to define a common data structure and eliminate the differences between blockchains through a mapping method.

[0044] Establish an abstract data model to map the raw data of different blockchains into a unified framework. The specific methods include: Definition: Block is the basic unit of storage in blockchain. Each block is linked to the previous block to form a blockchain. The properties of the block are set, including block hash H_b, parent block hash H_pb, block height Q_h, timestamp t_b and transaction record T_b (transaction list). Smart contract Z_h, block header information Q_t, and node status J_z are used as abstract definitions. Among them, the node status is arranged in the order of links in the block. The node status can predefine the status category, such as normal idle, normal working, use warning, use failure, etc. Defining an abstract data model includes two layers of mapping, namely attribute mapping and abstract definition mapping. The attribute mapping layer is , the abstract definition mapping layer is ; Map the raw data of different blockchains into a unified framework formed by an abstract data model, so that the raw data has a unified standard format.

[0045] For example, the fields of a Bitcoin block (block attributes): block_hash, previous_block_hash, height, timestamp, tx[]; Mapping to the attribute map layer: (H_b)=block_hash; (H_pb)=previous_block_hash; (Q_h)=height; (t_b)=timestamp; (T_b) = tx[] (each item in the transaction record is a transaction); MX= block_hash,previous_block_hash,height,timestamp,tx[] .

[0046] Data integration processing module: aggregates the stored raw data, obtains aggregate groups, analyzes the correlation between different aggregate groups, and constructs a three-dimensional map of data relationships; Methods for aggregate processing of raw data include: Step A1: Store the collected raw data in a hierarchical storage architecture, and then remove irrelevant characters and unify the writing format of the raw data. Removing irrelevant characters means removing punctuation marks, special characters, etc. in the text for unified processing. Unifying the writing format means converting all texts to the same writing standard to ensure that the same words are not considered different due to different capitalization when compared; construct a vocabulary based on the set of unique words in the raw data, regard each word in the text as a feature, represent the text through these features, list all the words that appear in the text, and form a vocabulary. For example, suppose our vocabulary is ["quick", "brown", "fox", "jumps", "over", "lazy", "dog"], calculate the TF-IDF value of each word, use each TF-IDF value as a feature of the word, construct a blank vector, and correspond each blank element in the blank vector to the TF-IDF value of the word in the vocabulary. In this way, the text can be represented by a vector, and each data type in the original data at the same time point can be converted into a feature vector representation, and each paragraph of text can be represented as a vector. Each position of the vector corresponds to a word in the vocabulary, and the frequency of the word in the text represents the value of the position. The TF-IDF value of each word is the product of TF and IDF. The term frequency (TF) indicates the frequency of each word in the document. The formula is: , Inverse Document Frequency (IDF) is used to measure the rarity of a word in all documents, and the formula is , is the total number of documents.

[0047] Step A2: Collect the feature vectors at each node on the blockchain, and for each data type, extract the maximum feature vector and minimum , the feature vector of each data type is divided into K_q blocks to obtain the magnetic center ,in, Indicates The magnetic center of the block, Indicates the index of the block; Step A3: Calculate the distance between each eigenvector and the magnetic center ,in, Represents the feature vector With magnetic center The distance Indicates feature vectors, represents the number of eigenvectors; Step A4: Assign each feature vector to the block to which the nearest magnetic center belongs, extract the number of feature vectors in each block, and if the number is less than the evaluation threshold, dissolve the block, and refill the dissolve feature vectors into the blocks to which other magnetic centers belong, thereby adaptively adjusting the preset K_q blocks; Step A5: After completing the distribution of feature vectors, update each magnetic center to the mean of all feature vectors in its block, repeat steps A3 to A4 until the preset number of distributions is reached, and stop, treating each block as an aggregation group.

[0048] The number of allocations can also be regarded as the number of iterations. The preset number of iterations should be large enough to ensure that all eigenvectors can converge and group data with similar correlations together to facilitate subsequent data analysis between blocks.

[0049] The method for constructing a three-dimensional graph of data relations includes: Construct a blank 3D layer, label the blockchain nodes horizontally, the data types vertically, and the time points vertically; Extract the nodes, data types and time points of each aggregation group, and simulate the aggregation group into solid points to fill in the blank three-dimensional layer to form point filling; Define the solid points to be connected by directed edges, set attribute values ​​on the directed edges, and set the attribute values ​​according to the formula ,in, and Used to represent the weight coefficient, Indicates the aggregation group at the same time point and Aggregation Group The number of times it appears at the same time, Indicates the aggregation group at a time point The number of occurrences, Indicates the aggregation group at a time point The number of occurrences, Indicates the aggregation group and Aggregation Group Overlapping content, Indicates the aggregation group and Aggregation Group All combined contents; Use directed edges to connect any two aggregation groups, calculate the attribute values ​​of the directed edges, and add the attribute values ​​to the directed edges to form a complete three-dimensional graph of data relationships.

[0050] Building a knowledge graph between data sets is a complex data analysis task that extracts entities and their relationships from the data and organizes and displays them in the form of a graph.

[0051] Visualization display module: used to enable users to interact with the data stored in the blockchain, with a three-dimensional map of data relationships as the underlying architecture, to increase users' multi-level visualization of data and provide a user-friendly interface; Methods for users to interact with data stored in the blockchain include: Use React.js or Vue.js front-end frameworks to build responsive user interfaces and visualization libraries (D3.js, ECharts, or Three.js) to create interactive displays (e.g., charts, network diagrams, 3D views, etc.); Integrate dashboard tools (such as Grafana, Tableau, etc.) to customize data display and monitoring.

[0052] With the three-dimensional graph of data relationships as the underlying architecture, users can gain multi-level visualization of data. Specific methods include: Set up multi-level data display. The macro view is used to display the overall status of the blockchain network (such as the transaction volume of the entire network, block speed, node distribution, on-chain asset flow, etc.); the meso view is used to display detailed information of a block or a node (such as the number of transactions contained in the block, basic information of each transaction, sending and receiving addresses, etc.); the micro view is used to display the specific details of a smart contract (such as the input and output of the transaction, the parameters and results of the contract call); With the data relationship three-dimensional graph as the underlying architecture, the Neo4j graph database and graph visualization tools are used to display the correlation between the primary color data on each node in the data relationship three-dimensional graph. By clicking on a block or node, you can view the relevant information layer by layer, and use the breadcrumb navigation to quickly switch between different levels.

[0053] Integrate authentication mechanisms such as OAuth 2.0 and JWT to ensure the authenticity of user identity and the correctness of access rights; design a detailed role permission model so that different user roles have different data access rights; implement interactive operations such as data filtering, searching, dragging, and zooming to enhance user experience; support exporting visualized data to PDF, Excel and other formats to facilitate users' offline analysis and sharing.

[0054] Data privacy protection ensures that sensitive data is not accessed by unauthorized users; efficient permission management mechanism supports dynamic permission adjustment and auditing.

[0055] Embodiment 2 Reference Figure 1 and Figure 2 , Example 2 further illustrates a visual blockchain data storage system proposed by the present invention.

[0056] The visual blockchain data storage method comprises the following steps: Step G1: Use multi-threading to collect raw data on different nodes in parallel; Step G2: Build a tiered storage architecture, including a hot storage layer, a cold storage layer, and an archive storage layer, and use an automated storage expansion mechanism to design the tiered storage architecture; Step G3: Establish an abstract data model, map the raw data of different blockchains into a unified framework, and obtain the raw data in a unified format; Step G4: Aggregate the stored raw data to obtain aggregation groups, build a blank three-dimensional layer, and analyze the correlation between different aggregation groups to obtain a three-dimensional map of data relationships; Step G5: View the storage of blockchain data in real time through visual display and user-friendly interface.

[0057] The visual blockchain data storage system is applied to a visual blockchain data storage method and includes: Data collection module: used to deploy lightweight proxy nodes in each blockchain and use multi-threading to collect raw data on different nodes in parallel; Data storage module: Use the automated storage expansion mechanism to design a layered storage architecture, establish an abstract data model, map the raw data of different blockchains into a unified framework, and obtain the raw data in a unified format; Data integration processing module: aggregates the stored raw data, obtains aggregate groups, analyzes the correlation between different aggregate groups, and constructs a three-dimensional map of data relationships; Visualization display module: used to enable users to interact with the data stored in the blockchain, with a three-dimensional map of data relationships as the underlying architecture, to increase users' multi-level visualization of data and provide a user-friendly interface; The modules are connected to each other via wired and / or wireless means.

[0058] In addition, according to the implementation of the present application, the process described in the figure of a visual blockchain data storage system can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided in the present application. Of course, the architecture shown in the figure of a visual blockchain data storage system is only exemplary. When implementing different devices, adaptive selection or adjustment can be made according to actual needs.

[0059] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0060] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical users in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A visual blockchain data storage system, characterized in that: The visual blockchain data storage system includes: Data collection module: used to deploy lightweight proxy nodes in each blockchain and use multi-threading to collect raw data on different nodes in parallel; Data storage module: Use the automated storage expansion mechanism to design a layered storage architecture, establish an abstract data model, map the raw data of different blockchains into a unified framework, and obtain the raw data in a unified format; Data integration processing module: aggregates the stored raw data, obtains aggregate groups, analyzes the correlation between different aggregate groups, and constructs a three-dimensional map of data relationships; Visualization display module: It is used to enable users to interact with the data stored in the blockchain. It uses a three-dimensional map of data relationships as the underlying architecture, increases users' multi-level visualization of data, and provides a user-friendly interface.

2. The visual blockchain data storage system according to claim 1, characterized in that: The types of raw data include transaction records, smart contracts, block header information, and node status; Methods for using multithreading to collect raw data on different nodes in parallel include: Select a lightweight proxy node and deploy it to the blockchain; Select multi-threading technology, where each thread is responsible for fetching the latest raw data from a node; Use a short-term buffering mechanism for continuous acquisition of raw data.

3. The visual blockchain data storage system according to claim 2 is characterized in that: The tiered storage architecture includes a hot storage layer, a cold storage layer and an archive storage layer. The hot storage layer is used to store active data, the cold storage layer is used to store data with low access frequency, and the archive storage layer is used to store data that is no longer accessed for a long time.

4. The visual blockchain data storage system according to claim 3 is characterized in that: The method for designing a tiered storage architecture using an automated storage expansion mechanism includes: Use monitoring tools to consolidate the health status of each storage layer , Capacity utilization , Throughput and delay ; Calculate the evaluation index of each storage layer ,in, and represents the weight coefficient; Taking the current time point t as the time base point, calculate the evaluation index at each time point in turn, and obtain the absolute difference between the evaluation index of each time point and the current time point, until the absolute difference of the evaluation index is greater than the absolute threshold, collect the evaluation index of all interval time points between this time point and the time base point, and combine the evaluation index of the time base point to form a p-order data group, where p represents the number of time points extracted from the time base point forward; Select d time points forward from the current time t as the time base points, obtain d p-order data groups, and train the fitting model , set the stopping condition of the fitting model to be that the absolute difference between the output values ​​of two consecutive iterations in o consecutive iterations is less than the fitting threshold; in, Indicates the last time point The random error term at Indicates The fitting coefficients, Indicates time point The evaluation index at represents the adjustment factor; The evaluation index at the current time point t Input into the trained fitting model to obtain the Value, when When the value is greater than Q1, use horizontal expansion to increase the storage nodes of the hot storage layer. When the value is greater than Q2, use automated data lifecycle management tools to migrate data in the cold storage layer to the hot storage layer or archive storage layer. When the value is greater than Q3, use cloud services to add storage nodes to the archive storage layer; Among them, Q1, Q2 and Q3 are preset constants.

5. The visual blockchain data storage system according to claim 4 is characterized in that: The abstract data model is established to map the original data of different blockchains into a unified framework. The specific method includes: Set the properties of the block including block hash H_b, parent block hash H_pb, block height Q_h, timestamp t_b and transaction record T_b, and use smart contract Z_h, block header information Q_t, and node status J_z as abstract definitions, where the node status is arranged in the order of links in the block; Defining an abstract data model includes two layers of mapping, namely attribute mapping and abstract definition mapping. The attribute mapping layer is , the abstract definition mapping layer is ; Map the raw data of different blockchains into a unified framework formed by an abstract data model, so that the raw data has a unified standard format.

6. The visual blockchain data storage system according to claim 5 is characterized in that: The method for performing aggregation processing on the raw data includes: Step A1: Store the collected raw data in a hierarchical storage architecture, then remove irrelevant characters and unify the writing format of the raw data, build a vocabulary based on the set of unique words in the raw data, calculate the TF-IDF value of each word, use each TF-IDF value as a feature of the word, build a blank vector, correspond each blank element in the blank vector to the TF-IDF value of the word in the vocabulary, and then convert each data type in the raw data at the same time point into a feature vector representation; Step A2: Collect the feature vectors at each node on the blockchain, and for each data type, extract the maximum feature vector and minimum , the feature vector of each data type is divided into K_q blocks to obtain the magnetic center ,in, Indicates The magnetic center of the block, Indicates the index of the block; Step A3: Calculate the distance between each eigenvector and the magnetic center ,in, Represents the feature vector With magnetic center The distance Indicates feature vectors, represents the number of eigenvectors; Step A4: Assign each feature vector to the block to which the nearest magnetic center belongs, extract the number of feature vectors in each block, and if the number is less than the evaluation threshold, dissolve the block, and refill the dissolve feature vectors into the blocks to which other magnetic centers belong, thereby adaptively adjusting the preset K_q blocks; Step A5: After completing the distribution of feature vectors, update each magnetic center to the mean of all feature vectors in its block, repeat steps A3 to A4 until the preset number of distributions is reached, and stop, treating each block as an aggregation group.

7. The visual blockchain data storage system according to claim 6, characterized in that: The method for constructing the three-dimensional graph of data relations comprises: Construct a blank 3D layer, label the blockchain nodes horizontally, the data types vertically, and the time points vertically; Extract the nodes, data types and time points of each aggregation group, and simulate the aggregation group into solid points to fill in the blank three-dimensional layer to form point filling; Define the solid points to be connected by directed edges, set attribute values ​​on the directed edges, and set the attribute values ​​according to the formula ,in, and Used to represent the weight coefficient, Indicates the aggregation group at the same time point and Aggregation Group The number of times it appears at the same time, Indicates the aggregation group at a time point The number of occurrences, Indicates the aggregation group at a time point The number of occurrences, Indicates the aggregation group and Aggregation Group Overlapping content, Indicates the aggregation group and Aggregation Group All combined contents; Use directed edges to connect any two aggregation groups, calculate the attribute values ​​of the directed edges, and add the attribute values ​​to the directed edges to form a complete three-dimensional graph of data relationships.

8. The visual blockchain data storage system according to claim 7, characterized in that: The method for the user to interact with the data stored in the blockchain includes: Use React.js or Vue.js front-end frameworks to build responsive user interfaces and use visualization libraries to create interactive displays; Integrated dashboard tools to customize data display and monitoring.

9. The visual blockchain data storage system according to claim 8, characterized in that: The three-dimensional graph of data relationships is used as the underlying architecture to increase the multi-level visualization of data for users. The specific methods include: Set up multi-level data display. The macro view is used to display the overall status of the blockchain network; the meso view is used to display detailed information of a block or a node; and the micro view is used to display specific details of a smart contract. With the data relationship three-dimensional graph as the underlying architecture, the Neo4j graph database and graph visualization tools are used to display the correlation between the primary color data on each node in the data relationship three-dimensional graph. By clicking on a block or node, you can view the relevant information layer by layer, and use the breadcrumb navigation to quickly switch between different levels.

10. A visual blockchain data storage method, applied to a visual blockchain data storage system according to any one of claims 1 to 9, characterized in that: The visual blockchain data storage method comprises: Step G1: Use multi-threading to collect raw data on different nodes in parallel; Step G2: Build a tiered storage architecture, including a hot storage layer, a cold storage layer, and an archive storage layer, and use an automated storage expansion mechanism to design the tiered storage architecture; Step G3: Establish an abstract data model, map the raw data of different blockchains into a unified framework, and obtain the raw data in a unified format; Step G4: Aggregate the stored original data to obtain aggregation groups, build a blank three-dimensional layer, and analyze the correlation between different aggregation groups to obtain a three-dimensional map of data relationships; Step G5: View the storage of blockchain data in real time through visual display and user-friendly interface.

Citation Information

Patent Citations

  • Block chain fragmentation method for spectrum transaction

    CN116963077A

  • Data processing method and device based on hierarchical block chain network, equipment and medium

    CN117056084A

  • Method and system for optimizing sub-library table based on dynamic capacity expansion

    CN118467531A

  • Natural language processing for blockchain-based management of multi-source, multi-format binary objects

    US20230409604A1

  • Blockchain data archiving storage method and apparatus, computer device and storage medium

    WO2021003985A1

Cited By

  • Bitcoin intelligent contract construction and control block generation method based on multi-path script

    CN121235693A

  • Bitcoin smart contract construction and control block generation method based on multi-path script

    CN121235693B

  • Method for sending medical image data to a cloud storage system, and method for retrieving medical image data from a cloud storage system

    EP4783186A1