A blockchain virtual currency data platform system and a virtual currency data processing method

Abstract

The application discloses a kind of blockchain virtual currency data middle platform system and virtual currency data processing method, it is related to data middle platform field, including: blockchain full node remote procedure call encapsulation module, for obtaining the original blockchain data of input data middle platform;Transaction data analysis module, for structured analysis original blockchain data;Transaction data release message module, for decoupling transaction data analysis module and other modules in data middle platform;Transaction data storage module: for saving structured transaction data;Address information statistics module, for the address information on blockchain and transaction behavior information;Computing platform, for carrying out data mining and behavior characteristic analysis;The data middle platform in the application can collect the full amount of transaction data of mainstream virtual currency;Can be structured analysis and storage and release subscription message;It can realize the aggregation statistical analysis of block virtual currency data.

Description

Technical Field

[0001] This invention relates to the field of data middleware, specifically to a blockchain virtual currency data middleware system and a virtual currency data processing method. Background Technology

[0002] Currently, data analysis of blockchain cryptocurrencies has led to numerous applications, such as tracking and tracing blockchain transaction behavior, monitoring large abnormal transactions, detecting abnormal fluctuations in address amounts, and extracting and analyzing address behavior features based on artificial intelligence, as well as address identity recognition. However, existing data platforms cannot guide the acquisition of data on various mainstream blockchain cryptocurrencies. The lack of interoperability between existing data platforms for cryptocurrency transaction data limits analysis to single-currency-based analysis, preventing the ability to perform cross-referencing analysis of transactions across various mainstream blockchain cryptocurrencies. Summary of the Invention

[0003] To address the aforementioned issues, this invention provides a blockchain virtual currency data platform system and a virtual currency data processing method. The virtual currency transaction data in the data platform of this invention is interconnected, enabling multi-currency analysis and transaction activities, and achieving the ability to correlate and analyze blockchain virtual currency transactions of various mainstream currencies.

[0004] To achieve the above-mentioned objectives, the present invention provides a blockchain virtual currency data platform system, the data platform system comprising: The blockchain full node remote procedure call encapsulation module is used to integrate the remote procedure call services provided by each blockchain full node, provide access to the original data of the blockchain full node, and obtain the original blockchain data of the input data platform; The transaction data parsing module is used to analyze the input raw blockchain data, parse the raw blockchain data in a structured way, and form structured data with a unified standard for the data platform, which serves as the data foundation of the data platform. The transaction data publishing message module is used to decouple the transaction data parsing module from other modules in the data platform, as well as to provide support for data analysis and serve as the source table for the computing platform. The transaction data storage module is used to store the structured transaction data obtained by the transaction data parsing module. The address information statistics module is used to collect address information and transaction behavior information on the blockchain. The computing platform is used for data mining and behavioral feature analysis. The computing platform uses the transaction information from the transaction data publishing module as the source table and stores the analysis results in the transaction data storage module as the output result table.

[0005] Research into existing technologies reveals an urgent need for a data platform to guide the comprehensive collection, storage, and in-depth analysis of information on mainstream cryptocurrencies. This platform would provide data support for further investigation and analysis of fund transfers and usage related to blockchain cryptocurrencies. This invention provides a blockchain cryptocurrency data platform that can quickly guide the construction of such a platform, providing underlying data support for blockchain-based data analysis activities.

[0006] Preferably, in this invention, the transaction data parsing module cyclically requests the current state of the full nodes of the blockchain, queries the latest first data block on the blockchain, compares the first data block with the latest second data block processed by the current data platform. If the comparison result of the first data block and the second data block is consistent, then there is no new data block on the blockchain. If the comparison result of the first data block and the second data block is inconsistent, then the newly added first data block on the blockchain is obtained. After obtaining the newly added first data block on the blockchain, the data platform initiates a request to obtain the content of the first data block on the blockchain and performs structured parsing of the data in the first data block.

[0007] Preferably, in this invention, the transaction data storage module listens to the transaction data published by the transaction data publishing message module and uses asynchronous development data storage logic to write the transaction data into the database.

[0008] Preferably, in this invention, the address information statistics module listens to the transaction messages published by the transaction data publishing message module, parses the address information contained therein, and updates the address balance information and statistical information stored in the database.

[0009] Preferably, the data platform in this invention further includes a fork problem resolution module, which is used to record the block number corresponding to the blockchain data block processed by the transaction data parsing module, and obtain the block number sequence. For any first blockchain data block with block number N, when a second blockchain data block with block number N+M appears in the block number sequence, the fork problem resolution module obtains the first hash information of the on-chain data block corresponding to the first blockchain data block, and obtains the second hash information recorded when the first blockchain data block was processed last time. It compares whether the first hash information and the second hash information are consistent. If the comparison result is inconsistent, it triggers the transaction data repair operation in the first blockchain data block and publishes a fork message. After receiving the fork message, the relevant modules in the data platform roll back and recalculate, where N and M are both integers greater than 1.

[0010] Preferably, in this invention, the address information statistics module listens to the fork information published by the transaction data publishing message module, rolls back the historically processed erroneous transaction data, and recalculates.

[0011] Preferably, the analysis content of the computing platform in this invention includes: transaction aggregation analysis and address feature extraction.

[0012] Preferably, in this invention, the transaction data parsing module uses Python asynchronous programming to write the data parsing service.

[0013] Preferably, the transaction data publishing message module in this invention is implemented using a distributed publish-subscribe message method.

[0014] This invention also provides a method for processing virtual currency data based on the aforementioned blockchain virtual currency data platform system, the method comprising: The raw data collected is backed up using full node servers, and multiple full node services are set up for each type of cryptocurrency. Raw blockchain data is collected based on a blockchain full-node remote procedure call encapsulation module. The original blockchain data is structured by parsing the transaction data parsing module to obtain structured data. On the one hand, the structured data is stored in the transaction data storage module, and on the other hand, the transaction data publishing module is used to process the structured data and publish transaction data messages. The address information statistics module is used to process transaction data messages to implement address resolution service, and the address resolution results are saved. The computing platform is used to process transaction data messages for aggregation and statistical analysis, and the analysis results are saved.

[0015] One or more technical solutions provided by this invention have at least the following technical effects or advantages: The data platform in this invention can collect all transaction data of mainstream cryptocurrencies; it can perform structured parsing of the collected data and provide persistent storage solutions and publish / subscribe messages; it can introduce a computing platform to realize the aggregation and statistical analysis of blockchain cryptocurrency data; and it can provide data and computing capabilities for analyzing the transaction activities of various mainstream blockchain cryptocurrencies.

[0016] The data platform in this invention enables the interconnection of virtual currency transaction data, allowing for multi-currency analysis and transaction activities, and providing the ability to correlate and analyze blockchain virtual currency transactions of various mainstream currencies. Attached Figure Description

[0017] The accompanying drawings, which are provided to further illustrate embodiments of the invention and constitute a part of this invention, are not intended to limit the scope of the invention. Figure 1 A schematic diagram of the composition of a blockchain virtual currency data platform system; Figure 2 This is a flowchart illustrating the data processing method for virtual currencies. Detailed Implementation

[0018] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, where there is no conflict, the embodiments of the present invention and the features thereof can be combined with each other.

[0019] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0020] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0021] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0022] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0023] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the composition of a blockchain virtual currency data platform system. The following example illustrates the blockchain virtual currency data platform. The blockchain virtual currency data platform in this invention is not limited to the following implementation.

[0024] To collect full transaction data for mainstream cryptocurrencies, it is necessary to select full nodes for these cryptocurrencies and set up full node servers (providing RPC interfaces) locally. Due to the large volume of blockchain cryptocurrency data, it is essential to back up the collected raw data to prevent significant time wastage caused by the need for full resynchronization of full node data in case of anomalies. Furthermore, to ensure the stability of the node servers, multiple full node servers need to be set up for each individual cryptocurrency. To guarantee the synchronization speed of the full nodes, a method for dynamically checking network nodes and finding the optimal node is provided. The specific implementation involves: cyclically monitoring the log information of the full nodes to obtain the network nodes currently used by the full node (the full node dynamically adds used network nodes); testing the network latency of each node after obtaining the network nodes; and adding nodes with low network latency to the static node group of the full node, thereby continuously optimizing and expanding the static node group.

[0025] To perform structured parsing of the collected raw blockchain cryptocurrency data, a data parsing service was written using asynchronous Python programming. The advantage of using asynchronous Python programming is that it makes full use of the server's computing resources, increases throughput, and reduces blockchain data synchronization latency (to prevent historical data processing and block accumulation on chains with fast block generation speeds).

[0026] The service consists of: Blockchain Full Node RPC Encapsulation Module: This module integrates the RPCs provided by full nodes across various blockchains, offering convenient access to raw data from the full node in an abstract and unified form. It utilizes gRPC, a language-neutral, platform-neutral, and open-source remote procedure call library developed by Google, and is implemented using the Python programming language. This results in standardized, easily scalable, and low-latency access to raw blockchain data.

[0027] The transaction data parsing module analyzes the input raw blockchain data to extract valuable data, performing structured parsing to form a unified standard for the data platform. This facilitates the use of structured data and serves as the data foundation for the data platform. The module iteratively requests the current state of full nodes via gRPC to obtain the latest block on the chain and compares it with the latest block being processed by the current program to identify newly added data blocks. After obtaining the newly added data blocks, it initiates a request to retrieve the blockchain's transaction data for structured parsing.

[0028] The transaction data publishing Kafka message module serves two purposes: firstly, it decouples the structured data parsing from other modules; secondly, it provides support for real-time dynamic data analysis; and thirdly, it serves as the source table for the big data framework Flink, ensuring the success of subsequent artificial intelligence analysis.

[0029] This module uses Confluence Kafka and Python to dynamically publish blockchain-related data.

[0030] Transaction data persistent storage module: Used to persistently save the structured transaction data parsed by the transaction data parsing module.

[0031] By listening to the transaction data published by the message module, we use asynchronous Python to develop the data persistence storage logic, and finally write the data to MongoDB or other databases.

[0032] The blockchain fork problem resolution module addresses data inaccuracies caused by blockchain forks. The data in the latest blockchain block may not be accurate and could change within a defined timeframe. If the content of a block changes, its block hash will change. The program recursively requests analysis from nodes N blocks backward from the latest block, following a chronological order. If a block hash differs from the one processed by the transaction data parsing module, a repair of the transaction data in that block is triggered. Simultaneously, a fork message is published, allowing other relevant modules to roll back and recalculate upon receiving the message.

[0033] The blockchain packages on-chain transactions into blocks, placing each transaction into a single block. However, the transactions in the initial block might be moved to a later block for various reasons. This changes the previously packaged blocks. For example, the blockchain has already packaged seven blocks (1, 2, 3, 4, 5, 6, and 7) according to time. When packaging reaches the seventh block (six blocks after the initial block 1), the system considers the first block stable and unlikely to fork, and repackages the contents of block 1. At this point, the blocks already processed in real-time may contain errors. A new verification calculation is performed at the seventh block, meaning several blocks' worth of time are needed to confirm the correctness of the initial block's content. If incorrect, a rollback and recalculation are required. The specific value of N can be flexibly adjusted according to actual needs.

[0034] The address information statistics module is used to collect statistics on addresses and transaction activities on the blockchain. Developed using Python, it listens to transaction messages published by Kafka, parses the address information contained within, and updates the address information stored in MongoDB. Simultaneously, it listens for fork information published by Kafka, rolls back previously processed erroneous transaction data, and recalculates. This achieves structured parsing of transaction data and address information in blockchain cryptocurrencies, serving as the data foundation for building a blockchain cryptocurrency data platform.

[0035] To address the challenges of data mining and behavioral feature analysis using AI algorithms on existing blockchain data, the cutting-edge big data framework and distributed stream processing engine, Flink, is introduced. Based on Flink, rapid and efficient transaction aggregation analysis and address feature extraction can be achieved. By leveraging blockchain data and the Flink framework, the ability to correlate and analyze transactions of various mainstream cryptocurrencies is realized. Flink uses Kafka transaction information as the source table and stores the statistical analysis results in MongoDB as the result table. This blockchain cryptocurrency data platform provides both comprehensive transaction and address data for all cryptocurrencies and powerful computational and analytical capabilities.

[0036] The present invention also provides a method for constructing the blockchain virtual currency data platform, the method comprising: Set up a full node server locally; Then, the blockchain virtual currency data platform is built.

[0037] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating a virtual currency data processing method. This embodiment of the invention provides a virtual currency data processing method based on the aforementioned blockchain virtual currency data platform, the method comprising: The raw data collected is backed up using full node servers, and multiple full node services are set up for each type of cryptocurrency. Raw blockchain data is collected based on a blockchain full-node remote procedure call encapsulation module. The original blockchain data is structured by parsing the transaction data parsing module to obtain structured data. On the one hand, the structured data is stored in the transaction data storage module, and on the other hand, the transaction data publishing module is used to process the structured data and publish transaction data messages. The address information statistics module is used to process transaction data messages to implement address resolution service, and the address resolution results are saved. The computing platform is used to process transaction data messages for aggregation and statistical analysis, and the analysis results are saved.

[0038] One or more technical solutions provided by this invention have at least the following technical effects or advantages: The data platform in this invention can collect all transaction data of mainstream cryptocurrencies; it can perform structured parsing of the collected data and provide persistent storage solutions and publish / subscribe messages; it can introduce a computing platform to realize the aggregation and statistical analysis of blockchain cryptocurrency data; and it can provide data and computing capabilities for analyzing the transaction activities of various mainstream blockchain cryptocurrencies.

[0039] The data platform in this invention enables the interconnection of virtual currency transaction data, allowing for multi-currency analysis and transaction activities, and providing the ability to correlate and analyze blockchain virtual currency transactions of various mainstream currencies.

[0040] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0041] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0042] Furthermore, those skilled in the art will understand that various aspects of this specification can be described and illustrated in several patentable ways or situations, including any new and useful combination of processes, machines, products, or substances, or any new and useful improvements thereof. Accordingly, various aspects of this specification can be implemented entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. All of the above hardware or software may be referred to as a “data block,” “module,” “engine,” “unit,” “component,” or “system.” Furthermore, various aspects of this specification may be represented as a computer product located on one or more computer-readable media, including computer-readable program code.

[0043] Computer storage media may contain a propagated data signal containing computer program code, for example, on baseband or as part of a carrier wave. This propagated signal may take various forms, including electromagnetic, optical, and suitable combinations thereof. Computer storage media can be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, apparatus, or device to enable communication, propagation, or transmission of a program for use. The program code located on the computer storage medium can be propagated through any suitable medium, including radio, cable, fiber optic cable, RF, or similar media, or any combination of the above media.

[0044] The computer program code required for the operation of each part of this manual can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc.; conventional procedural programming languages ​​such as C, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP; dynamic programming languages ​​such as Python, Ruby, and Groovy; or other programming languages. This program code can run entirely on the user's computer, or as a standalone software package on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network, such as a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as Software as a Service (SaaS).

[0045] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0046] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0047] For each patent, patent application, patent application publication, and other material, such as articles, books, specifications, publications, and documents, referenced in this specification, the entire contents of which are incorporated herein by reference. This excludes historical application documents that are inconsistent with or conflict with the content of this specification, as well as documents that limit the broadest scope of the claims in this specification (currently or subsequently appended to this specification). It should be noted that in the event of any inconsistency or conflict between the descriptions, definitions, and / or terminology used in the supplementary materials to this specification and the content of this specification, the descriptions, definitions, and / or terminology used in this specification shall prevail.

[0048] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

[0049] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0050] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A blockchain virtual currency data platform system, characterized in that, The data platform system includes: The blockchain full node remote procedure call encapsulation module is used to integrate the remote procedure call services provided by each blockchain full node, provide access to the original data of the blockchain full node, and obtain the original blockchain data of the input data system. The transaction data parsing module is used to analyze the input raw blockchain data, parse the raw blockchain data in a structured way, and form structured data with a unified standard for the data system, which serves as the data foundation of the data system. The transaction data publishing module adopts a distributed publish-subscribe messaging approach to decouple the transaction data parsing module from other modules in the data system. By publishing structured transaction data messages, it provides real-time data source support for data analysis and serves as a data source table for the computing platform. The transaction data storage module is used to store the structured transaction data obtained by the transaction data parsing module. The address information statistics module is used to collect address information and transaction behavior information on the blockchain. The computing platform is used for data mining and behavioral feature analysis. The computing platform uses the transaction information of the transaction data publishing module as the source table and stores the analysis results in the transaction data storage module as the output result table. The data system also includes a fork problem resolution module, which records the block number corresponding to the blockchain data block processed by the transaction data parsing module and obtains the block number sequence. For any first blockchain data block with block number N, when a second blockchain data block with block number N+M appears in the block number sequence, the fork problem resolution module obtains the first hash information of the on-chain data block corresponding to the first blockchain data block and obtains the second hash information recorded when the first blockchain data block was processed last time. It compares whether the first hash information and the second hash information are consistent. If the comparison result is inconsistent, it triggers the transaction data repair operation in the first blockchain data block and publishes a fork message. After receiving the fork message, the relevant modules in the data system roll back and recalculate. N and M are both integers greater than 1. This involves setting up a full node server locally; backing up the collected raw blockchain data; setting up multiple full node servers for each currency; continuously monitoring the log information of the blockchain full node; obtaining the network nodes currently used by the blockchain full node; testing the network latency of each network node after obtaining the network nodes; and adding the network nodes with low network latency to the static node group of the blockchain full node.

2. The blockchain virtual currency data platform system according to claim 1, characterized in that, The transaction data parsing module iteratively requests the current state of the full nodes in the blockchain, queries the latest first data block on the blockchain, and compares the first data block with the latest second data block processed by the current data system. If the comparison results of the first data block and the second data block are consistent, then there is no new data block on the blockchain. If the comparison results of the first data block and the second data block are inconsistent, then the newly added first data block on the blockchain is obtained. After obtaining the newly added first data block on the blockchain, the data system initiates a request to obtain the content of the first data block on the blockchain and performs structured parsing of the data in the first data block.

3. The blockchain virtual currency data platform system according to claim 1, characterized in that, The transaction data storage module listens for transaction data published by the transaction data publishing message module and uses asynchronous development data storage logic to write the transaction data into the database.

4. The blockchain virtual currency data platform system according to claim 1, characterized in that, The address information statistics module listens to transaction messages published by the transaction data publishing module, parses the address information contained therein, and updates the address balance information and statistics stored in the database.

5. The blockchain virtual currency data platform system according to claim 1, characterized in that, The address information statistics module listens for fork information published by the transaction data publishing message module, rolls back erroneous transaction data processed in the past, and recalculates.

6. The blockchain virtual currency data platform system according to claim 1, characterized in that, The computing platform's analysis includes transaction aggregation analysis and address feature extraction.

7. The blockchain virtual currency data platform system according to claim 1, characterized in that, The transaction data parsing module uses Python asynchronous programming to write the data parsing service.

8. A method for processing virtual currency data based on the blockchain virtual currency data platform system according to any one of claims 1-7, characterized in that, The method includes: The raw data collected is backed up using full node servers, and multiple full node services are set up for each type of cryptocurrency. Raw blockchain data is collected based on a blockchain full-node remote procedure call encapsulation module. The original blockchain data is structured by parsing the transaction data parsing module to obtain structured data. On the one hand, the structured data is stored in the transaction data storage module, and on the other hand, the transaction data publishing module is used to process the structured data and publish transaction data messages. The address information statistics module is used to process transaction data messages to implement address resolution service, and the address resolution results are saved. The computing platform is used to process transaction data messages for aggregation and statistical analysis, and the analysis results are saved.

Images