Streaming data processing method and device based on block chain

Abstract

The embodiment of the invention provides a streaming data processing method and device based on a block chain, which can be applied to various scenes such as a block chain technology, a cloud technology, artificial intelligence, vehicle-mounted and the like, and the method comprises the following steps: when a block chain node receives a streaming calculation starting instruction carrying a resident transaction identifier, starting the resident transaction identifier according to the streaming calculation starting instruction; and creating a streaming data queue corresponding to the resident transaction identifier and starting a resident contract process. Then, through the streaming data queue and the resident contract process, streaming data which is continuously generated in real time is received, streaming calculation is carried out on the streaming data, and a streaming calculation result is stored in the to-be-generated block, so that the streaming data is continuously and efficiently processed in real time, the processing timeliness of the streaming data is effectively improved, and the processing efficiency of the streaming data is improved. And the transaction processing delay is reduced. And secondly, the resident contract process continuously processes the streaming data associated with the resident transaction, and the streaming data is terminated only when the streaming calculation termination instruction is received, so that the performance overhead caused by frequently creating and stopping the process by the block chain node is avoided.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of blockchain technology, and in particular to a streaming data processing method and device based on blockchain. Background Art

[0002] Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus protocols, and encryption algorithms. In the blockchain system, data blocks are combined into a chain data structure in chronological order, and cryptography is used to ensure that they cannot be tampered with or forged. Due to the characteristics of decentralization, information cannot be tampered with, and autonomy, the application of blockchain is becoming more and more extensive.

[0003] Under the relevant technology, when streaming data is processed through blockchain, the client collects the streaming data generated in real time. Whenever the amount of collected streaming data reaches a preset threshold, the collected streaming data is encapsulated into a blockchain transaction, and then the blockchain transaction is sent to the blockchain node, which processes the streaming data in the blockchain transaction.

[0004] Then, the blockchain node needs to wait until all the streaming data in the blockchain transaction arrives before starting to process the streaming data in the blockchain transaction; this results in poor timeliness in processing streaming data. Summary of the invention

[0005] The embodiments of the present application provide a streaming data processing method and device based on blockchain, which are used to improve the processing timeliness of streaming data.

[0006] On the one hand, an embodiment of the present application provides a streaming data processing method based on blockchain, which is applied to a blockchain node, including:

[0007] In response to a streaming computing start instruction carrying a resident transaction identifier, a streaming data queue is created corresponding to the resident transaction identifier, and streaming data associated with the resident transaction identifier received in real time is stored as a queue element in the streaming data queue;

[0008] A resident contract process is started corresponding to the resident transaction identifier, and through the resident contract process, whenever a queue element is detected in the streaming data queue, a streaming calculation is performed on the queue element in the streaming data queue, wherein each time a streaming calculation result of a queue element is obtained by calculation, the streaming calculation result of the queue element is written into the block to be generated;

[0009] In response to a streaming computing termination instruction carrying the resident transaction identifier, the resident contract process is terminated.

[0010] On the one hand, the embodiment of the present application provides a streaming data processing device based on blockchain, which is applied to a blockchain node, including:

[0011] A receiving unit, configured to, in response to a streaming computing start instruction carrying a resident transaction identifier, create a streaming data queue corresponding to the resident transaction identifier, and store the streaming data associated with the resident transaction identifier received in real time as a queue element in the streaming data queue;

[0012] a processing unit, configured to start a resident contract process corresponding to the resident transaction identifier, and, through the resident contract process, perform streaming calculation on the queue elements in the streaming data queue whenever it is detected that the streaming data queue contains a queue element, wherein each time a streaming calculation result of a queue element is obtained by calculation, the streaming calculation result of the queue element is written into the block to be generated;

[0013] A termination unit is used to terminate the resident contract process in response to a streaming computing termination instruction carrying the resident transaction identifier.

[0014] Optionally, the receiving unit is specifically used for:

[0015] Receiving a streaming data push request sent by a client, wherein the streaming data push request at least includes: the resident transaction identifier and associated streaming data;

[0016] When the streaming data queue corresponding to the resident transaction identifier is in an available state, the associated streaming data is stored in the streaming data queue as a queue element.

[0017] Optionally, the processing unit is further configured to:

[0018] When the block to be generated meets the block generation condition, the resident contract process is notified to stop writing the calculated streaming calculation results into the block to be generated;

[0019] The blocks to be generated are packaged into blocks to be verified, and the blocks to be verified are broadcast to other blockchain nodes for consensus.

[0020] Optionally, the processing unit is further configured to:

[0021] After notifying the resident contract process to stop writing the calculated streaming calculation results into the block to be generated, it also includes:

[0022] The stream computing result obtained by the calculation is written into the corresponding buffer area through the resident contract process;

[0023] The streaming computing results in the buffer area are written in batches into the next block to be generated.

[0024] Optionally, the processing unit is further configured to:

[0025] The block to be verified is broadcasted to other blockchain nodes, so that each blockchain node that receives the block to be verified verifies the calculation results corresponding to each of the multiple transaction identifiers in the block to be verified, obtains multiple transaction verification results, and generates a block verification result based on the multiple transaction verification results.

[0026] Optionally, the transaction verification result corresponding to the resident transaction identifier among the multiple transaction identifiers is obtained by the blockchain node that receives the block to be verified performing streaming calculations on the associated streaming data through the corresponding resident contract process, and comparing the obtained streaming calculation results with the corresponding streaming calculation results in the block to be verified.

[0027] Optionally, the processing unit is further configured to:

[0028] Packing the blocks to be generated into blocks to be verified, and broadcasting the blocks to be verified to other blockchain nodes for consensus, and then receiving the block verification results of the blocks to be verified;

[0029] When the number of corresponding successfully verified block verification results is greater than a preset threshold, the block to be verified is added to the blockchain.

[0030] Optionally, the processing unit is further configured to:

[0031] Before creating the streaming data queue corresponding to the resident transaction identifier, the certificate signature of the streaming computing startup instruction is verified, and the verification result is passed; and the authority of the initiator of the streaming computing startup instruction is verified, and the verification result is passed.

[0032] Optionally, the processing unit is further configured to:

[0033] After creating a streaming data queue corresponding to the resident transaction identifier, adding the resident transaction identifier to a transaction pool;

[0034] Before terminating the resident contract process corresponding to the resident transaction identifier, disabling the streaming data queue corresponding to the resident transaction identifier;

[0035] After the streaming data in the streaming data queue is cleared, deleting the streaming data queue;

[0036] The resident transaction identifier is deleted from the transaction pool.

[0037] Optionally, the processing unit is further configured to:

[0038] Before disabling the streaming data queue corresponding to the resident transaction identifier, the certificate signature of the streaming computing termination instruction is verified, and the verification result is passed; and, the authority of the initiator of the streaming computing termination instruction is verified, and the verification result is passed.

[0039] On the one hand, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned blockchain-based streaming data processing method when executing the program.

[0040] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program that can be executed by a computer device. When the program is run on the computer device, the computer device executes the steps of the above-mentioned blockchain-based streaming data processing method.

[0041] On the one hand, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer device, the computer device executes the steps of the above-mentioned blockchain-based streaming data processing method.

[0042] In the embodiment of the present application, when the blockchain node receives a streaming computing start instruction carrying a resident transaction identifier, a streaming data queue is created corresponding to the resident transaction identifier and a resident contract process is started. Then, through the streaming data queue and the resident contract process, real-time and continuously generated streaming data is received, and streaming computing is performed on the streaming data, without the need for the client to first collect the streaming data and send the collected streaming data to the blockchain node for processing, and without waiting for a fixed time interval or the transaction pool to reach a threshold before starting to process the transaction, thereby effectively improving the processing timeliness of streaming data, reducing the delay in transaction processing, and at the same time improving the utilization of system resources and the overall performance of the blockchain system.

[0043] Secondly, the resident contract process continuously processes the streaming data associated with the resident transaction and terminates only when it receives a streaming calculation termination instruction, avoiding the performance overhead caused by the frequent creation and termination of processes by blockchain nodes.

[0044] In addition, by introducing resident transactions, resident contract processes, and streaming data queues into the blockchain system, efficient processing of real-time and continuously generated streaming data can be achieved, enabling the blockchain system to better adapt to scenarios such as real-time data processing, real-time detection, and real-time analysis, expanding the application of blockchain in the field of real-time data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0046] Figure 1 A system architecture diagram of a blockchain system applicable to an embodiment of the present application;

[0047] Figure 2 An architectural diagram of a block structure provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of the structure of a blockchain node provided in an embodiment of the present application;

[0049] Figure 4 A flowchart of a streaming data processing method based on blockchain provided in an embodiment of the present application;

[0050] Figure 5 A flowchart of a streaming data processing method based on blockchain provided in an embodiment of the present application;

[0051] Figure 6 A flowchart of a block consensus method provided in an embodiment of the present application;

[0052] Fig. 7A A flowchart of the blockchain streaming computing creation phase provided in the embodiment of the present application;

[0053] Figure 7B A flowchart of the blockchain streaming computing creation phase provided in the embodiment of the present application;

[0054] Fig. 8A A schematic diagram of the flow of the streaming data push phase provided in an embodiment of the present application;

[0055] Figure 8B A schematic diagram of the flow of the streaming data push phase provided in an embodiment of the present application;

[0056] Fig. 9A A flowchart of the blockchain streaming computing termination phase provided in an embodiment of the present application;

[0057] Fig. 9B A flowchart of the blockchain streaming computing termination phase provided in an embodiment of the present application;

[0058] Fig. 10A A flowchart of the blockchain node block execution phase provided in the embodiment of the present application;

[0059] Fig. 10B A flowchart of the blockchain node block execution phase provided in the embodiment of the present application;

[0060] Fig.11A A schematic diagram of the process of verifying a block in an embodiment of the present application;

[0061] Fig. 11B A schematic diagram of the process of verifying a block in an embodiment of the present application;

[0062] Fig.12 A schematic diagram of the structure of a streaming data processing device based on blockchain provided in an embodiment of the present application;

[0063] Fig.13 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and beneficial effects of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0065] For ease of understanding, the terms involved in the embodiments of the present invention are explained below.

[0066] Blockchain: A distributed ledger technology in the field of information technology, generally composed of consensus, transaction blocks and state data storage, cryptographic identity security, etc. Since the ledger is distributed and the blocks are consensus-based, it has the characteristics of being tamper-proof, traceable, and jointly maintained. Blockchains can be divided into public chains, alliance chains, and private chains according to their degree of openness. The streaming data processing method in the embodiment of the present application is applicable to any of the public chains, alliance chains, and private chains.

[0067] Smart contract: A smart contract is a computer protocol designed to communicate, verify or execute contracts in an information-based manner. Smart contracts allow for trusted transactions without a third party.

[0068] Consortium chain: A special type of blockchain, which is a semi-decentralized network. Unlike public chains, consortium chains only allow specific organizations or entities to join the network. These organizations or entities jointly manage and maintain the blockchain, share data and consensus processes. Consortium chains have higher performance, better privacy protection, and stricter permission control than public chains.

[0069] Process pool: A resource pool used to manage multiple processes. It is responsible for creating, maintaining, scheduling, and recycling processes. The process pool can improve the concurrent performance of the system because it allows multiple tasks to be executed simultaneously in different processes. In the blockchain system, the process pool can be used to handle concurrent transaction requests and improve the throughput of the entire system.

[0070] Contract repository: A data structure used to store and manage smart contracts. It contains the code, status, and metadata of all smart contracts deployed on the blockchain. The contract repository allows users to query, call, and update smart contracts, and also provides blockchain nodes with the infrastructure to execute smart contracts.

[0071] Streaming data: It can also be called streaming data, which refers to a set of data sequences that arrive in a sequential, large, fast, and continuous manner. Generally speaking, streaming data can be regarded as a dynamic data set that grows infinitely over time.

[0072] Resident transactions: A special type of blockchain transaction, whose main feature is that it exists in the blockchain system for a long time and continuously processes streaming data. The life cycle of ordinary blockchain transactions ends after the processing results are packaged into blocks and added to the blockchain. However, the processing results of resident transactions remain active after being added to the blockchain until a specific computation termination task is received, thereby improving the efficiency of the blockchain system in processing streaming data.

[0073] Resident contract process: A long-lived process or coroutine that continuously processes streaming data associated with resident transactions and generates transaction outputs based on the processing results until a specific computation termination task is received, avoiding the performance overhead caused by frequent creation and destruction of processes or coroutines.

[0074] Streaming data pool: In order to better support streaming data processing, this application introduces the concept of streaming data pool. Multiple streaming data queues can be created in the streaming data pool, and each streaming data queue corresponds to a resident transaction. The resident transaction will always obtain the streaming data in the streaming data queue and execute it. This design enables the blockchain system to process real-time, continuously generated data streams more efficiently, improving the system's throughput and response speed.

[0075] Queue: A special linear list that allows deletion only at the front of the list and insertion at the rear. The end where the insertion is performed is called the tail, and the end where the deletion is performed is called the head. The data elements of a queue are also called queue elements. Inserting a queue element into a queue is called enqueuing, and deleting a queue element from a queue is called dequeuing.

[0076] Stream computing: A big data processing technology that processes real-time, continuously generated data streams. Unlike traditional batch computing, stream computing does not need to wait for the data set to arrive completely before processing, but starts processing immediately when the data arrives. This enables stream computing to achieve low-latency, high-throughput data processing, which is suitable for real-time analysis, real-time detection and other scenarios.

[0077] Public key encryption algorithm: An encryption algorithm that uses a pair of keys for encryption and decryption operations. The key pair is divided into a public key and a private key. The public key is public and can be obtained by anyone; the private key is private and can only be accessed by the owner of the key. The core idea of ​​the public key encryption algorithm is that data encrypted with a public key can only be decrypted with the corresponding private key, and vice versa.

[0078] Blockchain ledger: The core data structure in the blockchain system, used to store and manage all confirmed blocks. The blockchain is organized in a chain structure, and each block contains a set of transactions, a block header (including the hash value of the previous block, timestamp and other metadata), and other information. The blockchain provides a public, unalterable transaction history record for the blockchain system, ensuring the transparency and consistency of the system.

[0079] State data: refers to the data structure used in the blockchain system to represent the current state of the blockchain system. State data includes: the balance of all accounts, the status of smart contracts, and other related information. State data is continuously updated as transactions are executed, reflecting the global state of the blockchain system at a certain point in time. In the blockchain system, state data is usually stored in the form of Merkle trees or other encrypted data structures to ensure its integrity and security.

[0080] The design concept of the embodiments of the present application is introduced below.

[0081] Due to the characteristics of blockchain such as decentralization, information immutability, and autonomy, the application of blockchain is becoming more and more extensive. Under the relevant technology, when processing streaming data through blockchain, the client collects the streaming data generated in real time. Whenever the amount of collected streaming data reaches a preset threshold, the collected streaming data is encapsulated into a blockchain transaction, and then the blockchain transaction is sent to the blockchain node, which processes the streaming data in the blockchain transaction. In actual application scenarios, the amount of streaming data generated by the client is often huge, so the client needs to continuously collect streaming data and encapsulate it into blockchain transactions one by one, and send it to the blockchain node for processing.

[0082] In traditional blockchain solutions, each blockchain transaction is independent and needs to be processed separately. Therefore, each time a blockchain node receives a blockchain transaction, it starts a processing process for the blockchain transaction. The processing process processes the streaming data in the blockchain transaction to obtain the processing result. After the processing result is packaged into a block and added to the blockchain, the life cycle of a blockchain transaction ends, and the blockchain node terminates the processing process assigned to the blockchain transaction.

[0083] In the above streaming data processing solution, the blockchain node needs to wait until all the streaming data in a blockchain transaction arrives before starting to process the streaming data in the blockchain transaction; in some scenarios, the blockchain node needs to wait for a fixed time interval or the transaction pool reaches a threshold before starting to process the transaction; this not only leads to poor timeliness in the processing of streaming data, but also leads to insufficient utilization of system resources during certain periods, thereby reducing overall performance.

[0084] On the other hand, since blockchain nodes need to create and terminate corresponding processing processes when processing each blockchain transaction, that is, blockchain nodes create and terminate processing processes frequently, resulting in a large performance overhead for blockchain nodes.

[0085] In order to better support streaming data processing, an embodiment of the present application proposes a streaming data processing method based on blockchain, in which resident transactions, resident contract processes, and streaming data queues are introduced to achieve continuous, real-time and efficient processing of streaming data.

[0086] Specifically, the blockchain node responds to the streaming computing start instruction carrying the resident transaction identifier, creates a streaming data queue corresponding to the resident transaction identifier, and saves the streaming data associated with the resident transaction identifier received in real time as a queue element in the streaming data queue. The blockchain node starts the resident contract process corresponding to the resident transaction identifier, and through the resident contract process, whenever it detects that the streaming data queue contains a queue element, it performs streaming computing on the queue elements in the streaming data queue, wherein each time the streaming computing result of a queue element is obtained by computing, the streaming computing result of the queue element is written into the block to be generated, so as to realize continuous and real-time processing of streaming data, until the streaming computing termination instruction carrying the resident transaction identifier is received, the resident contract process is terminated, that is, the processing of streaming data is stopped.

[0087] In the above technical solution, when a blockchain node receives a streaming computing start instruction carrying a resident transaction identifier, it creates a streaming data queue and starts a resident contract process corresponding to the resident transaction identifier. Then, through the streaming data queue and the resident contract process, real-time and continuously generated streaming data is received, and streaming computing is performed on the streaming data, without the need for the client to first collect the streaming data and send the collected streaming data to the blockchain node for processing, nor is it necessary to wait for a fixed time interval or the transaction pool to reach a threshold before starting to process the transaction, thereby effectively improving the processing timeliness of streaming data, reducing the delay in transaction processing, and at the same time improving the utilization of system resources and the overall performance of the blockchain system.

[0088] Secondly, the resident contract process continuously processes the streaming data associated with the resident transaction and terminates only when it receives a streaming calculation termination instruction, avoiding the performance overhead caused by the frequent creation and termination of processes by blockchain nodes.

[0089] In addition, by introducing resident transactions, resident contract processes, and streaming data queues into the blockchain system, efficient processing of real-time and continuously generated streaming data can be achieved, enabling the blockchain system to better adapt to scenarios such as real-time data processing, real-time detection, and real-time analysis, expanding the application of blockchain in the field of real-time data processing.

[0090] refer to Figure 1 , which is a system architecture diagram of a blockchain system applicable to the embodiment of the present application, the blockchain system 100 at least includes a client 101 and multiple blockchain nodes 102, wherein the blockchain node 102 can be any form of computing device in the access network, such as a server, a user terminal, etc. The blockchain nodes 102 can be directly or indirectly connected through wired or wireless communication, and the present application does not limit this.

[0091] The blockchain node 102 includes a hardware layer, an intermediate layer, an operating system layer, and an application layer. The functions involved in the blockchain node 102 include:

[0092] 1) Routing: a basic function of blockchain nodes, used to support communication between blockchain nodes.

[0093] In addition to the routing function, blockchain nodes can also have the following functions:

[0094] 2) Applications, which are deployed in the blockchain to implement specific businesses according to actual business needs, record data related to the implementation of functions to form record data, carry digital signatures in the record data to indicate the source of the task data, and send the record data to other blockchain nodes in the blockchain system for other blockchain nodes to add the record data to the temporary block when they successfully verify the source and integrity of the record data.

[0095] For example, the services implemented by the application include:

[0096] 2.1) Wallet, used to provide the function of conducting electronic currency transactions, including initiating transactions (i.e., sending the transaction record of the current transaction to other blockchain nodes in the blockchain system. After successful verification by other blockchain nodes, as a response to acknowledging the validity of the transaction, the transaction record data is stored in a temporary block of the blockchain; of course, the wallet also supports querying the remaining electronic currency in the electronic currency address.

[0097] 2.2) Shared ledger, which is used to provide functions such as storage, query and modification of account data, and send the record data of the operation on the account data to other blockchain nodes in the blockchain system. After other blockchain nodes verify the validity, they will store the record data in a temporary block as a response to acknowledge the validity of the account data, and can also send confirmation to the blockchain node that initiated the operation.

[0098] 2.3) Smart contracts are computerized protocols that can execute the terms of a contract. They are implemented by deploying code on a shared ledger that is executed when certain conditions are met. The code is used to complete automated transactions based on actual business needs, such as querying the logistics status of the goods purchased by the buyer and transferring the buyer's electronic currency to the merchant's address after the buyer signs for the goods. Of course, smart contracts are not limited to executing contracts for transactions, but can also execute contracts for processing received information.

[0099] 3) Blockchain, including a series of blocks that are connected to each other in the order of their generation. Once a new block is added to the blockchain, it will not be removed. The block records the record data submitted by the blockchain nodes in the blockchain system.

[0100] See also Figure 2 , is an optional schematic diagram of the block structure (Block Structure) provided in the embodiment of the present application. Each block includes the hash value of the transaction record stored in this block (the hash value of this block) and the hash value of the previous block. Each block is connected by the hash value to form a blockchain. In addition, the block can also include information such as the timestamp when the block was generated. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains relevant information for verifying the validity of its information (anti-counterfeiting) and generating the next block.

[0101] See also Figure 3, which is a schematic diagram of the structure of a blockchain node provided in an embodiment of the present application. The blockchain node 102 includes: a network module 301, a verification module 302, a streaming data pool 303, a transaction pool 304, a scheduling execution module 305, a consensus module 306, and a storage module 307.

[0102] The network module 301 is used to realize the communication between blockchain nodes, specifically including: connection management between blockchain nodes, data transmission encryption, data broadcasting and other functions.

[0103] Verification module 302 is used to verify transactions and blocks to ensure the legitimacy and correctness of transactions and blocks, including certificate verification and authority verification. Certificate verification refers to: verifying the certificate signature of transactions and blocks to ensure that transactions and blocks are initiated by legitimate participants. Authority verification refers to: checking whether the initiator of transactions and blocks has the corresponding authority to operate.

[0104] Streaming data pool 303: used to manage streaming data queues, including functions such as creation and deletion of streaming data queues, and receiving and sending data.

[0105] Transaction pool 304: used to manage pending transactions, including the reception, storage, and sorting of transactions. Each transaction corresponds to a unique identifier. The transaction pool 304 includes ordinary transactions and resident transactions.

[0106] Scheduling and execution module 305: used to schedule and execute transactions in the transaction pool 304. Scheduling and execution module 305 specifically includes: block generator, transaction scheduler, contract process pool, and contract warehouse. The block generator is used to generate blocks to be verified based on the calculation results of transactions in the transaction pool 304; the transaction scheduler is used to schedule transactions in the transaction pool 304 according to preset strategies, such as scheduling by priority, fee, etc.; the contract process pool includes: contract processes for executing smart contracts corresponding to transactions, including ordinary contract processes for executing ordinary transactions and resident transaction processes for executing resident transactions; the contract warehouse is used to store deployed smart contracts.

[0107] Consensus module 306: used to implement the consensus algorithm of the blockchain system. The consensus module specifically includes block broadcasting function and block verification function, wherein block broadcasting refers to: broadcasting the newly generated block to be verified to other blockchain nodes. Block verification refers to: verifying the correctness of the received block to be verified and adding it to the blockchain.

[0108] Storage module 307: used to store blockchain data, including block ledger, status data, etc. The block ledger includes block data in the blockchain; the status data includes status information in the blockchain system, such as account balance, status of smart contracts, etc.

[0109] In practical applications, the streaming data processing method in the embodiments of the present application, by combining streaming computing with blockchain, realizes efficient processing, low-latency response and secure storage of real-time data in various real-time data processing scenarios, improves system performance, and meets the real-time data processing needs of various industries.

[0110] The following are some examples of application scenarios:

[0111] Application scenario 1: Real-time payment and settlement scenario.

[0112] In real-time payment and clearing scenarios, the financial system will generate a large amount of continuous transaction data in real time, which is streaming data that needs to be processed in real time. The streaming data processing method in this application is used to process the above transaction data, which specifically includes the following process:

[0113] The financial system sends a streaming computing start instruction with a resident transaction identifier to the blockchain node. The blockchain node creates a streaming data queue and starts a resident contract process corresponding to the resident transaction identifier. Then, the streaming data queue saves the transaction data pushed by the financial system in real time and continuously, and the resident contract process performs streaming computing on the transaction data in real time and continuously until the streaming computing termination instruction is received.

[0114] That is to say, in real-time payment and settlement scenarios, the streaming data processing method in this application can realize fast and secure processing of relevant transaction data, thereby improving the efficiency of the financial system.

[0115] Application scenario 2: Logistics tracking scenario.

[0116] In the logistics tracking scenario, streaming data such as real-time cargo status and transaction information are involved. When the streaming data processing method in this application is used to process real-time cargo status and transaction information, a transparent and reliable logistics tracking service can be achieved, thereby improving logistics tracking efficiency. The specific processing process will not be described in detail at this time.

[0117] Application scenario three: energy trading and distribution scenario.

[0118] In energy trading and distribution scenarios, the generation, trading and distribution of distributed energy resources (such as solar energy, wind energy, etc.) require real-time detection and scheduling, which generates large amounts of continuous streaming data related to energy generation, energy trading and energy distribution.

[0119] This application can process streaming data related to energy generation, energy trading and energy distribution in real time, realize intelligent and transparent energy management, and improve energy utilization efficiency. The specific processing process will not be repeated at this time.

[0120] Application scenario 4: Real-time copyright management scenario.

[0121] In the digital content industry, such as music, video, literary works, etc., a large amount of copyright transaction data is generated. Real-time detection and processing of copyright transaction data is particularly important for protecting the rights and interests of creators.

[0122] This application can process copyright transaction data in real time, providing creators with safe and reliable copyright protection. The specific processing process will not be described in detail at this time.

[0123] Application scenario five: Supply chain finance scenario.

[0124] In the supply chain finance scenario, a large amount of real-time streaming data such as transactions, financing, and credit is involved. This solution can process these streaming data in real time, provide transparent and reliable financial services to all parties in the supply chain, reduce financing costs, and improve capital utilization. The specific processing process will not be repeated at this time.

[0125] It should be noted that the above-mentioned application scenarios are only used to illustrate the application scenarios to which the present application is applicable. The application scenarios to which the present application is applicable are not limited to the above-mentioned ones, but may also be other real-time data processing scenarios.

[0126] based on Figure 1 and Figure 3 The system architecture diagram shown in the figure, the embodiment of the present application provides a process of a streaming data processing method based on blockchain, such as Figure 4 As shown, the process of the method is executed by a computer device, which may be Figure 1 The blockchain node 102 shown includes the following steps:

[0127] Step 401, in response to a streaming computing start instruction carrying a resident transaction identifier, a streaming data queue is created corresponding to the resident transaction identifier.

[0128] Specifically, the blockchain node receives the streaming computing start instruction sent by the client, which carries the resident transaction identifier. Resident transactions exist in the blockchain system for a long time and continuously process streaming data. After the processing results of resident transactions are added to the blockchain, resident transactions remain active until a specific computing termination task is received. Each resident transaction corresponds to a unique identifier.

[0129] A streaming data queue corresponding to the resident transaction identifier is created in the streaming data pool, and the streaming data queue is used to store all streaming data corresponding to the resident transaction identifier. In addition, the corresponding relationship between the streaming data queue and the resident transaction identifier is recorded, and the resident transaction identifier is added to the transaction pool.

[0130] In some embodiments, the client signs the streaming computing startup instruction and sends it to the blockchain node. After the blockchain node receives the streaming computing startup instruction, it verifies the certificate signature of the streaming computing startup instruction through the verification module.

[0131] If the signature verification fails, the process ends directly; if the signature verification passes, the authority of the initiator of the streaming computing startup instruction is verified through the verification module.

[0132] If the permission verification fails, the process ends directly; if the permission verification passes, a streaming data queue is created for the corresponding resident transaction ID.

[0133] It should be noted that this application does not limit the order of the above-mentioned certificate signature verification and initiator's authority verification.

[0134] In the embodiment of the present application, by verifying the certificate signature of the streaming computing startup instruction and the authority of the initiator, the legitimacy and correctness of the streaming computing startup instruction are guaranteed, thereby improving the security of resident transaction processing.

[0135] In some embodiments, before creating a streaming data queue corresponding to a resident transaction identifier, it is also possible to first determine whether the contract repository contains the smart contract corresponding to the resident transaction identifier. If so, a streaming data queue is created corresponding to the resident transaction identifier; if not, the process ends directly.

[0136] Step 402: Save the streaming data associated with the resident transaction identifier received in real time as a queue element in the streaming data queue.

[0137] Specifically, the streaming data queue corresponds to a continuous storage space and is managed by two pointers: the front pointer and the rear pointer. The front pointer points to the head element, and the rear pointer points to the storage location of the next queued element.

[0138] In some embodiments, a blockchain node receives a streaming data push request sent by a client, wherein the streaming data push request includes at least: a resident transaction identifier and associated streaming data. When a streaming data queue corresponding to the resident transaction identifier is in an available state, the associated streaming data is saved as a queue element in the streaming data queue.

[0139] Specifically, the client encapsulates the streaming data and the resident transaction identifier in a data push request, which can carry single streaming data or batch streaming data. The client signs the data push request and sends it to the blockchain node. The blockchain node verifies the certificate signature of the data push request and the authority of the initiator through the verification module. When all verifications are passed, it is determined whether the streaming data queue corresponding to the resident transaction identifier is in an available state, where the streaming data queue is in an available state, which can be: the streaming data queue is not disabled, the number of queue elements in the streaming data queue does not reach the storage threshold, etc.

[0140] When the streaming data queue corresponding to the resident transaction identifier is in an available state, the associated streaming data is saved as a queue element in the streaming data queue. In actual applications, when the streaming data push request carries a single streaming data, the single streaming data is inserted as a queue element into the tail of the streaming data queue, that is, the storage location pointed to by the tail pointer rear, and then the tail pointer rear is incremented by 1.

[0141] When the streaming data push request carries batch streaming data (for example, 10 streaming data), the batch streaming data is inserted as a queue element at the end of the streaming data queue, that is, the storage location pointed to by the tail pointer rear, and then the tail pointer rear is increased by 1.

[0142] After the streaming data is added successfully, the blockchain node returns a message indicating that the streaming data has been added successfully to the client.

[0143] In the embodiment of the present application, by verifying the certificate signature of the data push request and the authority of the initiator, the legitimacy and correctness of the data push request are guaranteed, thereby improving the security of streaming data push. Secondly, a streaming data queue is set to store the streaming data pushed by the client in real time, so that the blockchain node can continuously obtain the streaming data in the streaming data queue and perform streaming calculations, so that the blockchain system can more efficiently process the real-time and continuously generated data streams, and improve the throughput and response speed of the system.

[0144] Step 403, starting the resident contract process corresponding to the resident transaction identifier.

[0145] Specifically, a resident contract process refers to a long-lived process or coroutine that continuously processes streaming data associated with resident transactions until a specific computation termination task is received.

[0146] When a blockchain node receives a streaming computing start instruction, it allocates a new resident contract process in the contract process pool for the resident transaction identifier carried by the streaming computing start instruction, and starts the new resident contract process; at the same time, it records the association between the resident contract process and the resident transaction identifier.

[0147] Step 404, through the resident contract process, each time it is detected that the streaming data queue contains a queue element, the queue element in the streaming data queue is stream-calculated, wherein each time the streaming calculation result of a queue element is obtained by calculation, the streaming calculation result of the queue element is written into the block to be generated.

[0148] Specifically, the resident contract process starts a detection client for detecting the streaming data queue corresponding to the resident transaction identifier, and uses the detection client to detect in real time whether the streaming data queue contains queue elements. When a queue element is stored in the streaming data queue, the resident contract process obtains the queue element from the head of the streaming data queue, that is, obtains the queue element pointed to by the head pointer front in the streaming data queue, and increases the head pointer front by 1. The queue element corresponds to a single streaming data or a batch of streaming data.

[0149] The resident contract process performs streaming calculations on the obtained queue elements by executing the smart contract corresponding to the resident transaction identifier, obtains the streaming calculation results, and then writes the streaming calculation results of the queue elements into the block to be generated.

[0150] Since the resident contract process performs streaming calculations on the queue elements in the streaming data queue every time it detects that the streaming data queue contains queue elements, there is no need to wait for a fixed time interval or for the transaction pool to reach a threshold before starting to process transactions. This can achieve faster transaction processing and low-latency response, help improve the utilization of system resources, and thus improve the overall performance of the blockchain system.

[0151] In some embodiments, the block to be generated includes a streaming result queue corresponding to the streaming data queue. The streaming result queue also corresponds to a continuous storage space and is managed by a queue head pointer front and a queue tail pointer rea.

[0152] After obtaining the streaming calculation result of the queue element, the resident contract process inserts the streaming calculation result as a queue element into the tail of the streaming result queue in the block to be generated, that is, the storage location pointed to by the tail pointer rear, and then increases the tail pointer rear by 1.

[0153] Step 405, in response to the streaming computing termination instruction carrying the resident transaction identifier, terminate the resident contract process.

[0154] Specifically, the blockchain node receives a streaming computing termination instruction sent by the client, and the streaming computing termination instruction carries a resident transaction identifier. The streaming computing termination instruction is used to instruct the blockchain node to terminate the resident contract process.

[0155] It should be noted that the above steps 401 to 404 take a resident transaction as an example to introduce the process of the blockchain node performing streaming calculations on the streaming data associated with the resident transaction. In actual applications, the blockchain node can execute multiple resident transactions in parallel through the scheduling execution module.

[0156] In addition, in addition to processing resident transactions, blockchain nodes can also process ordinary transactions. Specifically, the blockchain node starts an ordinary contract process for the ordinary transaction. The ordinary contract process executes the smart contract corresponding to the ordinary transaction, calculates the transaction data corresponding to the ordinary transaction, obtains the calculation result, and adds the processing result to the block to be generated. In actual applications, the blockchain node can execute multiple ordinary transactions in parallel through the scheduling execution module, and can also execute ordinary transactions and resident transactions in parallel through the scheduling execution module. This application does not make specific limitations on this.

[0157] In the embodiment of the present application, when the blockchain node receives a streaming computing start instruction carrying a resident transaction identifier, it creates a streaming data queue and starts a resident contract process corresponding to the resident transaction identifier. Then, through the streaming data queue and the resident contract process, the real-time and continuously generated streaming data is received, and the streaming data is streamed and computed, without the need for the client to collect the streaming data first, and the collected streaming data is sent to the blockchain node for batch processing, thereby effectively improving the processing timeliness of the streaming data and reducing the delay in transaction processing. Secondly, the resident contract process continuously processes the streaming data associated with the resident transaction, and only terminates when the streaming computing termination instruction is received, avoiding the performance overhead caused by the frequent creation and termination of the blockchain node. In addition, the introduction of resident transactions, resident contract processes, and streaming data queues in the blockchain system realizes efficient processing of real-time and continuously generated streaming data, so that the blockchain system can better adapt to scenarios such as real-time data processing, real-time detection, and real-time analysis, and expands the application of blockchain in the field of real-time data processing.

[0158] In some embodiments, the client signs the streaming computing termination instruction and sends it to the blockchain node. After the blockchain node receives the streaming computing termination instruction, the verification module verifies the certificate signature of the streaming computing termination instruction.

[0159] If the signature verification fails, a message indicating that the certificate signature verification failed is returned; if the signature verification passes, the authority of the initiator of the streaming computing termination instruction is verified through the verification module.

[0160] If the permission verification fails, a message indicating that the permission verification failed is returned; if the permission verification passes, the resident contract process is terminated.

[0161] In the embodiment of the present application, by verifying the certificate signature of the streaming computing termination instruction and the authority of the initiator, the legality and correctness of the streaming computing termination instruction are guaranteed, thereby improving the security of resident transaction processing.

[0162] In some embodiments, when a blockchain node receives a streaming computing termination instruction, the streaming data queue may still contain unprocessed streaming data. In order to ensure that these unprocessed streaming data are processed smoothly and avoid streaming data loss. In the embodiment of the present application, before terminating the resident contract process corresponding to the resident transaction identifier, the blockchain node first disables the streaming data queue corresponding to the resident transaction identifier, and then the streaming data arriving at the streaming data queue for a period of time will be rejected; at the same time, the blockchain node starts a countdown for the streaming data queue corresponding to the resident transaction identifier.

[0163] On the other hand, the resident contract process continues to obtain the remaining streaming data in the streaming data queue for streaming calculation, and writes the obtained streaming calculation results into the block to be generated. When the streaming data in the streaming data queue is cleared or the countdown ends, the blockchain node deletes the streaming data queue, deletes the resident transaction identifier from the transaction pool, and then terminates the resident contract process corresponding to the resident transaction identifier, and returns a message of successful termination of the streaming calculation to the client. This ensures that when the streaming calculation termination instruction is received, the unprocessed streaming data in the streaming data queue can be effectively processed to avoid data loss.

[0164] In some embodiments, when the block to be generated meets the block generation conditions, the resident contract process is notified to stop writing the calculated streaming calculation results into the block to be generated. Then the block to be generated is packaged into a block to be verified, and the block to be verified is broadcast to other blockchain nodes for consensus.

[0165] Specifically, the block generation condition may be that the size of the block to be generated reaches a configured threshold, or reaches a configured block generation time interval (for example, the block generation time interval is 2s, that is, a block needs to be generated every 2s, and when the 2s countdown ends, the block to be generated meets the block generation condition).

[0166] When the block to be generated meets the block generation conditions, the blockchain node notifies all started resident contract processes to stop writing the calculated streaming calculation results into the block to be generated. However, the blockchain node can still continue to perform streaming calculations on the queue elements in the streaming data queue through the resident contract process, and write the calculated streaming calculation results into its own cache or disk. Subsequently, the streaming calculation results in the cache or disk are batch-written into the next block to be generated.

[0167] In addition, the blockchain node closes the block to be generated through the block generator, calculates the corresponding block hash, transaction Merkle tree and other block information, obtains the block to be verified, and broadcasts the block to be verified to other blockchain nodes for consensus.

[0168] For example, see Figure 5 , the transaction pool of the blockchain node is set to include resident transaction 1, resident transaction 2, resident transaction 3 and ordinary transaction group. The data pool of the blockchain node includes: streaming data queue 1 corresponding to resident transaction 1, streaming data queue 2 corresponding to resident transaction 2, streaming data queue 3 corresponding to resident transaction 3. The blockchain node also includes: resident contract process 1 corresponding to resident transaction 1, resident contract process 2 corresponding to resident transaction 2, resident contract process 3 corresponding to resident transaction 3 and ordinary contract process.

[0169] The blocks to be generated include: streaming result queue 1 corresponding to resident transaction 1, streaming result queue 2 corresponding to resident transaction 1, streaming result queue 3 corresponding to resident transaction 3, and a common result queue.

[0170] Resident contract process 1, resident contract process 2, resident contract process 3 and ordinary contract process perform transaction processing in parallel.

[0171] Specifically, each time the blockchain node receives streaming data of resident transaction 1 pushed by the client, it saves the streaming data as a queue element in streaming result queue 1. Resident contract process 1 obtains queue elements from streaming result queue 1 in real time, performs streaming calculations on queue elements, and writes streaming calculation results into streaming result queue 1. The execution process of resident transaction 2 and resident transaction 3 is the same as that of resident transaction 1, and will not be repeated here.

[0172] When a blockchain node receives transaction data of a common transaction pushed by a client, it puts the transaction data into a common transaction group and starts a common contract process corresponding to the common transaction. The common contract process processes the transaction data of the common transaction to obtain the calculation result. The obtained calculation result is written into the common result queue.

[0173] When the size of the block to be generated reaches the configured threshold, the block to be generated is closed by the block generator, and the corresponding block hash, transaction Merkle tree and other block information are calculated to obtain the block to be verified, and the block to be verified is broadcast to other blockchain nodes for consensus.

[0174] In the embodiment of the present application, a dynamic block generation strategy is adopted to generate the block to be verified, that is, the blockchain node decides when to generate the block to be verified according to a preset block size threshold or time interval threshold. This can not only make full use of system resources, but also reduce the delay in transaction processing and improve the overall performance of the blockchain system.

[0175] In some embodiments, after the blocks to be generated are packaged into blocks to be verified, the blocks to be verified are broadcast to other blockchain nodes for consensus. The following describes the process of block consensus by taking the blockchain node that generates the blocks to be verified as the master node and other blockchain nodes as slave nodes. Figure 6 As shown, the following steps are included:

[0176] Step 601: The master node broadcasts the block to be verified to each slave node.

[0177] Step 602, the slave node verifies the calculation results corresponding to each of the multiple transaction identifiers in the block to be verified, and obtains multiple transaction verification results.

[0178] When the transaction identifier is a resident transaction identifier, the associated streaming data is stream-calculated through the corresponding resident contract process, and the obtained streaming calculation result is compared with the corresponding streaming calculation result in the block to be verified to obtain the transaction verification result.

[0179] Specifically, when the transaction identifier is a permanent transaction identifier, the slave node first determines whether there is a permanent contract process corresponding to the permanent transaction identifier. If not, the permanent contract process corresponding to the permanent transaction identifier is started, and then the subsequent transaction verification process is executed. Otherwise, the subsequent transaction verification process is executed directly.

[0180] The specific transaction verification process is as follows: the resident contract process obtains each streaming data associated with the resident transaction identifier from the block to be verified, and performs streaming calculations on each streaming data to obtain the corresponding streaming calculation results. Then, the obtained streaming calculation results are compared with the corresponding streaming calculation results in the block to be verified. If they are consistent, the transaction verification result of the resident transaction identifier is a successful transaction verification, otherwise the transaction verification result of the resident transaction identifier is determined to be a failed transaction verification.

[0181] When the transaction identifier is a common transaction identifier, the common contract process is started from the node, and the transaction data associated with the common transaction identifier is obtained from the block to be verified through the common contract process, and the transaction data is processed to obtain the corresponding transaction calculation result. The obtained transaction calculation result is compared with the corresponding transaction calculation result in the block to be verified; if they are consistent, the transaction verification result of the common transaction identifier is a successful transaction verification, otherwise it is determined that the transaction verification result of the common transaction identifier is a failed transaction verification.

[0182] It should be noted that the slave node can verify multiple resident transactions in parallel, multiple ordinary transactions in parallel, and resident transactions and ordinary transactions in parallel. This application does not make any specific limitations on this.

[0183] When the calculation results corresponding to all transaction identifiers in the block to be verified are verified, the block information such as the block hash and transaction Merkle tree of the block to be verified is verified.

[0184] Step 603: The slave node generates a block verification result of the block to be verified based on the multiple transaction verification results obtained.

[0185] Specifically, when the transaction verification results corresponding to all transaction identifiers in the block to be verified are successful transaction verification, and the verification result of the block information in the block to be verified is passed verification, the block verification result of the block to be verified is determined to be successful verification; otherwise, the block verification result of the block to be verified is determined to be failed verification.

[0186] Step 604, the slave node broadcasts the block verification result of the block to be verified.

[0187] Step 605, when the number of corresponding successfully verified block verification results received by the master node is greater than a preset threshold, the master node adds the block to be verified to the blockchain.

[0188] In the embodiment of the present application, a dynamic block generation strategy is adopted to generate the block to be verified, and the block to be verified is broadcast to other blockchain nodes for consensus. Other blockchain nodes use different contract processes for parallel verification of ordinary transactions and resident transactions to obtain the final block verification result and broadcast it, thereby improving the efficiency of block consensus and enhancing the overall performance of the blockchain system.

[0189] In order to better explain the embodiments of the present application, the following introduces a streaming data processing method based on blockchain provided by the embodiments of the present application in combination with the life cycle of the blockchain system solution, specifically including: blockchain streaming computing creation stage, streaming data push stage, blockchain streaming computing termination stage, blockchain node block execution stage and block verification stage.

[0190] See also Fig. 7A , which is a flowchart of the blockchain streaming computing creation phase provided in an embodiment of the present application, including the following steps:

[0191] Step 701: The client signs the streaming computing startup instruction and sends it to the blockchain node.

[0192] Among them, the streaming computing startup instruction carries a permanent transaction identifier.

[0193] In step 702, the blockchain node verifies whether the certificate signature of the streaming computing startup instruction is correct through the verification module. If so, execute step 703; otherwise, execute step 709.

[0194] In step 703, the blockchain node verifies through the verification module whether the authority of the initiator of the streaming computing start instruction is correct. If so, execute step 704; otherwise, execute step 710.

[0195] In step 704, the blockchain node determines whether the contract repository contains the smart contract corresponding to the resident transaction identifier. If so, execute step 705; otherwise, execute step 711.

[0196] Step 705: The blockchain node creates a streaming data queue corresponding to the resident transaction identifier in the streaming data pool.

[0197] Step 706: The blockchain node starts the resident contract process corresponding to the resident transaction identifier.

[0198] Step 707: The blockchain node starts a detection client in the resident contract process for real-time detection of new streaming data in the streaming data queue.

[0199] Step 708: The blockchain node returns a message indicating that the streaming computing has been successfully started to the client.

[0200] Step 709: The blockchain node returns a message indicating signature verification failure to the client.

[0201] Step 710: The blockchain node returns a message indicating that the permission verification failed to the client.

[0202] Step 711, the blockchain node returns a message to the client indicating that the smart contract does not exist.

[0203] Take the logistics tracking scenario as an example, see Figure 7B , the client sends a streaming computing startup instruction for the logistics tracking transaction to the blockchain node. The blockchain node verifies the certificate signature and permissions of the streaming computing startup instruction. When the verification is passed, it queries whether the smart contract corresponding to the logistics tracking transaction is included in the contract warehouse. If included, it builds a streaming data queue for the logistics tracking transaction and starts a resident contract process for the logistics tracking transaction. Then it returns a streaming computing startup success message to the client.

[0204] See also Fig. 8A , which is a flow chart of the streaming data push phase provided in an embodiment of the present application, comprising the following steps:

[0205] Step 801: The client encapsulates streaming data and a resident transaction identifier in a data push request.

[0206] A data push request can carry single streaming data or batch streaming data.

[0207] Step 802: The client signs the data push request and sends it to the blockchain node.

[0208] Step 803: The blockchain node verifies whether the certificate signature of the data push request is correct through the verification module. If so, execute step 804; otherwise, execute step 809.

[0209] In step 804, the blockchain node verifies whether the authority of the initiator of the data push request is correct through the verification module. If so, execute step 805, otherwise execute step 810.

[0210] In step 805, the blockchain node determines whether a streaming data queue corresponding to the resident transaction identifier exists. If so, step 806 is executed; if not, step 811 is executed.

[0211] In step 806, the blockchain node determines whether the streaming data queue corresponding to the resident transaction identifier has reached the storage threshold or is disabled. If so, execute step 812; if not, execute step 807.

[0212] Step 807: The blockchain node saves the streaming data in the data push request as a queue element in the streaming data queue.

[0213] Step 808: The blockchain node returns a message to the client indicating that the streaming data has been successfully added.

[0214] In step 809, the blockchain node determines that a signature verification failure occurs in the data push request, and then executes step 813.

[0215] In step 810, the blockchain node determines that the data push request has an error of failed permission verification, and then executes step 813.

[0216] In step 811, the blockchain node determines that an error occurs in the data push request that the streaming data queue does not exist, and then executes step 813.

[0217] In step 812, the blockchain node determines that an error occurs in the data push request that the streaming data queue is unavailable, and then executes step 813.

[0218] Step 813: The blockchain node returns a message to the client indicating that the streaming data addition failed.

[0219] Take the logistics tracking scenario as an example, see Figure 8B, the client sends a data push request for the logistics tracking transaction to the blockchain node. The data push request contains the order information (1234) and the latest logistics information (XX city) of commodity A. The blockchain node verifies the certificate signature and authority of the data push request. When the verification is passed, it queries whether the streaming data pool contains the streaming data queue corresponding to the logistics tracking transaction. If it is included and the streaming data queue is in an available state, the order information (1234) and the latest logistics information (XX city) are written to the end of the streaming data queue; then a message indicating that the streaming data has been successfully added is returned to the client.

[0220] See also Fig. 9A , which is a flowchart of the blockchain streaming computing termination phase provided in an embodiment of the present application, including the following steps:

[0221] Step 901: The client signs the streaming computing termination instruction and sends it to the blockchain node.

[0222] Among them, the streaming computing termination instruction carries a permanent transaction identifier.

[0223] Step 902: The blockchain node verifies whether the certificate signature of the streaming computing termination instruction is correct through the verification module. If so, execute step 903; otherwise, execute step 911.

[0224] Step 903, the blockchain node verifies through the verification module whether the authority of the initiator of the streaming computing termination instruction is correct. If so, execute step 904, otherwise execute step 912.

[0225] Step 904, the blockchain node determines whether the resident transaction identifier exists in the transaction pool. If so, execute step 905, otherwise execute step 913.

[0226] Step 905: The blockchain node disables the streaming data queue corresponding to the resident transaction identifier.

[0227] Step 906: The blockchain node starts counting down for the streaming data queue.

[0228] Step 907: When the countdown ends or the streaming data queue is cleared, the blockchain node deletes the streaming data queue.

[0229] Step 908: The blockchain node deletes the resident transaction identifier from the transaction pool.

[0230] Step 909: The blockchain node terminates the resident contract process corresponding to the resident transaction identifier.

[0231] Step 910: The blockchain node returns a message to the client indicating that the streaming computing has been successfully terminated.

[0232] In step 911, the blockchain node determines that a signature verification failure occurs in the streaming computing termination instruction, and then executes step 914.

[0233] In step 912, the blockchain node determines that the streaming computing termination instruction has an error of failed permission verification, and then executes step 914.

[0234] In step 913, the blockchain node determines that an error occurs in the streaming computing termination instruction that the transaction does not exist, and then executes step 914.

[0235] Step 914: The blockchain node returns a message to the client indicating that the streaming computing has failed to terminate.

[0236] Take the logistics tracking scenario as an example, see Fig. 9B , the client sends a streaming computing termination instruction for the logistics tracking transaction to the blockchain node. The blockchain node verifies the certificate signature and authority of the streaming computing termination instruction. When the verification is passed, it queries whether the logistics tracking transaction is included in the transaction pool. If so, the streaming data queue of the logistics tracking transaction is disabled and the countdown begins. At the end of the countdown, the logistics tracking transaction and the streaming data queue of the logistics tracking transaction are deleted, the resident contract process of the logistics tracking transaction is terminated, and the streaming computing termination success message is returned to the client.

[0237] See also Fig. 10A , which is a flowchart of the blockchain node block execution phase provided in an embodiment of the present application, including the following steps:

[0238] Step 1001, the blockchain node determines whether the size of the block to be generated reaches the configured threshold. If so, execute step 1002, otherwise execute step 1007.

[0239] Step 1002: The blockchain node notifies all started resident contract processes to stop writing the calculated streaming calculation results into the block to be generated.

[0240] Step 1003: The blockchain node writes the streaming calculation results obtained by calculation into the corresponding cache area or disk through the resident contract process.

[0241] Step 1004: The blockchain node writes the streaming computing results in the cache or disk in batches into the next block to be generated.

[0242] Step 1005, the blockchain node closes the block to be generated, and calculates the corresponding block information such as block hash, transaction Merkle tree, etc. to obtain the block to be verified.

[0243] Step 1006, the blockchain node broadcasts the block to be verified to other blockchain nodes for consensus.

[0244] Step 1007, the blockchain node determines whether the countdown to the block time has been reached. If so, execute step 1002, otherwise execute step 1008.

[0245] Step 1008, the blockchain nodes execute transactions in the transaction pool in parallel.

[0246] Step 1009, the blockchain node determines whether the executed transaction is a resident transaction, if so, executes step 1010, otherwise executes step 1013.

[0247] Step 1010: The blockchain node obtains the queue element in the corresponding streaming data queue through the resident contract process.

[0248] Step 1011: The blockchain node performs streaming calculations on the obtained queue elements through the resident contract process according to the smart contract corresponding to the resident transaction to obtain the streaming calculation results.

[0249] Step 1012: The blockchain node writes the streaming calculation result into the corresponding streaming result queue in the block to be generated through the resident contract process, and returns to execute step 1001.

[0250] Step 1013, the blockchain node starts a common contract process for a common transaction and executes the common transaction through the common contract process.

[0251] Step 1014, the blockchain node determines whether the ordinary transaction is executed successfully. If so, execute step 1016, otherwise execute step 1015.

[0252] Step 1015, the blockchain node records the execution result of the failed execution.

[0253] Step 1016, the blockchain node records the successful execution result.

[0254] Step 1017, the blockchain node stores the execution result in the block to be generated and returns to execute step 1001.

[0255] Take the logistics tracking scenario as an example, see Fig. 10B , the blockchain node obtains the queue elements in the corresponding streaming data queue (for example, the order information (1234) and the latest logistics information (XX city) of commodity A) through the resident contract process. According to the smart contract of the logistics tracking transaction, the obtained queue elements are stream-calculated to obtain the streaming calculation results, and the streaming calculation results are written into the block to be generated. When the countdown to the block time is reached, the block to be generated is closed, and the corresponding block hash, transaction Merkle tree and other block information are calculated to obtain the block to be verified. The block to be verified is broadcast to other blockchain nodes for consensus.

[0256] See also Fig.11A, is a flow chart of the verification block phase provided in an embodiment of the present application, specifically taking the blockchain node that generates the block to be verified as the master node and the other blockchain nodes as slave nodes for explanation, including the following steps:

[0257] Step 1101, the slave node receives the block to be verified broadcasted by the master node.

[0258] Step 1102, the slave node schedules and executes all transactions in the block to be verified.

[0259] Step 1103, the slave node determines whether the transaction identifier of the currently processed transaction is a resident transaction identifier, if so, execute step 1104, otherwise execute step 1113.

[0260] Step 1104, whether the resident contract process corresponding to the resident transaction identifier has been started in the slave node, if not, execute step 1105, otherwise execute step 1106.

[0261] Step 1105, start the resident contract process from the node corresponding to the resident transaction identifier, and then execute step 1106.

[0262] Step 1106, the slave node obtains each streaming data associated with the resident transaction identifier from the block to be verified through the resident contract process, and performs streaming calculations on each streaming data to obtain corresponding streaming calculation results.

[0263] Step 1107, the slave node determines whether the obtained streaming calculation result is consistent with the corresponding streaming calculation result in the block to be verified. If so, execute step 1108, otherwise execute step 1115.

[0264] Step 1108, the slave node determines whether all transactions have been verified. If so, execute step 1109, otherwise execute step 1102.

[0265] Step 1109, the slave node verifies the block information such as the block hash and transaction Merkle tree of the block to be verified.

[0266] Step 1110, the slave node determines whether the verification is successful, if so, execute step 1111, otherwise execute step 1112.

[0267] Step 1111, the slave node broadcasts the message of successful verification of the block to be verified to other nodes.

[0268] Step 1112, the slave node broadcasts the message that the verification of the block to be verified has failed to other nodes.

[0269] Step 1113, the slave node starts a common contract process for a common transaction, and executes the common transaction through the common contract process to obtain the transaction calculation result.

[0270] Step 1114, the slave node determines whether the obtained transaction calculation result is consistent with the corresponding transaction calculation result in the block to be verified. If so, execute step 1108, otherwise execute step 1115.

[0271] Step 1115, the slave node records the information of transaction verification failure.

[0272] Take the logistics tracking scenario as an example, see Fig. 11B , the slave node receives the block to be verified broadcast by the master node, and schedules the execution of all transactions in the block to be verified. For logistics tracking transactions, the corresponding resident contract process processes the associated streaming data to obtain the calculation results. For ordinary transactions, the ordinary contract process is started to execute the associated transaction data to obtain the calculation results.

[0273] All the calculation results obtained are compared with the corresponding calculation results in the block to be verified. When they are consistent, the block information such as block hash and transaction Merkle tree are verified. When the verification passes, the message of successful verification of the block to be verified is broadcast.

[0274] In the embodiment of the present application, when the blockchain node receives a streaming computing start instruction carrying a resident transaction identifier, it creates a streaming data queue corresponding to the resident transaction identifier and starts a resident contract process. Then, through the streaming data queue and the resident contract process, real-time and continuously generated streaming data is received, and streaming computing is performed on the streaming data, without the need for the client to first collect the streaming data and send the collected streaming data to the blockchain node for batch processing, thereby effectively improving the processing timeliness of the streaming data and reducing the delay in transaction processing.

[0275] Secondly, the resident contract process continuously processes the streaming data associated with the resident transaction and terminates only when it receives a streaming calculation termination instruction, avoiding the performance overhead caused by the frequent creation and termination of processes by blockchain nodes.

[0276] In addition, by introducing resident transactions, resident contract processes, and streaming data queues into the blockchain system, efficient processing of real-time and continuously generated streaming data can be achieved, enabling the blockchain system to better adapt to scenarios such as real-time data processing, real-time detection, and real-time analysis, expanding the application of blockchain in the field of real-time data processing.

[0277] The embodiments of the present application have a wide range of application scenarios in blockchain performance detection and optimization, fault diagnosis and troubleshooting, security auditing, visual analysis, and customer support, etc. This will help improve the performance, availability, security, and user experience of the blockchain system.

[0278] Based on the same technical concept, the embodiment of the present application provides a streaming data processing device based on blockchain, which is applied to blockchain nodes, such as Fig.12 As shown, the blockchain-based streaming data processing device 1200 includes:

[0279] The receiving unit 1201 is used to create a streaming data queue corresponding to the resident transaction identifier in response to the streaming computing start instruction carrying the resident transaction identifier, and store the streaming data associated with the resident transaction identifier received in real time as a queue element in the streaming data queue;

[0280] The processing unit 1202 is used to start a resident contract process corresponding to the resident transaction identifier, and through the resident contract process, whenever it is detected that the streaming data queue contains a queue element, perform streaming calculation on the queue element in the streaming data queue, wherein each time a streaming calculation result of a queue element is obtained by calculation, the streaming calculation result of the queue element is written into the block to be generated;

[0281] The termination unit 1203 is used to terminate the resident contract process in response to the streaming computing termination instruction carrying the resident transaction identifier.

[0282] Optionally, the receiving unit 1201 is specifically configured to:

[0283] receiving a streaming data push request sent by a client, the streaming data push request including at least: a resident transaction identifier and associated streaming data;

[0284] When the streaming data queue corresponding to the resident transaction identifier is in an available state, the associated streaming data is saved in the streaming data queue as a queue element.

[0285] Optionally, the processing unit 1202 is further configured to:

[0286] When the block to be generated meets the block generation conditions, notify the resident contract process to stop writing the calculated streaming calculation results into the block to be generated;

[0287] The blocks to be generated are packaged into blocks to be verified, and the blocks to be verified are broadcast to other blockchain nodes for consensus.

[0288] Optionally, the processing unit 1202 is further configured to:

[0289] After notifying the resident contract process to stop writing the calculated streaming calculation results into the block to be generated, it also includes:

[0290] The streaming calculation results obtained by calculation are written into the corresponding buffer area through the resident contract process;

[0291] Write the streaming computing results in the buffer area into the next block to be generated in batches.

[0292] Optionally, the processing unit 1202 is further configured to:

[0293] The block to be verified is broadcasted to other blockchain nodes, so that each blockchain node that receives the block to be verified verifies the calculation results corresponding to each of the multiple transaction identifiers in the block to be verified, obtains multiple transaction verification results, and generates a block verification result based on the multiple transaction verification results.

[0294] Optionally, the transaction verification result corresponding to the resident transaction identifier among multiple transaction identifiers is obtained by the blockchain node that receives the block to be verified performing streaming calculations on the associated streaming data through the corresponding resident contract process, and comparing the obtained streaming calculation results with the corresponding streaming calculation results in the block to be verified.

[0295] Optionally, the processing unit 1202 is further configured to:

[0296] Pack the blocks to be generated into blocks to be verified, and after broadcasting the blocks to be verified to other blockchain nodes for consensus, receive the block verification results of the blocks to be verified;

[0297] When the number of corresponding successfully verified block verification results is greater than a preset threshold, the block to be verified is added to the blockchain.

[0298] Optionally, the processing unit 1202 is further configured to:

[0299] Before creating a streaming data queue corresponding to the resident transaction identifier, the certificate signature of the streaming computing start instruction is verified, and the verification result is passed; and the authority of the initiator of the streaming computing start instruction is verified, and the verification result is passed.

[0300] Optionally, the processing unit 1202 is further configured to:

[0301] After creating a streaming data queue corresponding to the resident transaction identifier, add the resident transaction identifier to the transaction pool;

[0302] Before terminating the resident contract process corresponding to the resident transaction identifier, disable the streaming data queue corresponding to the resident transaction identifier;

[0303] After the streaming data in the streaming data queue is cleared, the streaming data queue is deleted;

[0304] Delete the resident transaction identifier from the transaction pool.

[0305] Optionally, the processing unit 1202 is further configured to:

[0306] Before disabling the streaming data queue corresponding to the resident transaction identifier, the certificate signature of the streaming computing termination instruction is verified, and the verification result is passed; and the authority of the initiator of the streaming computing termination instruction is verified, and the verification result is passed.

[0307] In the embodiment of the present application, when the blockchain node receives a streaming computing start instruction carrying a resident transaction identifier, a streaming data queue is created corresponding to the resident transaction identifier and a resident contract process is started. Then, through the streaming data queue and the resident contract process, real-time and continuously generated streaming data is received, and streaming computing is performed on the streaming data, without the need for the client to first collect the streaming data and send the collected streaming data to the blockchain node for processing, or wait for a fixed time interval or the transaction pool to reach a threshold before starting to process the transaction, thereby effectively improving the processing timeliness of the streaming data, reducing the delay in transaction processing, and at the same time improving the utilization of system resources and the overall performance of the blockchain system.

[0308] Secondly, the resident contract process continuously processes the streaming data associated with the resident transaction and terminates only when it receives a streaming calculation termination instruction, avoiding the performance overhead caused by the frequent creation and termination of processes by blockchain nodes.

[0309] In addition, by introducing resident transactions, resident contract processes, and streaming data queues into the blockchain system, efficient processing of real-time and continuously generated streaming data can be achieved, enabling the blockchain system to better adapt to scenarios such as real-time data processing, real-time detection, and real-time analysis, expanding the application of blockchain in the field of real-time data processing.

[0310] Based on the same technical concept, the embodiment of the present application provides a computer device, which can be Figure 1 The blockchain nodes shown are as follows Fig.13 As shown, it includes at least one processor 1301 and a memory 1302 connected to the at least one processor. The specific connection medium between the processor 1301 and the memory 1302 is not limited in the embodiment of the present application. Fig.13 For example, the processor 1301 and the memory 1302 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0311] In an embodiment of the present application, the memory 1302 stores instructions that can be executed by at least one processor 1301. The at least one processor 1301 can perform the steps of the above-mentioned blockchain-based streaming data processing method by executing the instructions stored in the memory 1302.

[0312] Among them, the processor 1301 is the control center of the computer device, and various interfaces and lines can be used to connect various parts of the computer device, and streaming data processing based on blockchain can be realized by running or executing instructions stored in the memory 1302 and calling data stored in the memory 1302. Optionally, the processor 1301 may include one or more processing units, and the processor 1301 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application program, etc., and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1301. In some embodiments, the processor 1301 and the memory 1302 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0313] Processor 1301 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware processor to be executed, or can be executed by a combination of hardware and software modules in the processor.

[0314] Memory 1302 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. Memory 1302 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), programmable read-only memory (Programmable Read Only Memory, PROM), read-only memory (Read Only Memory, ROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic memory, disk, optical disk, etc. Memory 1302 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer device, but is not limited thereto. The memory 1302 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0315] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program that can be executed by a computer device. When the program runs on the computer device, the computer device executes the steps of the above-mentioned blockchain-based streaming data processing method.

[0316] Based on the same inventive concept, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer device, the computer device executes the steps of the above-mentioned blockchain-based streaming data processing method.

[0317] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0318] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer device or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0319] These computer program instructions may also be stored in a computer readable memory capable of directing a computer device or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0320] These computer program instructions can also be loaded onto a computer device or other programmable data processing device so that a series of operation steps are executed on the computer device or other programmable device to produce a process implemented by the computer device, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0321] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0322] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A streaming data processing method based on blockchain, applied to blockchain nodes, characterized in that: include: In response to a streaming computing start instruction carrying a resident transaction identifier, a streaming data queue is created corresponding to the resident transaction identifier, and streaming data associated with the resident transaction identifier received in real time is stored as a queue element in the streaming data queue; A resident contract process is started corresponding to the resident transaction identifier, and through the resident contract process, whenever a queue element is detected in the streaming data queue, a streaming calculation is performed on the queue element in the streaming data queue, wherein each time a streaming calculation result of a queue element is obtained by calculation, the streaming calculation result of the queue element is written into the block to be generated; In response to a streaming computing termination instruction carrying the resident transaction identifier, the resident contract process is terminated.

2. The method according to claim 1, characterized in that The step of storing the real-time received streaming data associated with the resident transaction identifier as a queue element in the streaming data queue includes: Receiving a streaming data push request sent by a client, wherein the streaming data push request at least includes: the resident transaction identifier and associated streaming data; When the streaming data queue corresponding to the resident transaction identifier is in an available state, the associated streaming data is stored in the streaming data queue as a queue element.

3. The method according to claim 1, characterized in that Also includes: When the block to be generated meets the block generation condition, the resident contract process is notified to stop writing the calculated streaming calculation results into the block to be generated; The blocks to be generated are packaged into blocks to be verified, and the blocks to be verified are broadcast to other blockchain nodes for consensus.

4. The method according to claim 3, characterized in that After notifying the resident contract process to stop writing the calculated streaming calculation results into the block to be generated, the process further includes: The stream computing result obtained by the calculation is written into the corresponding buffer area through the resident contract process; The streaming computing results in the buffer area are written in batches into the next block to be generated.

5. The method according to claim 3, characterized in that The process of broadcasting the block to be verified to other blockchain nodes for consensus includes: The block to be verified is broadcasted to other blockchain nodes, so that each blockchain node that receives the block to be verified verifies the calculation results corresponding to each of the multiple transaction identifiers in the block to be verified, obtains multiple transaction verification results, and generates a block verification result based on the multiple transaction verification results.

6. The method according to claim 5, characterized in that The transaction verification result corresponding to the resident transaction identifier among the multiple transaction identifiers is obtained by the blockchain node that receives the block to be verified performing streaming calculations on the associated streaming data through the corresponding resident contract process, and comparing the obtained streaming calculation results with the corresponding streaming calculation results in the block to be verified.

7. The method according to claim 5, characterized in that After packaging the blocks to be generated into blocks to be verified and broadcasting the blocks to be verified to other blockchain nodes for consensus, the process further includes: Receiving a block verification result of the block to be verified; When the number of corresponding successfully verified block verification results is greater than a preset threshold, the block to be verified is added to the blockchain.

8. The method according to any one of claims 1 to 7, characterized in that: Before creating a streaming data queue corresponding to the resident transaction identifier, the method further includes: The certificate signature of the streaming computing startup instruction is verified, and the verification result is that the verification is passed; and the authority of the initiator of the streaming computing startup instruction is verified, and the verification result is that the verification is passed.

9. The method according to any one of claims 1 to 7, characterized in that: After creating a streaming data queue corresponding to the resident transaction identifier, the method further includes: Adding the resident transaction identifier to the transaction pool; Before terminating the resident contract process corresponding to the resident transaction identifier, the method further includes: disabling the streaming data queue corresponding to the resident transaction identifier; After the streaming data in the streaming data queue is cleared, deleting the streaming data queue; The resident transaction identifier is deleted from the transaction pool.

10. The method according to claim 9, characterized in that Before disabling the streaming data queue corresponding to the resident transaction identifier, the method further includes: The certificate signature of the streaming computing termination instruction is verified, and the verification result is that the verification is passed; and the authority of the initiator of the streaming computing termination instruction is verified, and the verification result is that the verification is passed.

11. A streaming data processing device based on blockchain, applied to a blockchain node, characterized in that: include: A receiving unit, configured to, in response to a streaming computing start instruction carrying a resident transaction identifier, create a streaming data queue corresponding to the resident transaction identifier, and store the streaming data associated with the resident transaction identifier received in real time as a queue element in the streaming data queue; a processing unit, configured to start a resident contract process corresponding to the resident transaction identifier, and, through the resident contract process, perform streaming calculation on the queue elements in the streaming data queue whenever it is detected that the streaming data queue contains a queue element, wherein each time a streaming calculation result of a queue element is obtained by calculation, the streaming calculation result of the queue element is written into the block to be generated; A termination unit is used to terminate the resident contract process in response to a streaming computing termination instruction carrying the resident transaction identifier.

12. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of any one of the methods of claims 1 to 10 are implemented.

13. A computer-readable storage medium, characterized in that: It stores a computer program executable by a computer device. When the program is run on the computer device, the computer device executes the steps of any method described in claims 1 to 10.

14. A computer program product, characterized in that The computer program product comprises a computer program stored on a computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer device, the computer device is caused to execute the steps of the method according to any one of claims 1 to 10.

Images