Data output method and device, computer equipment and storage medium

Abstract

The invention relates to a data output method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring at least one piece of transaction data from a distributed database node; the transaction data comprises node transactions, transaction time and transaction identifiers; for each piece of transaction data, generating a key value pair corresponding to the transaction data according to a node transaction, transaction time and a transaction identifier in the transaction data; the key value pair comprises a target key and a target value; and for each key-value pair, generating and outputting a transactional structured query language (SQL) statement of the key-value pair according to the key-value pair. By adopting the method, the parallel capability of the distributed database can be effectively utilized, and the high efficiency and consistency of data operation are ensured, so that potential errors are reduced, and the accuracy of transaction data processing is improved.

Description

Technical Field

[0001] This application relates to the field of cloud computing technology, and in particular, to a data output method, apparatus, computer device, and storage medium. Background Art

[0002] With the continuous growth of data storage and processing requirements, traditional stand-alone databases can no longer meet the application requirements of large-scale and high concurrency. To solve this problem, distributed database technology based on multiple servers has emerged.

[0003] In traditional technologies, the change data capture technology (CDC) is usually used to continuously monitor the change logs in the database, capture insert, delete, and update operations, and stream the change transactions to other systems.

[0004] Although this method can complete the processing of distributed transactions in a distributed database, since the change data capture technology cannot fully match the architecture and characteristics of the distributed database, there are usually problems with poor transaction processing consistency and real-time performance. Summary of the Invention

[0005] Based on this, it is necessary to provide a data output method, apparatus, computer device, and storage medium that can improve transaction processing consistency and real-time performance in view of the above technical problems.

[0006] In a first aspect, this application provides a data output method, including:

[0007] Obtain at least one transaction data from a distributed database node; the transaction data includes a node transaction, a transaction time, and a transaction identifier;

[0008] For each transaction data, generate a key-value pair corresponding to the transaction data according to the node transaction, transaction time, and transaction identifier in the transaction data; the key-value pair includes a target key and a target value;

[0009] For each key-value pair, generate and output a transaction structured query SQL statement for the key-value pair.

[0010] In one embodiment, generating a key-value pair corresponding to the transaction data according to the node transaction, transaction time, and transaction identifier in the transaction data includes:

[0011] Generate the target value in the key-value pair according to the operation type and operation target included in the node transaction; the operation type is at least one of an insert operation, a delete operation, and an update operation; and,

[0012] Combine the transaction time and the transaction identifier to obtain the target key in the key-value pair;

[0013] Generate key-value pairs corresponding to transaction data according to the target value and the target key.

[0014] In one embodiment, generate and output a transaction structured query SQL statement for the key-value pairs, including:

[0015] Sort the key-value pairs according to the target key in each key-value pair to obtain an ordered list;

[0016] Traverse each key-value pair in the ordered list and generate a transaction SQL statement for the key-value pair according to the target value in the key-value pair;

[0017] Output each transaction SQL statement according to the same situation of the target key in each key-value pair.

[0018] In one embodiment, sort the key-value pairs according to the target key in each key-value pair to obtain an ordered list, including:

[0019] Sort the key-value pairs according to the transaction time in the target key;

[0020] In the case of the same transaction time, sort the key-value pairs according to the transaction identifier in the target key.

[0021] In one embodiment, output a transaction structured query SQL statement for the key-value pairs, including:

[0022] Output each transaction SQL statement according to the same situation of the target key in each key-value pair.

[0023] In one embodiment, output each transaction SQL statement according to the same situation of the target key in each key-value pair, including:

[0024] Merge the transaction SQL statements corresponding to the key-value pairs with the same target key and output the merged transaction SQL statement;

[0025] For the key-value pairs with different target keys, output the transaction SQL statements corresponding to the key-value pairs.

[0026] In one embodiment, obtain at least one transaction data from the distributed database nodes, including:

[0027] Obtain at least one node data from the distributed database nodes;

[0028] Select the node data whose transaction time is equal to or later than the preset timestamp as candidate data;

[0029] Select the candidate data whose node transaction is a data change type transaction as transaction data.

[0030] Second aspect, the present application further provides a data output device, including:

[0031] A data acquisition module, configured to acquire at least one transaction data from a distributed database node; the transaction data includes node transactions, transaction time, and transaction identifiers;

[0032] A key-value generation module, configured to generate a key-value pair corresponding to each transaction data according to the node transactions, transaction time, and transaction identifiers in the transaction data; the key-value pair includes a target key and a target value;

[0033] A statement generation module, configured to generate and output a transaction structured query SQL statement for each key-value pair according to the key-value pair.

[0034] Third aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0035] Acquire at least one transaction data from a distributed database node; the transaction data includes node transactions, transaction time, and transaction identifiers;

[0036] For each transaction data, generate a key-value pair corresponding to the transaction data according to the node transactions, transaction time, and transaction identifiers in the transaction data; the key-value pair includes a target key and a target value;

[0037] For each key-value pair, generate and output a transaction structured query SQL statement for the key-value pair according to the key-value pair.

[0038] Fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0039] Acquire at least one transaction data from a distributed database node; the transaction data includes node transactions, transaction time, and transaction identifiers;

[0040] For each transaction data, generate a key-value pair corresponding to the transaction data according to the node transactions, transaction time, and transaction identifiers in the transaction data; the key-value pair includes a target key and a target value;

[0041] For each key-value pair, generate and output a transaction structured query SQL statement for the key-value pair according to the key-value pair.

[0042] Fifth aspect, the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0043] Obtain at least one transaction data from the distributed database nodes; the transaction data includes node transactions, transaction time, and transaction identifiers;

[0044] For each transaction data, generate a key-value pair corresponding to the transaction data according to the node transaction, transaction time, and transaction identifier in the transaction data; the key-value pair includes a target key and a target value;

[0045] For each key-value pair, generate and output a transaction structured query SQL statement for the key-value pair according to the key-value pair.

[0046] The above data output method, device, computer device, and storage medium obtain at least one transaction data from the distributed database nodes; the transaction data includes node transactions, transaction time, and transaction identifiers; for each transaction data, generate a key-value pair corresponding to the transaction data according to the node transaction, transaction time, and transaction identifier in the transaction data; the key-value pair includes a target key and a target value; for each key-value pair, generate and output a transaction structured query SQL statement for the key-value pair according to the key-value pair. In this embodiment, by constructing key-value pairs corresponding to transaction data, parallel execution of each node transaction, and outputting the SQL statement corresponding to the key-value pair, the parallel capabilities of the distributed database are effectively utilized to ensure the efficiency and consistency of data operations, thereby reducing potential errors and improving the accuracy of transaction data processing. Description of the Drawings

[0047] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0048] Figure 1 It is an application environment diagram of a data output method provided in this embodiment;

[0049] Figure 2 It is a flowchart of the first data output method provided in this embodiment;

[0050] Figure 3 It is a flowchart of a statement generation step provided in this embodiment;

[0051] Figure 4 It is a flowchart of the second data output method provided in this embodiment;

[0052] Figure 5 It is a structural block diagram of a data output device provided in this embodiment;

[0053] Figure 6The internal structure diagram of a computer device provided in this embodiment. Detailed implementation manners

[0054] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0055] The data output method provided by the embodiments of the present application can be applied to, for example Figure 1 the application environment shown in the figure. Among them, the terminal 102 communicates with the server 104 through a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or can be placed in the cloud or other network servers. The computer device obtains at least one transaction data from the distributed database node; the transaction data includes node transactions, transaction time, and transaction identifiers; for each transaction data, according to the node transactions, transaction time, and transaction identifiers in the transaction data, a key-value pair corresponding to the transaction data is generated; the key-value pair includes a target key and a target value; for each key-value pair, according to the key-value pair, a transaction structured query SQL statement of the key-value pair is generated and output. Among them, the computer device can be either the terminal 102 or the server 104. The terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0056] In an exemplary embodiment, as Figure 2 shown in the figure, a data output method is provided. Taking the computer device in Figure 1 as an example, the method includes the following steps 201 to step 203. Among them:

[0057] Step 201, obtain at least one transaction data from the distributed database node.

[0058] Among them, the distributed database includes a coordinator node (CN) and data nodes (DNs). The coordinator node is used to provide an external interface, receive query requests, and perform operations such as optimization of Structured Query Language (SQL), generation of execution plans, and integration of data. The data nodes are used to store shards of business data. Usually, each data node stores part of the business data in the form of tables. The data nodes usually execute the execution requests distributed by the coordinator node and return the execution results. The distributed database nodes can be understood as each data node in the distributed database that stores business data. It should be noted that by centrally managing the execution results returned by the data nodes through the coordinator node, the monitoring and management capabilities of the distributed database are enhanced.

[0059] Among them, transaction data can be understood as data for a transaction to be executed. Exemplarily, the transaction data can be a Write Ahead Log (WAL). The WAL log can be understood as a log for storing the executed transactions and operations in the database. The WAL log records the transaction time and transaction identifier as the basis for transaction visibility, and the executed SQL statements serve as the basis for subsequent replay. The transaction data includes node transactions, transaction time, and transaction identifiers. Node transactions can be understood as the execution content of a transaction, such as table name, operation type, operation target, primary key value, and column value changes. The operation type is at least one of insert operation, delete operation, and update operation.

[0060] Among them, the transaction time can be understood as the timestamp for recording transaction data (WAL log). It should be noted that the transaction time can be determined by a Global Transaction Manager (GTM). The global transaction manager is a mechanism in the distributed database system that maintains global timestamps. The global transaction manager is responsible for providing a global logical clock for each data node in the distributed system. Whenever a data node needs to record the timestamp of a node transaction, it requests a globally unique timestamp value from the global transaction manager. Since the clocks of each data node in the distributed database may deviate, through the global transaction manager, the consistency of the global timing relationship can be maintained to ensure the correctness and consistency of the stored data.

[0061] Among them, the transaction identifier can be understood as an identifier used to assign a globally unique identifier to a node transaction. Exemplarily, the transaction identifier can be a Global Transaction Identifier (GTID). The transaction identifier can be composed of the storage node identifier (Identity document, ID) where the node transaction occurs and the incremental sequence number on the storage node. Since in a distributed database, the stored data is distributed on multiple storage nodes, each node transaction can be identified through the transaction identifier, thereby ensuring the consistency and reliability of transaction data.

[0062] One optional implementation is as follows: For each distributed database node, at least one piece of transaction data of the distributed database node is read from a storage medium (such as a disk), and the transaction data is loaded into memory for subsequent processing.

[0063] Another optional implementation is as follows: At least one piece of node data is obtained from a distributed database node; the node data whose transaction time is equal to or later than a preset timestamp is selected as candidate data; the candidate data whose node transaction is a data change type transaction is selected as transaction data.

[0064] Exemplarily, for each distributed database node, at least one piece of node data of the distributed database node is read from a storage medium (such as a disk); a preset timestamp (such as the current global timestamp) is obtained, and at the same time, the corresponding transaction point position (such as the WAL position) on the storage node is parsed and jumped to this transaction point position, and this transaction point position is used as the CDC starting point; the node data whose transaction time is at and after this CDC starting point is selected as candidate data; it is determined whether the node transaction of the candidate data contains a data change operation; if it contains, it proves that the candidate data is a data change type transaction, and the candidate data is further selected as transaction data; otherwise, it proves that the candidate data is not a data change type transaction, and the selection of this candidate data is prohibited. Among them, the CDC starting point indicates that data is incrementally synchronized starting from this timestamp. That is to say, the WAL logs that have been processed before this timestamp can be directly skipped and do not need to be saved continuously, which can reduce the subsequent data processing burden and further increase the storage space.

[0065] It should be noted that in this embodiment, the transaction data can be further parsed to extract detailed information about data changes (such as table names, primary key values, changes in column values, etc.) from the node transactions of the transaction data, and output in a specified format (such as the protobuf format for structured data storage).

[0066] Step 202, for each piece of transaction data, generate a key-value pair corresponding to the transaction data according to the node transaction, transaction time, and transaction identifier in the transaction data.

[0067] Among them, the key-value pair includes a target key and a target value.

[0068] One optional implementation is as follows: for each transaction data, store the node transaction, transaction time, and transaction identifier in the transaction data in a key-value data structure to obtain the key-value pair corresponding to the transaction data. Among them, the target key is composed of the transaction time and the transaction identifier (such as GTM-GTID), and the target value is information included in the node transaction such as the table name, the primary key value, and the change of the column value.

[0069] Another optional implementation is as follows: generate the target value in the key-value pair according to the operation type and operation target included in the node transaction; and combine the transaction time and the transaction identifier to obtain the target key in the key-value pair; generate the key-value pair corresponding to the transaction data according to the target value and the target key.

[0070] Specifically, parse the operation type and operation target from the node transaction; use the operation type and operation target as the target value of the key-value pair; at the same time, splice and combine the transaction time and the transaction identifier, and use the splicing and combination result as the target key in the key-value pair; store the target value and the target key in the key-value data structure to obtain the key-value pair corresponding to the transaction data.

[0071] It should be noted that in this embodiment, the MapReduce algorithm can also be used. First, slice each transaction data and assign a processing node to each slice; for each slice in each processing node, read the node transaction, transaction time, and transaction identifier of the transaction data; map the node transaction, transaction time, and transaction identifier in the transaction data into the key-value pair of the transaction data; merge and output the key-value pairs of the transaction data processed by the left and right processing nodes. Among them, MapReduce is an algorithm for processing parallel computing, which mainly includes dividing the input transaction data set (a set containing multiple transaction data) into smaller transaction data subsets; parallelly processing each transaction data subset by the Map function to generate the key-value pair corresponding to each transaction data; and then summarizing and merging the key-value pairs corresponding to each transaction data by the Reduce function to complete the aggregation process, so as to obtain the final output result. The advantage of such a setting is that by processing transaction data through MapReduce, the processing efficiency and scalability of transaction data are improved.

[0072] Step 203, for each key-value pair, generate and output the transaction structured query SQL statement of the key-value pair.

[0073] Specifically, for each key-value pair, according to the information such as the operation type, operation target, table name, primary key value, and list changes in the target value of the key-value pair, a transaction-structured SQL statement for this key-value pair is combined and generated, and this transaction SQL statement is added to the transaction SQL statement list, and the transaction SQL statement is output.

[0074] The above data output method obtains at least one transaction data from the distributed database nodes; the transaction data includes node transactions, transaction times, and transaction identifiers; for each transaction data, according to the node transactions, transaction times, and transaction identifiers in the transaction data, key-value pairs corresponding to the transaction data are generated; the key-value pairs include target keys and target values; for each key-value pair, according to the key-value pair, a transaction-structured query SQL statement for the key-value pair is generated and output. In this embodiment, by constructing key-value pairs corresponding to transaction data and executing each node transaction in parallel, and outputting the transaction SQL statement corresponding to the key-value pair, the parallel ability of the distributed database is effectively utilized to ensure the efficiency and consistency of data operations, thereby reducing potential errors and improving the accuracy of transaction data processing.

[0075] Figure 3 It is a flow diagram of the statement generation step in an embodiment. In this embodiment, the step of generating and outputting a transaction-structured query SQL statement for a key-value pair in the above embodiment is refined, including the following steps:

[0076] Step 301, sort each key-value pair according to the target key in each key-value pair to obtain an ordered list.

[0077] Specifically, in this embodiment, each key-value pair can be sorted according to the transaction time of the target key in each key-value pair to obtain an ordered list; alternatively, each key-value pair can be sorted according to the transaction identifier of the target key in each key-value pair to obtain an ordered list.

[0078] In some embodiments, each key-value pair can also be sorted according to the transaction time in the target key; in the case where the transaction times are the same, each key-value pair is sorted according to the transaction identifier in the target key.

[0079] Specifically, each key-value pair is preferably sorted according to the transaction time in the target key; for key-value pairs with the same transaction time, each key-value pair is sorted according to the storage node ID corresponding to the transaction identifier in the target key; for key-value pairs with the same storage node ID corresponding to the transaction identifier, finally, each key-value pair is sorted according to the incrementing serial number corresponding to the transaction identifier in the target key to obtain an ordered list. The advantage of such a setting is that it can ensure the consistency and integrity of transaction data, and at the same time ensure the atomicity and orderliness of transaction data recovery.

[0080] Step 302: Traverse each key-value pair in the ordered list, and generate a transaction SQL statement for the key-value pair according to the target value in the key-value pair.

[0081] Specifically, for each key-value pair in the ordered list, extract information such as the table name, primary key value, operation type, operation target, and column value change of the target value in the key-value pair, combine them to obtain the transaction SQL statement for the key-value pair. At the same time, the transaction SQL statement can also be added to the transaction SQL statement list.

[0082] Step 303: Output each transaction SQL statement according to the same situation of the target keys in each key-value pair.

[0083] One optional implementation method is to merge the key-value pairs with the same target key into one transaction execution according to the same situation of the target keys in each key-value pair, and output each transaction SQL statement.

[0084] Another optional implementation method is to merge the transaction SQL statements corresponding to the key-value pairs with the same target key, and output the merged transaction SQL statements; for the key-value pairs with different target keys, output the transaction SQL statements corresponding to the key-value pairs.

[0085] Specifically, for the key-value pairs with the same target key, merge each key-value pair to obtain a merged transaction, and output the transaction SQL statement of the merged transaction; for the key-value pairs with different target keys, directly output the transaction SQL statements corresponding to each key-value pair.

[0086] In the above embodiment, according to the target keys in each key-value pair, sort each key-value pair to obtain an ordered list, traverse each key-value pair in the ordered list, generate a transaction SQL statement for the key-value pair according to the target value in the key-value pair, and output each transaction SQL statement according to the same situation of the target keys in each key-value pair. Sorting each key-value pair by the target key ensures the orderliness and integrity of the transaction data, reduces potential errors. At the same time, according to the same situation of each target key, output the SQL statements in the order of the ordered list, simply and efficiently utilize the parallel ability of the distributed database, further ensure the orderliness and consistency of data operations, and support breakpoint resumption to improve the system robustness and ensure data is not lost.

[0087] In one embodiment, this embodiment gives an optional way of data output, and takes the application of this method to a server as an example for illustration. As Figure 4 shown, this method includes the following steps:

[0088] Step 401: Obtain at least one node data from the distributed database nodes.

[0089] Step 402: Select the node data whose transaction time is equal to or later than the preset timestamp as the candidate data.

[0090] Step 403: Select the candidate data whose node transactions are data change type transactions as transaction data.

[0091] Among them, the transaction data includes node transactions, transaction time, and transaction identification.

[0092] Step 404: For each transaction data, generate the target value in the key-value pair according to the operation type and operation target included in the node transaction, and combine the transaction time and transaction identification to obtain the target key in the key-value pair.

[0093] Among them, the operation type is at least one of insert operation, delete operation, and modification operation.

[0094] Step 405: Generate the key-value pair corresponding to the transaction data according to the target value and target key.

[0095] Step 406: Sort each key-value pair according to the transaction time in the target key.

[0096] Step 407: When the transaction times are the same, sort each key-value pair according to the transaction identification in the target key.

[0097] Step 408: Traverse each key-value pair in the ordered list, and generate the transaction SQL statement of the key-value pair according to the target value in the key-value pair.

[0098] Step 409: Merge the transaction SQL statements corresponding to the key-value pairs with the same target key, and output the merged transaction SQL statement; for the key-value pairs with different target keys, output the transaction SQL statements corresponding to the key-value pairs.

[0099] It should be understood that although the steps in the flowcharts involved in the above-mentioned embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0100] Based on the same inventive concept, an embodiment of the present application further provides a data output device for implementing the data output method involved above. The solution provided by this device for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the data output device provided below can refer to the limitations on the data output method in the foregoing, and will not be elaborated here.

[0101] In an exemplary embodiment, as Figure 5 shown, a data output device is provided, including: a data acquisition module 10, a key-value generation module 11, and a statement generation module 12, where:

[0102] The data acquisition module 10 is configured to acquire at least one transaction data from a distributed database node; the transaction data includes a node transaction, a transaction time, and a transaction identifier;

[0103] The key-value generation module 11 is configured to generate a key-value pair corresponding to the transaction data for each transaction data according to the node transaction, the transaction time, and the transaction identifier in the transaction data; the key-value pair includes a target key and a target value;

[0104] The statement generation module 12 is configured to generate and output a transaction structured query SQL statement of the key-value pair for each key-value pair according to the key-value pair.

[0105] In one embodiment, the key-value generation module 11 is further configured to generate the target value in the key-value pair according to the operation type and the operation target included in the node transaction; the operation type is at least one of an insert operation, a delete operation, and a modification operation; and, combine the transaction time and the transaction identifier to obtain the target key in the key-value pair; generate a key-value pair corresponding to the transaction data according to the target value and the target key.

[0106] In one embodiment, the statement generation module 12 includes:

[0107] A sorting unit, configured to sort each key-value pair according to the target key in each key-value pair to obtain an ordered list;

[0108] A generation unit, configured to traverse each key-value pair in the ordered list and generate a transaction SQL statement of the key-value pair according to the target value in the key-value pair;

[0109] An output unit, configured to output each transaction SQL statement according to the same situation of the target key in each key-value pair.

[0110] In one embodiment, the sorting unit is further configured to sort each key-value pair according to the transaction time in the target key; in the case where the transaction times are the same, sort each key-value pair according to the transaction identifier in the target key.

[0111] In one embodiment, the output unit is further configured to merge the transaction SQL statements corresponding to the key-value pairs with the same target key, and output the merged transaction SQL statements; for the key-value pairs with different target keys, output the transaction SQL statements corresponding to the key-value pairs.

[0112] In one embodiment, the data acquisition module 10 is further configured to obtain at least one node data from the distributed database nodes; select the node data whose transaction time is equal to or later than the preset time stamp as the candidate data; and select the candidate data whose node transaction is a data change type transaction as the transaction data.

[0113] Each module in the above data output device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0114] In an exemplary embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 6 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a data output method.

[0115] Those skilled in the art can understand that Figure 6 the structure shown in

[0116] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0117] Obtain at least one transaction data from a distributed database node; the transaction data includes node transactions, transaction times, and transaction identifiers;

[0118] For each transaction data, generate a key-value pair corresponding to the transaction data according to the node transaction, transaction time, and transaction identifier in the transaction data; the key-value pair includes a target key and a target value;

[0119] For each key-value pair, generate and output a transaction structured query SQL statement for the key-value pair according to the key-value pair.

[0120] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0121] Generate the target value in the key-value pair according to the operation type and operation target included in the node transaction; the operation type is at least one of an insert operation, a delete operation, and a modification operation; and,

[0122] Combine the transaction time and the transaction identifier to obtain the target key in the key-value pair;

[0123] Generate a key-value pair corresponding to the transaction data according to the target value and the target key.

[0124] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0125] Sort each key-value pair according to the target key in each key-value pair to obtain an ordered list;

[0126] Traverse each key-value pair in the ordered list, and generate a transaction SQL statement for the key-value pair according to the target value in the key-value pair;

[0127] Output each transaction SQL statement according to the same situation of the target keys in each key-value pair.

[0128] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0129] Sort each key-value pair according to the transaction time in the target key;

[0130] When the transaction times are the same, sort each key-value pair according to the transaction identifier in the target key.

[0131] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0132] Merge the transaction SQL statements corresponding to the key-value pairs with the same target key, and output the merged transaction SQL statement;

[0133] For the key-value pairs with different target keys, output the transaction SQL statements corresponding to the key-value pairs.

[0134] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0135] Obtain at least one piece of node data from the distributed database nodes;

[0136] Select the node data whose transaction time is equal to or later than the preset timestamp as candidate data;

[0137] Select the candidate data whose node transaction is a data change type transaction as transaction data.

[0138] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the following steps are implemented:

[0139] Obtain at least one piece of transaction data from the distributed database nodes; the transaction data includes node transactions, transaction times, and transaction identifiers;

[0140] For each piece of transaction data, generate a key-value pair corresponding to the transaction data according to the node transaction, transaction time, and transaction identifier in the transaction data; the key-value pair includes a target key and a target value;

[0141] For each key-value pair, generate and output a transaction structured query SQL statement for the key-value pair according to the key-value pair.

[0142] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0143] Generate the target value in the key-value pair according to the operation type and operation target included in the node transaction; the operation type is at least one of an insert operation, a delete operation, and a modification operation; and,

[0144] Combine the transaction time and the transaction identifier to obtain the target key in the key-value pair;

[0145] Generate a key-value pair corresponding to the transaction data according to the target value and the target key.

[0146] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0147] Sort the key-value pairs according to the target keys in each key-value pair to obtain an ordered list;

[0148] Traverse each key-value pair in the ordered list, and generate a transaction SQL statement for the key-value pair according to the target value in the key-value pair;

[0149] Output each transaction SQL statement according to the same situation of the target keys in each key-value pair.

[0150] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0151] Sort each key-value pair according to the transaction time in the target key;

[0152] When the transaction times are the same, sort each key-value pair according to the transaction identifier in the target key.

[0153] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0154] Merge the transaction SQL statements corresponding to the key-value pairs with the same target key, and output the merged transaction SQL statements;

[0155] For the key-value pairs with different target keys, output the transaction SQL statements corresponding to the key-value pairs.

[0156] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0157] Obtain at least one node data from the distributed database nodes;

[0158] Select the node data whose transaction time is equal to or later than the preset timestamp as the candidate data;

[0159] Select the candidate data whose node transaction is a data change type transaction as the transaction data.

[0160] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0161] Obtain at least one transaction data from the distributed database nodes; the transaction data includes node transaction, transaction time, and transaction identifier;

[0162] For each transaction data, generate a key-value pair corresponding to the transaction data according to the node transaction, transaction time, and transaction identifier in the transaction data; the key-value pair includes a target key and a target value;

[0163] For each key-value pair, generate and output the transaction structured query SQL statement of the key-value pair.

[0164] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0165] Generate the target value in the key-value pair according to the operation type and operation target included in the node transaction; the operation type is at least one of an insert operation, a delete operation, and a modification operation; and,

[0166] Combine the transaction time and the transaction identifier to obtain the target key in the key-value pair;

[0167] Generate key-value pairs corresponding to the transaction data according to the target value and the target key.

[0168] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0169] Sort the key-value pairs according to the target key in each key-value pair to obtain an ordered list;

[0170] Traverse each key-value pair in the ordered list, and generate a transaction SQL statement for the key-value pair according to the target value in the key-value pair;

[0171] Output each transaction SQL statement according to the same situation of the target key in each key-value pair.

[0172] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0173] Sort the key-value pairs according to the transaction time in the target key;

[0174] In the case of the same transaction time, sort the key-value pairs according to the transaction identifier in the target key.

[0175] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0176] Merge the transaction SQL statements corresponding to the key-value pairs with the same target key, and output the merged transaction SQL statements;

[0177] For the key-value pairs with different target keys, output the transaction SQL statements corresponding to the key-value pairs.

[0178] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0179] Obtain at least one node data from the distributed database nodes;

[0180] Select the node data whose transaction time is equal to or later than the preset time stamp as the candidate data;

[0181] Select the candidate data whose node transaction is a data change type transaction as the transaction data.

[0182] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0183] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0184] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0185] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A data output method, characterized in that: The method comprises: Acquire at least one transaction data from a distributed database node; the transaction data includes a node transaction, a transaction time, and a transaction identifier; For each transaction data, generating a key-value pair corresponding to the transaction data according to the node transaction, the transaction time and the transaction identifier in the transaction data; the key-value pair includes a target key and a target value; For each key-value pair, a transaction structured query SQL statement of the key-value pair is generated and output according to the key-value pair.

2. The method according to claim 1, characterized in that Generating a key-value pair corresponding to the transaction data according to the node transaction, the transaction time and the transaction identifier in the transaction data includes: generating a target value in the key-value pair according to an operation type and an operation target included in the node transaction; the operation type is at least one of an insert operation, a delete operation, and a modify operation; and, Combining the transaction time and the transaction identifier to obtain a target key in the key-value pair; A key-value pair corresponding to the transaction data is generated according to the target value and the target key.

3. The method according to claim 1, characterized in that The step of generating and outputting a transaction structured query SQL statement of the key-value pair according to the key-value pair includes: According to the target key in each of the key-value pairs, the key-value pairs are sorted to obtain an ordered list; Traversing each key-value pair in the ordered list, and generating a transaction SQL statement for the key-value pair according to the target value in the key-value pair; According to the same situation of the target key in each of the key-value pairs, each of the transaction SQL statements is output.

4. The method according to claim 3, characterized in that The step of sorting the key-value pairs according to the target key in each key-value pair to obtain an ordered list includes: Sort the key-value pairs according to the transaction time in the target key; When the transaction times are the same, the key-value pairs are sorted according to the transaction identifier in the target key.

5. The method according to claim 3, characterized in that: Outputting each of the transaction SQL statements according to the same situation of the target key in each of the key-value pairs includes: Merge the transaction SQL statements corresponding to the key-value pairs with the same target key, and output the merged transaction SQL statements; For key-value pairs with different target keys, the transaction SQL statements corresponding to the key-value pairs are output.

6. The method according to claim 1, characterized in that Obtaining at least one transaction data from a distributed database node, including: Obtain at least one node data from a distributed database node; Select node data whose transaction time is equal to or later than the preset timestamp as candidate data; The node transaction is selected as candidate data of the data change transaction as the transaction data.

7. A data output device, characterized in that: The device comprises: A data acquisition module, used to acquire at least one transaction data from a distributed database node; the transaction data includes a node transaction, a transaction time and a transaction identifier; A key-value generation module, configured to generate, for each transaction data, a key-value pair corresponding to the transaction data according to the node transaction, the transaction time and the transaction identifier in the transaction data; the key-value pair includes a target key and a target value; The statement generation module is used to generate and output a transaction structured query SQL statement of each key-value pair according to the key-value pair.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images