Data writing and reading method and device

Abstract

The present invention relates to the field of financial technology (Fintech), and discloses a data writing and reading method and device, which performs syntax parsing on a received replication request for first data to determine an abstract syntax tree; the abstract syntax tree includes N first writing task nodes; for each first writing task node: determining the first database to be written and the corresponding converter; converting the first writing task node into a first writing task node in an execution tree of the replication request through the converter; when executing the N first writing task nodes of the execution tree, sending N first writing tasks to the N first databases respectively; the first writing task is used to write the first data into the corresponding first database. In this way, it is possible to generate a corresponding first writing task for the first data in any business scenario, reduce the dependence on the storage engine of the first database corresponding to each first writing task, and improve the efficiency and flexibility of synchronous replication of different business data.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of financial technology (Fintech), and in particular to a method and device for writing and reading data. Background Art

[0002] With the development of computer technology, more and more technologies (such as big data, cloud computing or blockchain) are being applied in the financial field. The traditional financial industry is gradually transforming into financial technology. In the context of big data technology, it is necessary to store massive data based on databases.

[0003] To ensure high data availability, one approach is to write to a single primary cluster and then synchronize data from the primary cluster to the standby cluster through asynchronous replication to achieve eventual data consistency. However, with this master-standby asynchronous replication solution, if user write operations are frequent and the data volume is large, asynchronous replication across clusters can incur significant synchronization costs, such as very high bandwidth requirements. When asynchronous replication experiences high latency, a primary cluster failure can cause the standby cluster to be unavailable for an extended period, significantly impacting clients. Furthermore, this solution imposes requirements on the versions of the primary and standby clusters, generally forcing the primary and standby clusters to use the same storage engine version. Otherwise, data replicated and synchronized by the primary cluster may not be properly parsed by the standby cluster. Furthermore, if an anomaly in the primary cluster forces a user to switch to the standby cluster, the standby cluster will then serve as the primary cluster. Once the original primary cluster recovers, a manual failback is required to initiate asynchronous replication and synchronization of data from the new primary cluster (the original standby cluster) to the new standby cluster (the original primary cluster), resulting in significant operational costs. Summary of the Invention

[0004] The present application provides a data writing and reading method and device for quickly responding to replication / rereading requests in any business scenario without using a specified storage engine.

[0005] In a first aspect, an embodiment of the present application provides a data writing method, which can be executed by a data read-write server, including: receiving a copy request for first data; performing syntax analysis on the copy request for the first data, and determining an abstract syntax tree of the copy request for the first data; the abstract syntax tree includes N first write task nodes; N is a positive integer greater than 1; for each of the N first write task nodes: determining the first database written by the first write task node and the converter corresponding to the first database; through the converter corresponding to the first database, converting the first write task node into a first write task node in an execution tree of the copy request for the first data, the execution tree of the copy request for the first data includes: N first write task nodes; when executing the N first write task nodes of the execution tree of the copy request for the first data, sending N first write tasks to the N first databases respectively; the first write task is used to write the first data into the corresponding first database.

[0006] The first data in different business scenarios may be written into different first databases, which results in the need to design corresponding execution codes for each business when executing the replication request and writing into the database in the prior art. Through the above method, after the data read-write server performs syntax parsing on the replication request for the received first data, the abstract syntax tree obtained can be independent of the business scenario corresponding to the first data, and for each write task node, after determining the first database to which the first write task node is written, the first write task node can be converted into the first write task node in the execution tree based on the converter corresponding to each first database. When the data read-write server executes the first write task node in the execution tree, the first data is written into the first database, meeting the requirement of writing different data into different databases. It can be seen that in this solution, the overall architecture of the abstract syntax tree and the execution tree can be set in a unified manner. Through the converters corresponding to different first databases, the generation of the first write task node in the corresponding execution tree can be achieved, and different data can be written into different databases. Thus, it is avoided to generate a complete set of execution codes for the data corresponding to each business, reducing the user's usage cost and operation and maintenance cost.

[0007] In one possible implementation method, when it is determined that one of the N first write tasks fails to write, the execution tree of the rewrite request of the first data is triggered, and a new first playback task is added; the first playback task is a playback task of the failed first write task; after executing the first playback task, the failed first write task is sent to the playback database.

[0008] Based on this solution, in the process of writing N first write tasks to the corresponding first database, one or more of the first write tasks may fail to write. At this time, in order to improve the accuracy of data writing, the execution tree of the replication request of the first data can be triggered to add a new first playback task. After executing the first playback task, the failed first write task is written to the playback database. The playback database temporarily stores the failed first write task, so that after the first database returns to normal, the first write task stored in the playback database can be rewritten into the first database. Through the above method, the fault tolerance of the data replication architecture is increased.

[0009] In one possible implementation method, when there is at least one failed first write task in the playback database, a first write-back task is added to the execution tree of the replication request for the first data that is triggered; after executing the first write-back task, the failed first write task is resent to the first database corresponding to the failed first write task.

[0010] Based on this solution, after at least one failed first write task is written to the replay database due to an abnormality in the first database, in order to be able to promptly rewrite at least one failed first write task temporarily stored in the replay database back to the first database, the execution tree of the first data replication request can be triggered to add a new first write-back task, and after executing the first write-back task, the failed first write task is resent to the first database corresponding to the failed first write task. In this way, the first data temporarily stored in the replay database can be written to the first database that has returned to normal in a timely manner.

[0011] In one possible implementation method, a copy request for second data is received, and the copy request for the second data is associated with the copy request for the first data; the copy request for the second data is parsed to determine the abstract syntax tree of the copy request for the second data; the abstract syntax tree of the copy request for the second data includes: K second write task nodes; K is a positive integer greater than 1; for each second write task node among the K second write task nodes: determine the second database written by the second write task node and the converter corresponding to the second database; through the converter corresponding to the second database, convert the second write task node into a second write task node in the execution tree of the copy request for the second data; when the execution tree of the copy request for the first data is not successfully executed, add a second playback task in the execution tree of the copy request for the second data; after executing the second playback task, write the K second write tasks into the playback database.

[0012] Based on this solution, in the process of writing the first data to the corresponding first database, if a write exception occurs and a copy request for second data that is associated with the copy request for the first data is also received, then at this time, in order to ensure the correctness of the data writing, a second playback task can be added to the execution tree of the copy request for the second data, and after executing the second playback task, K second write tasks corresponding to the copy request for the second data are written into the playback database.

[0013] In one possible implementation method, when the execution tree of the replication request for the first data is successfully executed, the execution tree of the replication request for the second data is triggered to add a second write-back task; the second write-back task is used to write back at least one second write task of the playback database to the second database corresponding to the at least one second write task; after executing the second write-back task, the at least one second write task is called from the playback database, and the at least one second write task is sent to the second database corresponding to the at least one second write task.

[0014] Based on this scheme, after the K second write tasks corresponding to the replication request of the second data are written into the playback database, at a certain moment, if the execution tree of the replication request of the first data is executed successfully, in order to be able to write the second write task located in the playback database into the corresponding second database in a timely manner, the execution tree of the replication request of the second data can be triggered to add a new second write-back task, so that after the second write-back task is executed, at least one second write task in the playback database can be called to write the second write task into the corresponding second database.

[0015] In a second aspect, an embodiment of the present application provides a data reading method, the method comprising: receiving a reread request for first data; performing syntax analysis on the reread request for the first data, and determining an abstract syntax tree of the reread request for the first data; the abstract syntax tree comprising N first read task nodes; the N being a positive integer greater than 1; for each of the N first read task nodes: determining the first database read by the first read task node and the converter corresponding to the first database; converting the first read task node into a first read task node in an execution tree of the reread request for the first data through the converter corresponding to the first database, the execution tree of the reread request for the first data comprising: N first read task nodes; when executing the N first read task nodes of the execution tree of the reread request for the first data, sending N first read tasks to the N first databases respectively; the first read task is used to read the first data from the corresponding first database.

[0016] At present, for a read request of a data, when reading the data from the corresponding database, the read request of the data is first parsed to determine the business code for reading the data from the corresponding database. In this way, for each read request of data in different business scenarios, it is necessary to parse it to obtain the corresponding business code for reading from the database. Obviously, the data reading process does not achieve good decoupling from the business scenario, which increases the user's usage cost and operation and maintenance cost. Based on this solution, for the re-read request of the first data received, it is possible to determine the abstract syntax tree including N first read task nodes to which it corresponds by performing syntax parsing, and for each read task node, after determining the first database read by the first read task node and the converter corresponding to the first database, the first read task node can be converted into the first read task node in the execution tree through the converter, so that when the first read task node in the execution tree is executed, the first read task can be sent to the corresponding first database, thereby achieving decoupling from the business scenario and reducing the user's usage cost and operation and maintenance cost.

[0017] In a third aspect, an embodiment of the present application provides a data writing device, which includes: a receiving unit for receiving a copy request for first data; an abstract syntax tree determination unit for performing syntax analysis on the copy request for the first data, and determining the abstract syntax tree of the copy request for the first data; the abstract syntax tree includes N first write task nodes; N is a positive integer greater than 1; a converter determination unit for determining, for each of the N first write task nodes: the first database written by the first write task node and the converter corresponding to the first database; a conversion unit for converting the first write task node into a first write task node in an execution tree of the copy request for the first data through the converter corresponding to the first database, the execution tree of the copy request for the first data including: N first write task nodes; a sending unit for sending N first write tasks to N first databases respectively when executing the N first write task nodes of the execution tree of the copy request for the first data; the first write task is used to write the first data into the corresponding first database.

[0018] In one possible implementation method, the device also includes a processing unit for triggering an execution tree of a rewrite request for the first data and adding a new first replay task when it is determined that one of the N first write tasks has failed to write; the first replay task is a replay task of the failed first write task; the sending unit is further used to send the failed first write task to the replay database after executing the first replay task.

[0019] In one possible implementation method, the processing unit is further used to trigger a new first write-back task in the execution tree of the rewrite request for the first data when there is at least one failed first write task in the playback database; the sending unit is further used to resend the failed first write task to the first database corresponding to the failed first write task after executing the first write-back task.

[0020] In one possible implementation method, the receiving unit is further used to receive a copy request for second data, and the copy request for the second data is associated with the copy request for the first data; the abstract syntax tree determination unit is further used to perform syntax analysis on the copy request for the second data to determine the abstract syntax tree of the copy request for the second data; the abstract syntax tree of the copy request for the second data includes: K second write task nodes; K is a positive integer greater than 1; the converter determination unit is further used to: determine the second database written by the second write task node and the converter corresponding to the second database for each second write task node among the K second write task nodes; the conversion unit is further used to convert the second write task node into a second write task node in the execution tree of the copy request for the second data through the converter corresponding to the second database; the processing unit is further used to add a second playback task in the execution tree of the copy request for the second data when the execution tree of the copy request for the first data is not successfully executed; the processing unit is further used to write the K second write tasks into the playback database after executing the second playback task.

[0021] In one possible implementation method, the processing unit is also used to trigger the execution tree of the replication request of the second data to add a second write-back task when the execution tree of the replication request of the first data is successfully executed; the second write-back task is used to write back at least one second write task of the playback database to the second database corresponding to the at least one second write task; the sending unit is also used to call the at least one second write task from the playback database after executing the second write-back task, and send the at least one second write task to the second database corresponding to the at least one second write task.

[0022] In a fourth aspect, an embodiment of the present application provides a data reading device, which includes: a receiving unit for receiving a reread request for first data; an abstract syntax tree determination unit for performing syntax analysis on the reread request for the first data, and determining an abstract syntax tree for the reread request for the first data; the abstract syntax tree includes N first read task nodes; N is a positive integer greater than 1; a converter determination unit for determining, for each of the N first read task nodes in the abstract syntax tree: a first database read by the first read task node and a converter corresponding to the first database; a conversion unit for converting the first read task node into a first read task node in an execution tree of the reread request for the first data through the converter corresponding to the first database; the execution tree of the reread request for the first data includes: N first read task nodes; a sending unit for sending N first read tasks to N first databases respectively when executing the N first read task nodes of the execution tree of the reread request for the first data; the first read task is used to read the first data from the corresponding first database.

[0023] In a fifth aspect, an embodiment of the present application provides a computing device, including:

[0024] a memory for storing program instructions;

[0025] The processor is used to call the program instructions stored in the memory and execute any implementation method of the first aspect or the second aspect according to the obtained program.

[0026] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute any implementation method of the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of a system architecture provided in an embodiment of the present application;

[0028] Figure 2 This is a schematic diagram of a data double-write solution;

[0029] Figure 3 A flowchart of a data writing method provided in an embodiment of the present application;

[0030] Figure 4 A schematic diagram of converting a first data replication request into a first data write event provided by an embodiment of the present application;

[0031] Figure 5 A schematic diagram of a logic execution tree provided in an embodiment of the present application;

[0032] Figure 6 A schematic diagram of a logic execution tree provided in an embodiment of the present application;

[0033] Figure 7 A schematic diagram of a physical execution tree provided in an embodiment of the present application;

[0034] Figure 8a A schematic diagram of writing a task into a playback database provided in an embodiment of the present application;

[0035] Figure 8b A schematic diagram of retrying a failed first write task provided by an embodiment of the present application;

[0036] Figure 9 A schematic diagram of a playback database write provided in an embodiment of the present application;

[0037] Figure 10 A schematic diagram of a method for writing second data provided in an embodiment of the present application;

[0038] Figure 11 A schematic diagram of a write-back task provided in an embodiment of the present application;

[0039] Figure 12 A schematic diagram of a task writing when catching up with data provided in an embodiment of the present application;

[0040] Figure 13 A flowchart of a data reading method provided in an embodiment of the present application;

[0041] Figure 14 A schematic diagram of a single data reading method provided in an embodiment of the present application;

[0042] Figure 15 A schematic diagram of a data dual reading method provided in an embodiment of the present application;

[0043] Figure 16 A schematic structural diagram of a data writing device provided in an embodiment of the present application;

[0044] Figure 17 A schematic structural diagram of a data reading device provided in an embodiment of the present application;

[0045] Figure 18 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] To make the objectives, technical solutions, and advantages of this application more clear, this application will be further described in detail below with reference to the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0047] like Figure 1 As shown, it is a schematic diagram of a system architecture provided by an embodiment of the present application, wherein the system includes a client 110, a data read-write server 111 and at least two databases, such as Figure 1 As shown, it is assumed that database 112 and database 113 are included.

[0048] The client 110 may be a terminal device such as a smartphone, computer, iPad, or an application or component that performs a data write request function, and is used to send a copy request for the first data to the data read / write server 111. The copy request for the first data may include indication information, which is used to indicate the identification information of a database, which may be the target database to which the data carried by the copy request for the first data is written. The database 112 may be an independent database or a database service cluster composed of multiple databases, which is not limited in this application; the database 113 may be an independent database or a database service cluster composed of multiple databases, which is not limited in this application.

[0049] The data read-write server 111 may parse the replication request for the first data sent by the client 110 , determine the database to which the first data is to be written, and write the first data into the database.

[0050] In one possible way, in order to ensure high availability of data, the data read-write server 111 can write the first data into multiple databases through asynchronous replication. The multiple databases can be primary clusters and backup clusters. The following takes the primary and backup clusters as the databases where the first data is written as an example for explanation. Figure 2 As shown, it is a schematic diagram of a data dual-write solution, wherein the data read-write server 111 can first write to the primary cluster, and then synchronize the data of the primary cluster to the backup cluster through asynchronous replication to achieve final data consistency.

[0051] because Figure 2The data dual-write solution shown has the drawbacks of high synchronization costs, dependence on the storage engine, and high operation and maintenance costs. Currently, there are technical solutions for simultaneous dual-write in active-active clusters. However, most of the known technical solutions for active-active clusters, such as active-active clusters and dual-write data from the client source, are tightly coupled to business application scenarios. They can only implement active-active dual-write in a specific business application scenario and for a specific storage engine. For example, in the HBase engine, in some scenarios where real-time writes are rare and real-time queries are frequent, and most writes are done through batch imports, the client only needs to open two threads for direct dual-writes, because the impact of low real-time writes on overall business queries is minimal, thus achieving a multi-active effect. Clearly, this technical solution for simultaneous dual-write in active-active clusters does not have universal applicability regardless of business or storage engine.

[0052] In view of the above technical problems, the embodiments of the present application are based on Figure 1 The architecture shown provides a data writing method, such as Figure 3 As shown, the method includes:

[0053] Step 301: Receive a request for duplication of first data.

[0054] In step 301, a data read-write server may receive a request for copying first data sent by a client.

[0055] The data read-write server may be configured to receive a replication request for the first data sent by the client. In some embodiments, the database may be a relational database or a non-relational database.

[0056] Taking the database as a relational database as an example, the data read and write server can implement operations on the relational database through the Structured Query Language (SQL). In this case, the replication request for the first data can be an SQL statement. For example, the replication request for the first data can be a storage task write request (StorageJobWriteReq). The SQL statement may include: keywords, database table names, database column names and other information. Among them, the keywords are used to indicate operation information such as "querying the table" or "writing the first data in the database". The Database Management System (DBMS) can set different SQL statements according to the instructions corresponding to the operations on the database.

[0057] For another example, if the database is a non-relational database, such as an HBase database, the data read / write server can generate an abstract syntax tree corresponding to the HBase database according to the replication request of the first data. Of course, other types of databases can also be used, and this application does not limit this.

[0058] Step 302: Parse the request for copying the first data to determine an abstract syntax tree of the request for copying the first data.

[0059] The abstract syntax tree includes N first writing task nodes; N is a positive integer greater than 1.

[0060] In step 302, using SQL as an example, the data read / write server can parse a request submitted by a client to replicate first data, obtaining a grammatically structured description of the SQL statement. This parsing method can include generating a tree structure corresponding to the SQL statement. Depending on the parsing method, the tree structure can include an abstract syntax tree, a logical execution tree, and a physical execution tree.

[0061] In order to facilitate the data read and write server to parse the copy request of the first data, a write event of the first data can be established for the copy request of the first data. For example, the write event of the first data can be a storage write event (StorageWriteJob). The write event of the first data is the starting module or trigger module of the tree structure after the copy request of the first data is parsed. The write event of the first data may include information about the copy request of the first data. Figure 4 , which is a schematic diagram of converting a first data replication request into a first data write event provided by an embodiment of the present application.

[0062] The write event of the first data may be used to generate and execute an abstract syntax tree (ABST) and an execution tree. The execution tree may include at least one of the following: a logical execution tree (LOGICAL TREE) and a physical execution tree (PHYSICAL TREE).

[0063] In one possible implementation, a data read / write server's compiler Parser can perform lexical analysis on the write event of the first data to generate an abstract syntax tree (ABST). During the lexical analysis of the write event of the first data, the SQL statement can be tokenized according to a predefined SQL lexical grammar, outputting a token sequence (TOKEN) including SQL keywords, constants, variables, and operators. The token sequence is parsed according to a predefined SQL syntax grammar to establish a hierarchical ABST. Specifically, the ABST can be a lexical, syntactic, and semantic analysis of the SQL statement by the data read / write server, which determines the keywords (e.g., Select, From, Where, etc.) in the SQL statement, automatically splits them into different tokens according to preset rules, and forms the basic tree structure of the ABST as nodes. The information of the first data's replication request (e.g., query fields, table names, conditions, etc.) is then loaded in a binary tree format, ultimately forming an ABST with a binary tree structure. The ABST includes N first write task nodes.

[0064] Considering that the first write task node may fail to write, in order to distinguish the first write task node of the first data from other write task nodes, when executing the abstract syntax tree, the execution process can be identified as the storage write stage of the write event of the first data, for example, storage write stage 1 (StorageWriteStage1).

[0065] Step 303 : For each of the N first writing task nodes, determine the first database written by the first writing task node and the converter corresponding to the first database.

[0066] Taking the SQL architecture as an example, after obtaining the abstract syntax tree, the data read-write server can determine the first database written by each of the N first write task nodes and the converter corresponding to the first database according to business needs. Through the converter, a logical execution tree can be generated based on the abstract syntax tree. Through the logical execution tree, two first write tasks can be generated when the replication request for the first data is a double write request. The two first write tasks are used by the data read-write server to send their respective first write tasks to the corresponding two databases. Each of the two databases is a first database. Furthermore, a physical execution tree for executing each write task node is generated through the logical execution tree.

[0067] Step 304: Convert the first write task node into a first write task node in the execution tree of the replication request for the first data through a converter corresponding to the first database. The execution tree of the replication request for the first data includes: N first write task nodes.

[0068] Taking the SQL architecture as an example, the execution tree of the replication request for the first data may include: a logical execution tree of the replication request for the first data and a physical execution tree of the replication request for the first data.

[0069] like Figure 5 As shown, it is a schematic diagram of a logic execution tree provided in an embodiment of the present application. The data read and write server can generate two leaf nodes of the logic execution tree for the storage write phase (for example, storage write phase 1) of the abstract syntax tree through a converter, and each leaf node corresponds to a first write task node. For example, the first first write task node is storage write task 1 (StorageWriteTask1), and the second first write task node can be storage write task 2 (StorageWriteTask2). For example, storage write task 1 (StorageWriteTask1) can be the first write task that writes the first data to the first database 1 (for example, database 212), and storage write task 2 (StorageWriteTask2) can be the first write task that writes the first data to the first database 2 (for example, database 213). In some embodiments, the first database 1 can be the primary database, and the first database 2 can be the backup database. The storage engine of the first database 1 can be different from the storage engine of the first database 2.

[0070] In some embodiments, the converter may be a planner, thereby converting the abstract syntax tree into a logical execution tree. In other embodiments, the converter may be a separate converter that is triggered by the planner to generate two leaf nodes for the storage write phase of the logical execution tree, each leaf node corresponding to a first write task node.

[0071] At this time, after generating N first write task nodes, the planner can also generate corresponding task nodes for the storage write phase 1 of the logical execution tree and the storage write event. The task nodes are used by the data read and write server to determine the logical execution tree in the write event of the first data based on the relationship between the tasks corresponding to the storage write event. For example, Figure 6 As shown, the task nodes corresponding to the storage writing stage 1 of the logic execution tree include: the starting task of the storage writing stage (SGStageTask(Start)) and the ending task of the storage writing stage (SGStageTask(End)).

[0072] Any node corresponding to the storage write event includes: a start task (SGJobTask(Start)) storing the write event, and an end task (SGJobTask(End)) storing the write event.

[0073] It should be noted that since the names of the nodes in the following tree structure are long, the names of the nodes are simplified starting from the logical execution tree, and Storage is abbreviated as SG, and Write is ignored.

[0074] Through the starting task of the storage write event, the storage write event is triggered to start execution, and then the starting task of the storage write phase of the logical execution tree in the storage write event is executed, triggering the execution of storage write task 1 and storage write task 2. After storage write task 1 and storage write task 2 are executed, the ending task of the storage write phase of the logical execution tree is executed. After the ending task of the storage write phase of the logical execution tree is executed, the ending task of the storage write event is executed.

[0075] After generating the logical execution tree, the data read and write server converts the logical execution tree into a physical execution tree through the optimizer. The optimizer mainly optimizes the write event of the first data based on data reading, table connection method, table connection order, sorting and other technologies, corresponding to the execution task of generating the physical execution tree.

[0076] like Figure 7 As shown, a schematic diagram of a physical execution tree provided by an embodiment of the present application. The execution task nodes of the physical execution tree of the write event of the first data may include: a start execution task (SGJobExecTask(Start)) for storing the write event, an end execution task (SGJobExecTask(End)) for storing the write event; a start execution task (SGStageExecTask(Start)) for storing the write stage, an end execution task (SGStageExecTask(End)) for storing the write stage, a write execution task 1 (SGExecTask1) for storing the write stage, and a write execution task 2 (SGExecTask2) for storing the write stage.

[0077] Storage write execution task 1 may be the first write task 1 in the physical execution tree, and storage write execution task 2 may be the first write task 2 in the physical execution tree. Storage write execution task 1 may be obtained by optimizing storage write task 1 (the first write task 1 in the logical execution tree). When executing storage write execution task 1, the data read / write server may send first write task 1 to the first database corresponding to storage write execution task 1 (e.g., first database 1).

[0078] Storage write execution task 2 may be obtained after the optimizer optimizes storage write task 2 (the first write task 2 in the logical execution tree). When executing storage write execution task 2, the data read / write server may send the first write task 2 to the first database corresponding to storage write execution task 2 (e.g., first database 2).

[0079] Through the start execution task of the storage write event, the storage write event is triggered to start execution, and the start execution task of the storage write phase of the physical execution tree in the storage write event is executed, which triggers the execution of storage write execution task 1 and storage write execution task 2. After the execution of storage write execution task 1 and storage write execution task 2, the end execution task of the storage write phase of the physical execution tree is triggered. After the end execution task of the storage write phase of the physical execution tree is executed, the end execution task of the storage write event is executed.

[0080] Step 305 : When executing the N first write task nodes of the execution tree of the replication request for the first data, N first write tasks are respectively sent to the N first databases; the first write tasks are used to write the first data into the corresponding first databases.

[0081] The data read-write server may execute the physical execution tree corresponding to the write event of the first data, and send N first write tasks to the N first databases respectively.

[0082] Combine Figure 7 When the data read and write server executes the physical execution tree, when executing storage write execution task 1 and storage write execution task 2, the first write task 1 can be sent to the first database 1 corresponding to storage write execution task 1, and the first write task 2 can be sent to the first database 2 corresponding to storage write execution task 2. Each first write task is used to write the first data that needs to be written this time into the corresponding database. When the first write task 1 and the first write task 2 are executed successfully, the end execution task of the storage write phase of the physical execution tree is triggered. After the end execution task of the storage write phase of the physical execution tree is executed, the end execution task of the storage write event is executed. At this time, it can be determined that the write event of the first data has run successfully, that is, the replication request of the first data has been written successfully.

[0083] In some embodiments, considering that the first write task may fail to write, in order to improve the success rate of the first write task, when it is determined that M of the N first write tasks have failed to execute, the M failed first write tasks can be stored in a replay database; wherein, the replay database can be a database other than the first database 1 and the first database 2, and M is a positive integer less than or equal to N. Thus, when the first database corresponding to the subsequent first write task returns to normal, the failed first write tasks stored in the replay database can be synchronized to the first database. For example, when there is at least one failed first write task in the replay database, the failed first write task can be resent to the first database corresponding to the failed first write task based on any of the at least one failed first write task.

[0084] Taking into account the compatibility of the write request architecture, when it is determined that one of the N first write tasks fails to write, the execution tree of the replication request of the first data can be triggered, and a new first playback task can be added; the first playback task is the playback task of the failed first write task; after executing the first playback task, the failed first write task is sent to the playback database.

[0085] In one possible implementation, Figure 8a As shown, a schematic diagram of a task writing to a playback database provided by an embodiment of the present application is provided. After detecting that the storage write execution task 2 fails to execute, the retry module can add a first playback task after the storage write execution task 2 (SGExecTask2). The first playback task is used to write the storage write execution task 2 to the playback database (PlaybackService). For example, the first playback task can be a write task (PlaybackWrittenTask) of the playback database.

[0086] Assume that Figure 7 In the example, if the storage write task 2 fails, to improve the rewrite efficiency, the retry module Reheater can be triggered before writing the failed first write task to the playback database. The retry module detects the failure of the storage write task 2 and corrects the physical execution tree. For example, Figure 8b As shown, it is a schematic diagram of a failed first write task retry provided by an embodiment of the present application, wherein the retry module can add a storage write execution task 2-retry 1 (SGExecTask2(retry1)) node behind the storage write execution task 2 node of the physical execution tree.

[0087] It should be noted that the retry module is a retry capability when the physical execution tree fails. Retry allows the physical execution tree to be modified again and then submitted for execution based on different circumstances after the physical execution tree fails, ensuring that the first data write event is ultimately executed successfully.

[0088] If the storage write execution task 2-retry 1 node is executed successfully, the execution retry module will not be triggered, and the replication request for the first data will be completed normally. Here, if G retries (the value of G can be configured as a label, for example, in the embodiment of the application, G is set to 2) all fail, the retry module will be triggered to mark the task as failed and trigger playback.

[0089] In one possible implementation, both the first playback task and the retry can be executed, for example, Figure 9As shown in FIG. 1 , a schematic diagram of a task writing to a playback database provided by an embodiment of the present application is provided. The retry module can add a first playback task after the storage write execution task 2-retry 2 (SGExecTask2(retry2)). The first playback task is used to write the storage write execution task 2 to the playback database (PlaybackService). Thus, the success rate of the first write task is improved.

[0090] It should be noted that the process of retrying the failed first write task through the retry module can be performed before the failed first write task is stored in the playback database, or after the failed first write task is stored in the playback database, or after the failed first write task is stored in the playback database, the retry of the failed first write task is not performed. This improves the writing efficiency of the first write task. The specific implementation method can be determined according to actual needs and is not limited here.

[0091] Considering that the data read-write server may receive write requests for other data, for example, a copy request for second data. For the copy request for the second data, the present application can perform syntax parsing on the copy request for the second data to determine the abstract syntax tree of the copy request for the second data; the abstract syntax tree of the copy request for the second data includes: K second write task nodes; K is a positive integer greater than 1; for each second write task node in the K second write task nodes: determining the second database written by the second write task node and the converter corresponding to the second database; through the converter corresponding to the second database, converting the second write task node into the second write task node in the execution tree of the copy request for the second data.

[0092] In some embodiments, the request to replicate the second data and the request to replicate the first data may be associated with each other.

[0093] The following uses Method 1 and Method 2 as examples to illustrate possible association relationships.

[0094] In method 1, the association relationship may mean that the write condition for the second data is the successful writing of the first data. That is, the write operation for the second data can only be performed after the first data is successfully written. The write condition for the second data can be determined by the write priority between the data, for example, the write priority of the first data is higher than the write priority of the second data. When the data read / write server determines that the write priority of the first data is higher than the write priority of the second data, it can be determined that the rewrite request for the second data and the rewrite request for the first data may be associated.

[0095] To improve the success rate of writing the second data, when the first data and the second data are associated, the data read-write server may write all K second writing tasks to the playback database and wait for playback.

[0096] One possible implementation method is to add a second playback task to the execution tree of the replication request for the second data when the execution tree of the replication request for the first data is not executed successfully; after executing the second playback task, the K second write tasks are written into the playback database.

[0097] In mode 2, the association relationship between the replication request for the second data and the replication request for the first data can also be: the first database (for example, including the first database 1 and the first database 2) written by at least one first write task and the second database written by at least one second write task are located in the same database cluster, or the second database is the first database. In this case, when the first write task fails in at least one of the first databases 1 and the first database 2, the failed first write task is written to the replay database. Since the failure of the first write task may be caused by the unavailability of the first database, in order to avoid the failure of writing the second data to the second database, at least one of the K second write tasks can be written to the replay database.

[0098] In one possible implementation, a second playback task may be added to the execution tree of the second data replication request. The added second playback task may be used by the data read / write server to write at least one of the K second write tasks into the playback database. Alternatively, the added second playback task may include at least one second playback subtask, each of which may be used by the data read / write server to write the second write task corresponding to each second playback subtask into the playback database when executing the second playback subtask.

[0099] In some possible scenarios, at least one of the K second write tasks may be generated based on the execution of a write request for the first data and a copy request for the first data. When the data read-write server determines that there is at least one second write task that is written to the first database among the K second write tasks, the data read-write server may determine whether to execute the at least one second write task based on the availability of the first database. For example, when it is determined that the first database 1 is unavailable, the at least one second write task may be written to the playback database. When it is determined that the first database 1 is available, the at least one second write task may be written to the first database. Alternatively, after determining that the first database 1 is available, the second write task stored in the playback database may be written to the first database.

[0100] Below is Figure 10This example illustrates a possible scenario for adding a second playback task. For scenarios related to the second playback subtask, you can refer to this example. Figure 10 As shown, taking the second writing task including the second writing task 1 and the second writing task 2 as an example, the second writing task 1 corresponds to writing into the first database 1, and the second writing task 2 corresponds to writing into the second database 1, wherein the second database 1 and the first database 1 are different databases.

[0101] At this time, the execution tasks of the physical execution tree corresponding to the write event of the second data may include:

[0102] The starting execution task of the storage write event and the ending execution task of the storage write event; the starting execution task of the storage write phase of the physical execution tree, the ending execution task of the storage write phase of the physical execution tree, the second playback task and the storage write execution task 2.

[0103] Among them, the second playback task can be generated by the data read and write server based on the second write task 1 written to the first database 1. The specific generation method of the second playback task can refer to the generation method of the first playback task based on the first write task 2, which will not be repeated here. The storage write execution task 2 can be generated corresponding to the second write task 2 written to the second database 1. When the data read and write server executes the physical execution tree, it can execute the second playback task, write the second write task 1 to the playback database, and send the first write task 1 to the first database corresponding to the storage write execution task 1. When the data read and write server executes the physical execution tree, it can execute the storage write execution task 2, write the second write task 2 to the playback database, and send the second write task 2 to the second database 1 corresponding to the storage write execution task 2.

[0104] For another example, when any first writing task fails, K second writing tasks are triggered and all write to the replay database. In this case, the K second writing tasks may be K replay writing tasks generated in the physical execution tree.

[0105] In certain implementations of the present application, when the execution tree of the replication request for the first data is successfully executed, a second write-back task is added to the execution tree of the replication request for the second data; the second write-back task is used to write back at least one second write task of the playback database to the second database corresponding to the at least one second write task; after executing the second write-back task, the at least one second write task is called from the playback database and sent to the second database corresponding to the at least one second write task.

[0106] For example, the data read-write server determines that the replication request for the first data is successfully executed by not writing the failed first write task to the playback database in the first database, or after the failed first write task resent from the playback database to the first database is completed, at which time the first database may have returned to normal, such as Figure 11 As shown, this is a schematic diagram of a newly added second write-back task provided in an embodiment of the present application. When there is at least one failed first write task in the playback database, the data read-write server can trigger the execution tree of the replication request of the second data to add a second write-back task; after executing the second write-back task, call at least one second write task from the playback database, and resend the at least one second write task to the second database corresponding to the at least one second write task.

[0107] In some embodiments, the second write-back task can be used to test whether the database has been restored. Alternatively, the second write-back task can be generated after determining that the second database has been restored. For example, the second write-back task can be a new write task for writing to the database. This write task can be a task with lower complexity than the second write task, avoiding repeated testing that would otherwise result in high computational overhead.

[0108] In combination with method 1, the second write-back task may be to resend the second write task to the second database of the second write task corresponding to the second write-back task. In this case, when the second write task is written successfully, it can be determined that the first write task is successful, and the second write task can be written to the second database.

[0109] In combination with method 2, the second write-back task may be to resend the second write task to the first database corresponding to the second write task. In this case, when the second write task is successfully written, it can be determined that the first database has been restored.

[0110] For example, in the above-mentioned method 2, the second write-back task is used to confirm whether the faulty database (for example, the first database) has been restored to normal availability. The confirmation results may include the following two:

[0111] Result 1: The second write-back task fails, and the system waits for the next trigger.

[0112] Result 2: The second write-back task is successful. At this point, the second write-back task triggers a broadcast trial playback success event. This trial playback success event instructs the playback listener to rewrite the failed first or second write task stored in the playback database to the corresponding database. When the playback listener rewrites the failed first or second write task stored in the playback database to the corresponding database, the first or second write task rewritten to the corresponding database can add a tag of the corresponding database. This tag indicates that the rewriting of the first or second write task to the corresponding database will not trigger a double write.

[0113] In some implementations of the present application, a writing speed of the failed first writing task to the corresponding first database is greater than a writing speed of the second writing task to the playback database.

[0114] Among them, for result 2, it is necessary to discuss the scenario of data equalization during playback: Figure 12 As shown, a schematic diagram of task writing during data balancing provided by an embodiment of the present application is provided, wherein, on the one hand, the write-back listener keeps writing back (for example, the first write task of the first data is executing the first write-back task), and on the other hand, new write tasks (for example, the newly added failed second write task of the second data, or the newly added second write task) are continuously written to the playback database; if the write-back speed cannot keep up with the speed of writing the new write tasks to the playback database, the tasks that need to be written back will continue to grow, triggering alarms and event broadcasts; if the write-back speed is faster, then after a certain period of time, there will be only a few tasks that need to be replayed in the playback database. At this time, the write-back listener completes the write-back, and the write-back optimizer merges the remaining at least one write task with the newly added playback task. For example, the first write-back task corresponding to the newly added first playback task and the remaining first write-back tasks in the playback database can be merged to generate a merged first write-back task, which can be added to the physical execution tree corresponding to the replication request for the first data submitted by the user. For another example, the second write-back task corresponding to the newly added second playback task can be merged with the remaining second write-back tasks in the playback database to generate a merged second write-back task, which is then added to the physical execution tree corresponding to the second data replication request submitted by the user. Thus, while the merged write-back task (e.g., the first write-back task or the second write-back task) is being executed, the new playback task is also executed to complete the write-back. When all write-back tasks are completed, the write-back task and the playback task are equalized.

[0115] Based on the same concept, the embodiment of the present application also provides a data reading method, such as Figure 13 As shown, the method includes:

[0116] Step 1301: Receive a request for rereading first data.

[0117] Step 1302: perform syntax analysis on the reread request of the first data to determine an abstract syntax tree of the reread request of the first data; the abstract syntax tree of the reread request of the first data includes N first read task nodes; N is a positive integer greater than 1.

[0118] Step 1303 : For each of the N first reading task nodes in the abstract syntax tree, determine the first database read by the first reading task node and the converter corresponding to the first database.

[0119] Step 1304 : Convert the first read task node into the first read task node in the execution tree of the re-read request of the first data through a converter corresponding to the first database.

[0120] Step 1305 : When executing the N first read task nodes of the execution tree of the re-read request of the first data, N first read tasks are respectively sent to the N first databases.

[0121] In this way, a corresponding first read task can be generated for the first data in any business scenario, reducing the dependence on the storage engine of the first database corresponding to each first read task. There is no need to separately generate a calling interface adapted to the storage engine of the first database for the first database of the first data read corresponding to each business scenario, thereby improving the efficiency and flexibility of synchronous re-reading of different business data.

[0122] like Figure 14 FIG. 1 is a schematic diagram of a single data reading method provided by an embodiment of the present application, wherein after a data reading server fails to read one of the first databases, it can switch to another first database for reading. Figure 14 (a) in the figure indicates that when the data reading server receives a data reading request, it parses the data reading request and establishes a reading event, a reading phase, and a reading task; Figure 14 (b) in the figure indicates that data is first read from database 1 corresponding to read execution task 1. Figure 14 (c) in FIG. 1 indicates that when data cannot be read from database 1, data is read from database 2 corresponding to read execution task 2.

[0123] In some implementations of the present application, when it is determined that M of the N first read tasks have failed to execute, the M failed first read tasks are ignored; where M is a positive integer less than or equal to N.

[0124] like Figure 15 FIG. 1 is a schematic diagram of a data dual reading method provided by an embodiment of the present application, wherein: Figure 15(a) in the figure indicates that when the data reading server receives a data reading request, it parses the data reading request and establishes a read event, a read phase, and two read tasks; Figure 15 (b) in the figure indicates that Figure 15 The logical execution tree corresponding to (a) in the figure; Figure 15 (c) in the table indicates that Figure 15 (b) corresponds to the physical execution tree. In this method, the first one to return data to the client is considered to have read the data successfully.

[0125] If a database is currently in an abnormal state (for example, when any write task to the database fails, the database is considered to be in an abnormal state; or when the number of unsuccessful write tasks to the database exceeds a preset threshold, the database is considered to be in an abnormal state), and the dual read is used as follows Figure 15 The solution shown in the figure, then the double reading will be automatically optimized to single reading, that is, the first and second steps are as follows Figure 15 As shown, it will be converted into the logical execution tree of the dual-read task; but when optimized into a physical execution tree, it will be converted into Figure 14 The physical execution tree of a single read task is shown.

[0126] like Figure 16 As shown, an embodiment of the present application further provides a data writing device, the device comprising:

[0127] The receiving unit 1601 is configured to receive a request for copying first data.

[0128] The abstract syntax tree determining unit 1602 is configured to perform syntax analysis on the replication request for the first data to determine an abstract syntax tree of the replication request for the first data; the abstract syntax tree includes N first write task nodes; and N is a positive integer greater than 1.

[0129] The converter determining unit 1603 is configured to determine, for each of the N first writing task nodes, a first database written by the first writing task node and a converter corresponding to the first database.

[0130] The conversion unit 1604 is used to convert the first write task node into the first write task node in the execution tree of the replication request of the first data through the converter corresponding to the first database. The execution tree of the replication request of the first data includes: N first write task nodes.

[0131] The sending unit 1605 is used to send N first write tasks to N first databases respectively when executing N first write task nodes of the execution tree of the replication request of the first data; the first write task is used to write the first data into the corresponding first database.

[0132] Furthermore, the device also includes a processing unit 1606, which is used to trigger the execution tree of the rewrite request of the first data and add a new first replay task when it is determined that one of the N first write tasks fails to write; the first replay task is a replay task of the failed first write task; the sending unit is also used to send the failed first write task to the replay database after executing the first replay task.

[0133] Furthermore, for the device, the processing unit 1606 is also used to trigger a new first write-back task in the execution tree of the rewrite request of the first data when there is at least one failed first write task in the playback database; the sending unit 1605 is also used to resend the failed first write task to the first database corresponding to the failed first write task after executing the first write-back task.

[0134] Furthermore, for the device, the receiving unit 1601 is also used to receive a copy request for second data, and the copy request for the second data is associated with the copy request for the first data; the abstract syntax tree determination unit 1602 is also used to perform syntax analysis on the copy request for the second data to determine the abstract syntax tree of the copy request for the second data; the abstract syntax tree of the copy request for the second data includes: K second write task nodes; the K is a positive integer greater than 1; the converter determination unit 1603 is also used to: determine the second database written by the second write task node and the converter corresponding to the second database for each second write task node among the K second write task nodes; the conversion unit 1604 is also used to convert the second write task node into a second write task node in the execution tree of the copy request for the second data through the converter corresponding to the second database; the processing unit 1606 is also used to add a second playback task in the execution tree of the copy request for the second data when the execution tree of the copy request for the first data is not successfully executed; the processing unit 1606 is also used to write the K second write tasks into the playback database after executing the second playback task.

[0135] Furthermore, for the device, the processing unit 1606 is also used to trigger the execution tree of the replication request of the second data to add a second write-back task when the execution tree of the replication request of the first data is successfully executed; the second write-back task is used to write back at least one second write task of the playback database to the second database corresponding to the at least one second write task; the sending unit 1605 is also used to call the at least one second write task from the playback database after executing the second write-back task, and send the at least one second write task to the second database corresponding to the at least one second write task.

[0136] Based on the same concept, the embodiment of the present application also provides a data reading device, such as Figure 17 As shown, the device includes:

[0137] The receiving unit 1701 is configured to receive a request for rereading first data.

[0138] The abstract syntax tree determination unit 1702 performs syntax analysis on the reread request of the first data to determine the abstract syntax tree of the reread request of the first data; the abstract syntax tree of the reread request of the first data includes N first read task nodes; and N is a positive integer greater than 1.

[0139] The converter determining unit 1703 is configured to determine, for each of the N first read task nodes in the abstract syntax tree, a first database read by the first read task node and a converter corresponding to the first database.

[0140] The conversion unit 1704 is configured to convert the first read task node into a first read task node in the execution tree of the reread request of the first data through a converter corresponding to the first database.

[0141] The sending unit 1705 is configured to send N first read tasks to N first databases respectively when executing the N first read task nodes of the execution tree of the reread request of the first data.

[0142] The present application also provides a computing device, which may be a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (PDA), etc. The computing device may include a central processing unit (CPU), a memory, input / output devices, etc. The input device may include a keyboard, a mouse, a touch screen, etc. The output device may include a display device, such as a liquid crystal display (LCD), a cathode ray tube (CRT), etc.

[0143] The memory may include a read-only memory (ROM) and a random access memory (RAM), and provides the processor with program instructions and data stored in the memory. In the embodiment of the present application, the memory may be used to store program instructions for data writing and reading methods;

[0144] The processor is used to call the program instructions stored in the memory and execute the data writing and reading method according to the obtained program.

[0145] like Figure 18FIG. 1 is a schematic diagram of a computing device provided in an embodiment of the present application, wherein the computing device includes:

[0146] Processor 1801, memory 1802, transceiver 1803, bus interface 1804; wherein the processor 1801, memory 1802 and transceiver 1803 are connected via bus 1805;

[0147] The processor 1801 is configured to read the program in the memory 1802 and execute the above-mentioned data writing and reading methods;

[0148] Processor 1801 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP. It may also be a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0149] The memory 1802 is used to store one or more executable programs and can store data used by the processor 1801 when performing operations.

[0150] Specifically, the program may include program code, which includes computer operating instructions. Memory 1802 may include volatile memory, such as random-access memory (RAM); memory 1802 may also include non-volatile memory, such as flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); and memory 1802 may also include a combination of the aforementioned types of memory.

[0151] The memory 1802 stores the following elements, executable modules or data structures, or a subset or an extension thereof:

[0152] Operation instructions: include various operation instructions, used to implement various operations.

[0153] Operating system: includes various system programs used to implement various basic services and process hardware-based tasks.

[0154] The bus 1805 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 18 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0155] The bus interface 1804 may be a wired communication access port, a wireless bus interface, or a combination thereof. The wired bus interface may be, for example, an Ethernet interface. The Ethernet interface may be an optical interface, an electrical interface, or a combination thereof. The wireless bus interface may be a WLAN interface.

[0156] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a data writing and reading method.

[0157] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0158] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0159] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0160] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0161] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0162] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A data writing method, characterized in that: include: receiving a request for copying first data; Performing syntax analysis on the replication request for the first data to determine an abstract syntax tree of the replication request for the first data; The abstract syntax tree includes N first writing task nodes; Said N is a positive integer greater than 1; For each of the N first writing task nodes: determining a first database written by the first writing task node and a converter corresponding to the first database; Converting the first write task node into a first write task node in an execution tree of a replication request for the first data through a converter corresponding to the first database, where the execution tree of the replication request for the first data includes: N first write task nodes; When executing N first write task nodes of the execution tree of the replication request for the first data, N first write tasks are respectively sent to N first databases; the first write tasks are used to write the first data into corresponding first databases.

2. The method according to claim 1, wherein The method further comprises: When it is determined that one of the N first writing tasks fails to write, triggering the execution tree of the replication request for the first data, adding a first playback task; the first playback task is a playback task of the failed first writing task; After executing the first playback task, the failed first write task is sent to the playback database.

3. The method according to claim 1 or 2, wherein: The method further comprises: When there is at least one failed first write task in the playback database, a new first write-back task is added to the execution tree of the replication request for the first data; After executing the first write-back task, the failed first write task is resent to the first database corresponding to the failed first write task.

4. The method according to claim 1 or 2, wherein: The method further comprises: receiving a request for copying second data, where the request for copying the second data is associated with the request for copying the first data; Performing syntax parsing on the replication request for the second data to determine an abstract syntax tree of the replication request for the second data; the abstract syntax tree of the replication request for the second data includes: K second write task nodes; K is a positive integer greater than 1; For each second writing task node among the K second writing task nodes: determining a second database written by the second writing task node and a converter corresponding to the second database; Converting the second write task node into a second write task node in the execution tree of the replication request for the second data through a converter corresponding to the second database; When the execution tree of the replication request for the first data is not successfully executed, adding a second playback task in the execution tree of the replication request for the second data; After executing the second playback task, the K second write tasks are written into the playback database.

5. The method according to claim 4, wherein The method further comprises: When the execution tree of the replication request for the first data is successfully executed, triggering the execution tree of the replication request for the second data to add a second write-back task; the second write-back task is used to write back the at least one second write task of the playback database to the second database corresponding to the at least one second write task; After executing the second write-back task, the at least one second write task is called from the playback database, and the at least one second write task is sent to a second database corresponding to the at least one second write task.

6. A data reading method, characterized in that: The method comprises: receiving a request for rereading first data; Performing syntax analysis on the reread request for the first data to determine an abstract syntax tree of the reread request for the first data; the abstract syntax tree includes N first read task nodes; and N is a positive integer greater than 1. For each of the N first reading task nodes: determining a first database read by the first reading task node and a converter corresponding to the first database; Converting the first read task node into a first read task node in an execution tree of a reread request for the first data through a converter corresponding to the first database, where the execution tree of the reread request for the first data includes: N first read task nodes; When executing N first read task nodes of the execution tree of the reread request of the first data, N first read tasks are respectively sent to N first databases; the first read tasks are used to read the first data from the corresponding first databases.

7. A data writing device, characterized in that: include: a receiving unit, configured to receive a request for copying first data; an abstract syntax tree determining unit, configured to perform syntax analysis on the replication request for the first data and determine an abstract syntax tree of the replication request for the first data; The abstract syntax tree includes N first writing task nodes; Said N is a positive integer greater than 1; a converter determining unit, configured to determine, for each of the N first writing task nodes, a first database written by the first writing task node and a converter corresponding to the first database; a conversion unit, configured to convert the first write task node into a first write task node in an execution tree of a replication request for the first data through a converter corresponding to the first database, the execution tree of the replication request for the first data including: N first write task nodes; A sending unit is used to send N first write tasks to N first databases respectively when executing N first write task nodes of the execution tree of the replication request of the first data; the first write task is used to write the first data into the corresponding first database.

8. A data reading device, characterized in that: include: A receiving unit, configured to receive a request for rereading first data; an abstract syntax tree determining unit, performing syntax analysis on the reread request for the first data, and determining an abstract syntax tree of the reread request for the first data; The abstract syntax tree includes N first read task nodes; Said N is a positive integer greater than 1; a converter determining unit, configured to determine, for each of the N first read task nodes in the abstract syntax tree: a first database read by the first read task node and a converter corresponding to the first database; a conversion unit, configured to convert the first read task node into a first read task node in an execution tree of a reread request for the first data through a converter corresponding to the first database; The execution tree of the reread request of the first data includes: N first read task nodes; A sending unit is used to send N first read tasks to N first databases respectively when executing N first read task nodes of the execution tree of the reread request of the first data; the first read task is used to read the first data from the corresponding first database.

9. A computer device, characterized in that: include: Memory for storing computer programs; A processor is configured to call a computer program stored in the memory, and execute the method according to any one of claims 1 to 5 or the method according to claim 6 according to the obtained program.

10. A computer-readable storage medium, characterized in that The storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method according to any one of claims 1 to 5 or the method according to claim 6.

Images