Data processing method and distributed data storage system

Abstract

The embodiment of the invention provides a data processing method and a distributed data storage system, and the method comprises the steps: responding to a data processing request, and executing and submitting a structure adjustment transaction corresponding to the data processing request; according to the structure adjustment transaction, an initial version of the data storage structure is updated, an updated version of the data storage structure is obtained, and the initial version is a version before the data storage structure is updated; and receiving a data query request, and according to the receiving time of the data query request and the submission time of the structure adjustment transaction, determining to pass through the initial version or the updated version, and responding to the data query request. The data storage node does not need to be locked by requesting the writing node through the reading node, information interaction between the reading node and the writing node is avoided, and data transmission resources of the reading node and the writing node are saved.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of database technology, and in particular, to a data processing method. Background Art

[0002] In a data storage system, data read and write transactions can usually be used to access data stored in the data storage system. Specifically, when a read node needs to read data stored in a data storage node in the data storage system, it can usually send a lock request to the write node. After the write node locks the data storage node, the write node replies to the read node with a locked notification. After the read node receives the locked notification, it can read the data stored in the data storage node, preventing other read nodes or write nodes from accessing the data storage node, thereby avoiding read-write conflicts.

[0003] However, this method of locking data storage nodes requires multiple information interactions between read nodes and write nodes, and the frequency of reading data in data storage nodes is extremely high, resulting in a large amount of data transmission resources of read nodes and write nodes being occupied. In addition, if the read nodes and write nodes crash, the data storage nodes will not be unlocked subsequently, which will also affect the data reading and writing operations of the data storage nodes, further affecting data security. Summary of the invention

[0004] In view of this, an embodiment of this specification provides a data processing method. One or more embodiments of this specification also relate to a distributed data storage system, a computing device, a computer-readable storage medium and a computer program to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a data processing method is provided, which is applied to a distributed data storage system, including:

[0006] In response to a data processing request, executing and submitting a structure adjustment transaction corresponding to the data processing request;

[0007] According to the structure adjustment transaction, an initial version of the data storage structure is updated to obtain an updated version of the data storage structure, wherein the initial version is a version of the data storage structure before being updated;

[0008] A data query request is received, and according to a time when the data query request is received and a time when the structure adjustment transaction is submitted, it is determined whether to respond to the data query request through the initial version or the updated version.

[0009] According to a second aspect of the embodiments of this specification, a distributed data storage system is provided, including a write node and a read node, wherein:

[0010] The write node is configured to execute and submit a structure adjustment transaction corresponding to the data processing request in response to the data processing request;

[0011] The write node is further configured to update an initial version of the data storage structure according to the structure adjustment transaction to obtain an updated version of the data storage structure, wherein the initial version is a version of the data storage structure before being updated;

[0012] The reading node is configured to receive a data query request, and determine whether to respond to the data query request through the initial version or the updated version according to the reception time of the data query request and the submission time of the structure adjustment transaction.

[0013] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0014] Memory and processor;

[0015] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above data processing method are implemented.

[0016] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned data processing method are implemented.

[0017] According to a fifth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned data processing method.

[0018] An embodiment of the present specification provides a data processing method, which is applied to a distributed data storage system, and in response to a data processing request, executes and submits a structural adjustment transaction corresponding to the data processing request; according to the structural adjustment transaction, updates an initial version of a data storage structure to obtain an updated version of the data storage structure, wherein the initial version is a version before the data storage structure is updated; receives a data query request, and determines whether to respond to the data query request through the initial version or the updated version based on the reception time of the data query request and the submission time of the structural adjustment transaction.

[0019] In the above method, by maintaining multiple versions of the data storage structure, the data storage structure of the distributed data storage system is adjusted according to the structure adjustment transaction to obtain an updated version of the data storage structure. After receiving a data query request, it is determined whether to process the data query request through an initial version or an updated version according to the reception time of the data query request and the submission time of the structure adjustment transaction. When processing the data query request, the initial version or updated version of the data storage structure is read to realize the reading of the data, and there is no need to request the write node to lock the data storage node through the read node, thereby avoiding information interaction between the read node and the write node, saving data transmission resources of the read node and the write node, and even when the read node or the write node is down, the corresponding data can be read through the initial version or the updated version, ensuring that the data reading and writing operations on the data storage node are not affected, and further ensuring data security. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of an application scenario of a data processing method provided by an embodiment of this specification;

[0021] Figure 2 is a flow chart of a data processing method provided by an embodiment of this specification;

[0022] Figure 3 It is a schematic diagram of executing a structure adjustment transaction in a data processing method provided by an embodiment of this specification;

[0023] Figure 4 is a processing flow chart of a data processing method provided by an embodiment of this specification;

[0024] Figure 5 It is a schematic diagram of the structure of a distributed data storage system provided by an embodiment of this specification;

[0025] Figure 6 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0026] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0027] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0028] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0029] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0030] First, the terms involved in one or more embodiments of this specification are explained.

[0031] B+Tree: A self-balancing tree in computer science that keeps data in order. This data structure allows data lookup, sequential access, data insertion, and deletion to be completed in logarithmic time. B+Tree is suitable for storage systems that read and write relatively large blocks of data, such as disks. B+Tree reduces the intermediate process of locating records, thereby speeding up access.

[0032] SMO: Structure Modification Operation, the process of changing the structure of B+Tree.

[0033] Undo: The historical versions of database data are stored in undo. Through undo, users can query the past versions of data or roll back the data to a certain historical version.

[0034] Transaction: A transaction is a collection of a series of strict operations in a database system. Generally speaking, transactions need to meet four characteristics: Atomicity: When operating these instructions, either all are executed successfully or none are executed. As long as one of the instructions fails to execute, all instructions fail to execute, and the data is rolled back to the data state before the instruction was executed. Consistency: The execution of a transaction causes the data to be converted from one state to another, but the integrity of the entire data remains stable. Isolation: If multiple transactions are executed concurrently, the modifications made by each transaction must be isolated from other transactions. Durability: After the transaction is completed, the modifications to the database data are persistently stored.

[0035] MVCC: Multi version concurrency control, is a common concurrency control used in database management systems. MVCC is intended to solve the conflict problem of multiple, long-term read and write operations caused by read-write locks. Under the MVCC mechanism, read operations read historical versions and can be executed concurrently without blocking write operations.

[0036] In this specification, a data processing method is provided. This specification also relates to a distributed data storage system, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0037] See also Figure 1 , Figure 1 A schematic diagram of an application scenario of a data processing method provided according to an embodiment of the present specification is shown.

[0038] Figure 1 It includes a terminal device 102 and a distributed data storage system 104.

[0039] In specific implementation, the data processing method provided in the embodiments of this specification can be applied to the distributed data storage system 104 of the e-commerce platform. The distributed data storage system 104 includes multiple data storage nodes, and each data storage node can store the daily business data of different merchants. For example, the data storage node 1042 stores the daily business data of merchant A, and the data storage node 1044 stores the daily business data of merchant B.

[0040] like Figure 1As shown, the data management personnel of the e-commerce platform can send a data processing request to the distributed data storage system 104 through the end-side device 102. The data processing request can be, for example, "write the daily business data of merchant A in June". The distributed data storage system 104 responds to the data processing request and executes the data processing transaction corresponding to the data processing request for the data storage node 1042. Specifically, when the data storage node 1042 is written with the daily business data of merchant A in June, the storage space of the data storage node 1042 is insufficient, and the data processing transaction fails to execute. At this time, the structure adjustment transaction can be executed and submitted, that is, a new data storage node 1046 is added to store the daily business data of merchant A in June. At this time, the data storage structure of the distributed data storage system 104 changes, and the initial version of the data storage structure is updated to obtain the updated version of the data storage structure. In the case where the data management personnel send a data query request for the data storage node 1042 to the distributed data storage system 104 through the end-side device 102, the distributed data storage system can determine whether to process the data query request through the initial version or the updated version according to the reception time of the data query request and the submission time of the structure adjustment transaction. To avoid conflicts in data read and write operations in distributed data storage systems.

[0041] See also Figure 2 , Figure 2 A flow chart of a data processing method provided according to an embodiment of the present specification is shown, which is applied to a distributed data storage system and specifically includes the following steps.

[0042] Step 202: In response to the data processing request, execute and submit the structure adjustment transaction corresponding to the data processing request.

[0043] Specifically, a distributed data storage system may include multiple data storage nodes, each of which may be used to store data. For example, for a distributed data storage system within a product manufacturing enterprise, each data storage node may store production data and sales data of different products. Alternatively, for a distributed data storage system of a payment platform, each data storage node may store payment data and transfer data of platform users.

[0044] In specific implementation, after responding to a data processing request, a data processing transaction corresponding to the data processing request may be executed. When it is determined that the data processing transaction fails to be executed, a structural adjustment transaction corresponding to the data processing request is executed and submitted.

[0045] Among them, the data processing request can be understood as a processing request initiated for the data stored in the distributed data storage system, such as a data write request, a data modification request, and a data deletion request. The data processing transaction corresponding to the data processing request can be understood as a processing transaction for the data stored in the distributed data storage system. When the data processing request is a data write request, the data processing transaction is a data write transaction, and when the data processing request is a data modification request, the data processing transaction is a data modification transaction. The structural adjustment transaction can be understood as an SMO transaction, and the SMO transaction can be used to adjust the data storage structure. Determine that the data processing transaction execution failed, for example, it may be that there is insufficient storage space in the data storage node corresponding to the data processing transaction.

[0046] Based on this, a processing transaction corresponding to a processing request initiated for data stored in a distributed data storage system can be executed in response to the processing request. When it is determined that the storage space in the data storage node corresponding to the processing transaction is insufficient, a structure adjustment transaction corresponding to the data processing request is executed and submitted, and the data storage structure is adjusted according to the structure adjustment transaction.

[0047] In practical applications, the write node of the distributed data storage system can be used to respond to the data processing request, execute and submit the structure adjustment transaction corresponding to the data processing request.

[0048] For example, in a distributed data storage system within a product manufacturing enterprise, in response to a data write request for the production data of product A stored in the distributed data storage system, the production data of product A on June 20 can be written to the distributed data storage system, and the data write transaction corresponding to the data write request can be executed. When it is determined that there is insufficient storage space in the data storage node corresponding to the data write transaction, the structural adjustment transaction corresponding to the data write request can be executed and submitted, and the data storage structure can be adjusted according to the structural adjustment transaction.

[0049] In practical applications, B+Tree can be used to query data stored in a distributed data storage system. It is understandable that since the distributed data storage system includes multiple data storage nodes, each data storage node stores different data. When querying data from the distributed data storage system, it is necessary to search each data storage node in turn to query the data. This method results in a long query time and low query efficiency. Therefore, B+Tree can be used. B+Tree is a multi-way balanced search tree that can be used to organize row record data in a distributed data storage system. B+Tree can be understood as a data storage structure of a distributed data storage system, which can search along the B+Tree to find row records that meet the conditions, thereby querying data. When adding or deleting data to a distributed data storage system, an increase or decrease in data storage nodes may occur. For example, when writing data to a data storage node, if the storage space of the data storage node is insufficient, another data storage node will be added. At this time, due to the change in the number of data storage nodes, the data storage structure (i.e., B+Tree) of the distributed data storage system changes accordingly. Based on this, the initial version before the data storage structure changes can be stored in the version storage space, so as to realize the subsequent acquisition of the initial version of the data storage structure and further realize data query. The specific implementation method is as follows:

[0050] Before executing and submitting the structure adjustment transaction corresponding to the data processing request, the method further includes:

[0051] An initial version of the data storage structure is determined, and the initial version is stored in a version storage space.

[0052] The version storage space can be understood as a storage space in a distributed data storage system, which can be a data storage node. The initial version of the data storage structure can be understood as a version before the structure adjustment transaction is executed and when the data storage structure is unchanged.

[0053] Based on this, before executing and submitting the structure adjustment transaction corresponding to the data processing request, the initial version of the data storage structure corresponding to the distributed data storage system can be determined and the initial version can be retained in the version storage space.

[0054] In practical applications, the version storage space may be an undo tablespace in a distributed data storage structure, which is used to store historical versions (i.e., initial versions) of the data storage structure. The initial version of the data storage structure may include only changed data storage nodes. For example, the data storage structure includes three data storage nodes. After the data storage structure is adjusted according to the structure adjustment transaction, one data storage node is added to the data storage structure. At this time, the initial version of the data storage structure includes three data storage nodes, and the updated version includes four data storage nodes. When retaining the initial version to the version storage space, only the data storage nodes that have changed in the initial version may be retained. The changes here may be changes in the association relationship (i.e., pointers) between the data storage nodes.

[0055] Continuing with the above example, for a distributed data storage system within a product manufacturing enterprise, the data storage structure corresponding to the system includes data storage node 1 and data storage node 2, wherein data storage node 1 stores the production data of product A, and data storage node 2 stores the production data of product B, and data storage node 1 points to data storage node 2 (the pointing relationship between data storage nodes indicates that data storage node 1 and data storage node 2 are adjacent to each other. During the data query process, when querying production data, data storage node 1 and data storage node 2 can be searched in sequence along the data storage structure to obtain the production data to be queried). Before executing the above structural adjustment transaction, the initial version of the data storage structure can be determined, "the data storage structure includes data storage node 1 and data storage node 2, and data storage node 1 points to data storage node 2", and the initial version can be stored in the version storage space.

[0056] In addition, the address information of the version storage space may also be recorded in the data storage node corresponding to the data processing request.

[0057] In summary, by determining the initial version of the data storage structure corresponding to the distributed data storage system before executing the structure adjustment transaction, and retaining and storing the initial version in the version storage space, it is convenient to read data later by obtaining the initial version of the data storage structure.

[0058] In specific implementation, since the data is stored in different data storage nodes in the distributed data storage system, based on this, the corresponding structure adjustment transaction can be executed in the data storage node corresponding to the data processing request. The specific implementation method is as follows:

[0059] The executing and submitting the structure adjustment transaction corresponding to the data processing request includes:

[0060] Determine the data storage node and structure adjustment transaction corresponding to the data processing request;

[0061] Execute and submit the structure adjustment transaction corresponding to the data processing request at the data storage node.

[0062] The data storage node corresponding to the data processing request may be understood as the data storage node corresponding to the data to be processed carried in the data processing request.

[0063] Based on this, the data to be processed carried by the data processing request can be determined, and the data storage node and structure adjustment transaction corresponding to the data to be processed can be determined, and the structure adjustment transaction can be executed and submitted at the data storage node.

[0064] During specific implementation, the data storage node corresponding to the data to be processed may be determined according to the data storage structure (ie, B+Tree).

[0065] Using the above example, the production data of product A on June 20 carried in the data write request can be determined, and the data storage node 1 corresponding to the production data of product A can be determined, and the structure adjustment transaction can be executed and submitted on the data storage node 1.

[0066] In summary, by executing the structure adjustment transaction in the corresponding data storage node, it can be ensured that there is sufficient storage space to process the data processing request in the future.

[0067] In specific implementation, when SMO occurs, the write node initiates a structure adjustment transaction (i.e., an internal sub-transaction), and can allocate a storage space in the distributed data storage system as the transaction slot (i.e., transaction execution space) of the sub-transaction. The specific implementation method is as follows:

[0068] After executing and submitting the structure adjustment transaction corresponding to the data processing request, the method further includes:

[0069] Determine the transaction execution space;

[0070] The transaction information of the structure adjustment transaction is stored in the transaction execution space, and the address information of the transaction execution space is recorded in the data storage node.

[0071] The transaction execution space can be used to record the transaction information of the structure adjustment transaction. The transaction information can be, for example, the transaction identification information and the transaction submission number. The transaction submission number can be understood as a number that increases monotonically as the transaction is submitted. For example, for data storage node 1, structure adjustment transaction 1 is executed and submitted, and the transaction submission number is 10 at this time. For data storage node 1, structure adjustment transaction 2 is continued to be executed and submitted, and the transaction submission number is monotonically increased to 11 at this time.

[0072] Based on this, a transaction execution space can be allocated in the distributed data storage system, and the identification information and the commit number of the structure adjustment transaction can be stored in the transaction execution space, and the address information of the transaction execution space can be recorded in the data storage node.

[0073] In practical applications, data storage nodes can be understood as data pages, and fields can be added to the data storage nodes. The first field can be used to indicate the transaction commit number of the structural adjustment transaction that most recently modified the data page (i.e., the commit number of the structural adjustment transaction that most recently executed on the data storage node), the second field can be used to indicate the address information of the transaction execution space of the structural adjustment transaction that most recently modified the data page, and the third field can be used to indicate the rollback segment address (i.e., the address information of the version storage space hereinafter). Through the rollback segment address, the data page can be traced back to the historical version (i.e., the initial version) before the structural adjustment transaction that most recently modified the data page. When executing a structural adjustment transaction (i.e., executing an SMO transaction to modify a data page), the historical version of the data page can be stored in the undo tablespace in the distributed data storage system, and the third field on the data page can be updated to point to the historical version data.

[0074] Continuing with the above example, the data write transaction can be executed and submitted in data storage node 1, and the transaction ID (i.e., identification information) and transaction submission number of the data write transaction can be stored in the transaction execution space, and the address information of the transaction execution space can be recorded in data storage node 1.

[0075] In summary, by allocating transaction execution space, the transaction information of the submitted structural adjustment transaction can be stored, so that the submission time and other information of the structural adjustment transaction can be determined based on the transaction information of the structural adjustment transaction, and it can be further determined whether the data query is queried through the initial version or the updated version.

[0076] Step 204: According to the structure adjustment transaction, the initial version of the data storage structure is updated to obtain an updated version of the data storage structure, wherein the initial version is a version of the data storage structure before being updated.

[0077] Specifically, the data storage structure may be adjusted according to the structure adjustment transaction, and the initial version of the data storage structure may be updated according to the adjustment information to obtain an updated version of the data storage structure.

[0078] In a specific implementation, updating the initial version of the data storage structure according to the structure adjustment transaction to obtain the updated version of the data storage structure includes:

[0079] According to the structure adjustment transaction, adjusting the data storage structure;

[0080] The initial version of the data storage structure is updated according to the adjustment information of the data storage structure to obtain an updated version of the data storage structure.

[0081] The structure adjustment transaction will cause the data storage nodes included in the data storage structure to change, which can be understood as a change in the number of data storage nodes.

[0082] Based on this, a structural adjustment transaction can be executed. The execution of the structural adjustment transaction will adjust the data storage structure, resulting in changes in the data storage structure. The initial version of the data storage structure can be updated according to the adjustment information of the data storage structure to obtain an updated version of the data storage structure.

[0083] In practical applications, adjusting the data storage structure includes but is not limited to increasing data storage nodes or reducing data storage nodes.

[0084] Continuing with the above example, a data write transaction is executed on data storage node 1, that is, the production data of product A is written to data storage node 1. If the storage space of data storage node 1 is insufficient, the data write transaction fails. At this time, a structure adjustment transaction can be executed for data storage node 1, and a new data storage node 3 is added to continue writing the production data of product A (that is, continue to execute the data write transaction). At this time, the number of data storage nodes changes, and accordingly, the data storage structure changes, and an updated version of the data storage structure is obtained: "The data storage structure includes data storage node 1, data storage node 3 and data storage node 2, and data storage node 1 points to data storage node 3, and data storage node 3 points to data storage node 2."

[0085] In summary, by recording the address information of the version storage space in the data storage node, different versions of the data storage structure can be obtained when querying the data in the data storage node later.

[0086] Step 206: receiving a data query request, and determining whether to respond to the data query request through the initial version or the updated version according to the receiving time of the data query request and the commit time of the structure adjustment transaction.

[0087] In practical applications, the read nodes in the distributed data storage system can be used to receive data query requests, and based on the reception time of the data query request and the submission time of the structure adjustment transaction, it is determined whether to respond to the data query request through the initial version or the updated version.

[0088] In specific implementation, the commit time of the structure adjustment transaction can be determined according to the commit number of the structure adjustment transaction. The transaction commit number of the structure adjustment transaction can be obtained from the data storage node or the transaction execution space. The specific implementation method is as follows:

[0089] The receiving of the data query request comprises:

[0090] receiving a data query request for the data storage node;

[0091] Correspondingly, after receiving the data query request, the method further includes:

[0092] When the transaction commit number is obtained in the data storage node, the commit time of the structure adjustment transaction corresponding to the data storage node is determined according to the transaction commit number.

[0093] Specifically, after receiving a data query request for a data storage node, when the data storage node obtains a transaction commit number, the commit time of the structure adjustment transaction corresponding to the data storage node can be determined according to the transaction commit number.

[0094] In actual applications, due to the atomicity of transactions, data queries cannot query the intermediate state of the structural adjustment transaction before the transaction is committed. In other words, the transaction commit number cannot be obtained during the transaction execution. After the transaction is committed, if the transaction commit number recorded on the data storage node is still null, the transaction commit number can be obtained from the transaction execution space. The transaction commit number can be the transaction commit number corresponding to the last structural adjustment transaction. The specific implementation method is as follows:

[0095] After receiving the data query request, the method further includes:

[0096] When the transaction submission number is not obtained in the data storage node, obtaining address information of the transaction execution space in the data storage node;

[0097] Determine the transaction execution space according to the address information of the transaction execution space, and obtain the transaction submission number in the transaction execution space;

[0098] The commit time of the structure adjustment transaction corresponding to the data storage node is determined according to the transaction commit number.

[0099] Specifically, when the data storage node fails to obtain the transaction submission number, the address information of the transaction execution space can be obtained at the data storage node, and the corresponding transaction execution space can be determined based on the address information of the transaction execution space, thereby obtaining the transaction submission number in the transaction execution space, so as to determine the submission time of the structural adjustment transaction corresponding to the data storage node based on the transaction submission number.

[0100] In one embodiment of the present specification, the transaction submission number can also be determined as the submission time of the structure adjustment transaction. The transaction submission number can be obtained, indicating that the structure adjustment transaction has been submitted. If the query time of the data query request is before the submission of the structure adjustment transaction, it means that the updated version of the data storage structure obtained after the submission of the structure adjustment transaction is not required by this data query request. Based on this, the third field on the data storage node can be used to trace back to the previous historical version (i.e., the initial version) of the data storage structure.

[0101] In addition, when the data storage node obtains the transaction submission number, after obtaining the transaction submission number from the transaction execution space, the transaction submission number can be recorded in the data storage node, so that the next time it is accessed, it can avoid accessing the transaction execution space and directly obtain the transaction submission number from the data storage node.

[0102] In summary, by obtaining the transaction commit number, it is possible to determine whether the data required by the data query request is an updated version or an initial version, thereby achieving subsequent acquisition of the initial version or the updated version.

[0103] In a specific implementation, determining, based on the reception time of the data query request and the submission time of the structure adjustment transaction, to respond to the data query request through the initial version or the updated version includes:

[0104] In the case where it is determined that the receiving time of the data query request is before the commit time of the structure adjustment transaction, determining to respond to the data query request through the initial version of the data storage structure; or

[0105] In the case where it is determined that the receiving time of the data query request is after the commit time of the structure adjustment transaction, it is determined to respond to the data query request through the updated version of the data storage structure.

[0106] If the data query request is received before the structural adjustment transaction is submitted, it means that the updated version is not required for this data query request. If the data query request is received after the structural adjustment transaction is submitted, it means that the updated version is required for this data query request.

[0107] Based on this, when it is determined that the receiving time of the data query request is before the commit time of the structure adjustment transaction, the address information of the version storage space can be obtained in the data storage node corresponding to the data query request, and the version storage space can be determined according to the address information, and the initial version of the data storage structure can be obtained in the version storage space, and the data query request can be processed according to the initial version. When it is determined that the receiving time of the data query request is after the commit time of the structure adjustment transaction, the updated version of the data storage structure can be obtained in the table space, and the data query request can be processed according to the updated version.

[0108] The tablespace can be understood as a storage space for storing the latest version (i.e., updated version). In practical applications, after obtaining an updated version of a data storage structure in the tablespace, if it is determined that the updated version is not the version corresponding to the data query request, the historical version of the data query structure can be searched through the version storage space (i.e., undo tablespace) until the historical version corresponding to the data query request is obtained.

[0109] In one embodiment of the present specification, a data query request carries a transaction submission number, and the data query request can be processed by determining the initial version or updated version of the data storage structure based on the transaction submission number carried in the data query request and the transaction submission number of the structure adjustment transaction.

[0110] In practical applications, see Figure 3 , Figure 3 This is a schematic diagram of executing a structure adjustment transaction in a data processing method provided according to an embodiment of the present specification, and the specific implementation steps are as follows.

[0111] Step 302: For data page 5, execute the SMO transaction and allocate a transaction slot, wherein the transaction slot address is 0x021002210585.

[0112] Specifically, Figure 3 As shown, when the B+Tree (i.e., data storage structure) is in version V(n), an SMO transaction (i.e., structure adjustment transaction) is executed. The transaction ID of the SMO transaction is 45. The SMO transaction is a split action (i.e., the original data page 8 is split into new data pages 8 and 9). Specifically, the SMO transaction modifies three data pages respectively, adds a pointer to data page 9 to data page 5, adds a right sibling pointer to data page 9 to data page 8, and newly allocates data page 9 to save more row records (i.e., to store more data), and finally obtains the V(n+1) version of the B+Tree. Among them, a right sibling pointer is added to data page 8 to point to data page 9, which can be understood as data page 9 is a sibling node of data page 8.

[0113] It is understandable that before the modification action on data page 5 occurs, the data of data page 5 will be saved in the Undo tablespace at address 0x1270f3570200. In other words, when saving the historical version (ie, the initial version), only the modified data storage node can be saved.

[0114] Step 304: Modify the rollback segment address field of data page 5 to 0x1270f3570200, the transaction slot address to 0x021002210585, and the transaction commit number to NULL value (indicating that the SMO transaction has not been committed and there is no transaction commit number; or that the SMO transaction has been committed, but the transaction commit number field on the data page is null)

[0115] Step 306: Modify data page 5.

[0116] Specifically, the SMO will add a pointer to data page 9 to the original data page 5, and the original data page 8 will be split into new data page 8 and data page 9, which can also be understood as adding data page 9.

[0117] In addition, after the SMO transaction is completed, the SMO transaction is committed, and the obtained transaction commit number TCN=90 is updated to the transaction commit number field on the transaction slot.

[0118] Specifically, a data query request for data page 5 is received, wherein the data query request carries a transaction submission number query_TCN=80, while the transaction submission number TCN of the above SMO transaction is 90, indicating that the data query request was initiated earlier than the submission of the above SMO transaction. The submission number obtained on data page 5 is NULL, that is, the transaction submission number cannot be obtained. The corresponding transaction slot is searched through the transaction slot address, and it is found that the transaction has been submitted, and the transaction submission number is 90.

[0119] Step 308: Since the data query request occurs before the transaction is committed, this version is not required for this query. It will trace back to the previous historical version of the data page 5 through the rollback segment address field.

[0120] Specifically, the historical version shows that data page 5 has only two subpages, namely data page 7 and data page 8.

[0121] Step 310: Update the transaction commit number TCN equal to 90 in the transaction commit number field of data page 5.

[0122] Specifically, the next query can avoid accessing the transaction slot again.

[0123] In summary, in the above method, by maintaining multiple versions of the data storage structure, the data storage structure of the distributed data storage system is adjusted according to the structure adjustment transaction to obtain an updated version of the data storage structure. After receiving the data query request, it is determined whether to process the data query request through the initial version or the updated version according to the reception time of the data query request and the submission time of the structure adjustment transaction. When processing the data query request, the initial version or the updated version of the data storage structure is read to realize the reading of the data, and there is no need to request the write node to lock the data storage node through the read node, so as to avoid information interaction between the read node and the write node, save the data transmission resources of the read node and the write node, and even in the case of downtime of the read node or the write node, the corresponding data can be read through the initial version or the updated version, so as to ensure that the data reading and writing operations of the data storage node are not affected, and further ensure data security.

[0124] The following combination Figure 4 , taking the application of the data processing method provided in this specification in the distributed data storage system of the payment platform as an example, the data processing method is further explained. Figure 4 A processing flow chart of a data processing method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0125] Step 402: In response to the data processing request, execute and submit the structure adjustment transaction corresponding to the data processing request.

[0126] Specifically, the distributed data storage system of the payment platform may store the user's payment records.

[0127] The data processing request may be, for example, a data writing request for writing the user's payment data.

[0128] Step 404: According to the structure adjustment transaction, the initial version of the data storage structure is updated to obtain an updated version of the data storage structure, wherein the initial version is a version before the data storage structure is updated.

[0129] Step 406: Store the initial version and the updated version of the data storage structure into a version storage space, and record the address information of the version storage space into the data storage node.

[0130] Step 408: receiving a data query request, and determining whether to respond to the data query request through the initial version or the updated version according to the receiving time of the data query request and the commit time of the structure adjustment transaction.

[0131] The data query request may be, for example, a request to query the user's payment data.

[0132] It can be understood that the specific implementation process is similar to the above and will not be repeated here.

[0133] In summary, in the above method, by maintaining multiple versions of the data storage structure, the data storage structure of the distributed data storage system is adjusted according to the structure adjustment transaction to obtain an updated version of the data storage structure. After receiving the data query request, it is determined whether to process the data query request through the initial version or the updated version according to the reception time of the data query request and the submission time of the structure adjustment transaction. When processing the data query request, the initial version or the updated version of the data storage structure is read to realize the reading of the data, and there is no need to request the write node to lock the data storage node through the read node, so as to avoid information interaction between the read node and the write node, save the data transmission resources of the read node and the write node, and even in the case of downtime of the read node or the write node, the corresponding data can be read through the initial version or the updated version, so as to ensure that the data reading and writing operations of the data storage node are not affected, and further ensure data security.

[0134] Corresponding to the above method embodiment, this specification also provides a distributed data storage system embodiment. Figure 5 FIG. 1 shows a schematic diagram of a distributed data storage system provided by an embodiment of the present specification. Figure 5 As shown, the system 500 includes:

[0135] Write node 502 and read node 504, where

[0136] The write node 502 is configured to execute and submit a structure adjustment transaction corresponding to the data processing request in response to the data processing request;

[0137] The write node 502 is further configured to update the initial version of the data storage structure according to the structure adjustment transaction to obtain an updated version of the data storage structure, wherein the initial version is a version before the data storage structure is updated;

[0138] The read node 504 is configured to receive a data query request, and determine whether to respond to the data query request through the initial version or the updated version according to the reception time of the data query request and the submission time of the structure adjustment transaction.

[0139] In an optional embodiment, the write node 502 is further configured as follows:

[0140] Determine the data storage node and structure adjustment transaction corresponding to the data processing request;

[0141] Execute and submit the structure adjustment transaction corresponding to the data processing request at the data storage node.

[0142] In an optional embodiment, the write node 502 is further configured as follows:

[0143] Determine the transaction execution space;

[0144] The transaction information of the structure adjustment transaction is stored in the transaction execution space, and the address information of the transaction execution space is recorded in the data storage node.

[0145] In an optional embodiment, the reading node 504 is further configured as follows:

[0146] receiving a data query request for the data storage node;

[0147] When the transaction commit number is obtained in the data storage node, the commit time of the structure adjustment transaction corresponding to the data storage node is determined according to the transaction commit number.

[0148] In an optional embodiment, the reading node 504 is further configured as follows:

[0149] When the transaction submission number is not obtained in the data storage node, obtaining address information of the transaction execution space in the data storage node;

[0150] Determine the transaction execution space according to the address information of the transaction execution space, and obtain the transaction submission number in the transaction execution space;

[0151] The commit time of the structure adjustment transaction corresponding to the data storage node is determined according to the transaction commit number.

[0152] In an optional embodiment, the reading node 504 is further configured as follows:

[0153] In the case where it is determined that the receiving time of the data query request is before the commit time of the structure adjustment transaction, determining to respond to the data query request through the initial version of the data storage structure; or

[0154] In the case where it is determined that the receiving time of the data query request is after the commit time of the structure adjustment transaction, it is determined to respond to the data query request through the updated version of the data storage structure.

[0155] In an optional embodiment, the write node 502 is further configured as follows:

[0156] In the case where the data storage node is determined to be changed according to the transaction execution result, determining a data storage structure change;

[0157] According to the change information of the data storage structure, the initial version of the data storage structure is updated to obtain an updated version of the data storage structure;

[0158] An updated version of the data storage structure is stored in a version storage space, and address information of the version storage space is recorded in the data storage node.

[0159] In an optional embodiment, the write node 502 is further configured as follows:

[0160] An initial version of a data storage structure is determined, and the initial version is stored in the version storage space.

[0161] In summary, in the above system, by maintaining multiple versions of the data storage structure, the data storage structure of the distributed data storage system is adjusted according to the structure adjustment transaction to obtain an updated version of the data storage structure. After receiving a data query request, it is determined whether to process the data query request through the initial version or the updated version according to the reception time of the data query request and the submission time of the structure adjustment transaction. When processing the data query request, the initial version or the updated version of the data storage structure is read to realize the reading of the data, and there is no need to request the write node to lock the data storage node through the read node, thereby avoiding information interaction between the read node and the write node, saving data transmission resources of the read node and the write node, and even when the read node or the write node is down, the corresponding data can be read through the initial version or the updated version, ensuring that the data reading and writing operations on the data storage node are not affected, and further ensuring data security.

[0162] The above is a schematic scheme of a distributed data storage system of this embodiment. It should be noted that the technical scheme of the distributed data storage system and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the distributed data storage system can be referred to the description of the technical scheme of the above data processing method.

[0163] Figure 6 The block diagram of a computing device 600 according to an embodiment of the present specification is shown. The components of the computing device 600 include but are not limited to a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and the database 650 is used to store data.

[0164] The computing device 600 also includes an access device 640 that enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 640 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0165] In one embodiment of the present application, the above components of the computing device 600 and Figure 6 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 6 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0166] The computing device 600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 600 may also be a mobile or stationary server.

[0167] The processor 620 is used to execute the following computer executable instructions, which implement the steps of the above data processing method when executed by the processor.

[0168] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above data processing method.

[0169] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which can implement the steps of the above-mentioned data processing method when executed by a processor.

[0170] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above data processing method.

[0171] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned data processing method.

[0172] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be referred to the description of the technical scheme of the above data processing method.

[0173] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0174] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0175] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0176] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0177] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A data processing method, applied to a distributed data storage system, comprising: In response to a data processing request, executing and submitting a structure adjustment transaction corresponding to the data processing request; According to the structure adjustment transaction, an initial version of the data storage structure is updated to obtain an updated version of the data storage structure, wherein the initial version is a version of the data storage structure before being updated; A data query request is received, and according to a time when the data query request is received and a time when the structure adjustment transaction is submitted, it is determined whether to respond to the data query request through the initial version or the updated version.

2. The method according to claim 1, wherein the step of executing and submitting the structure adjustment transaction corresponding to the data processing request comprises: Determine the data storage node and structure adjustment transaction corresponding to the data processing request; Execute and submit the structure adjustment transaction corresponding to the data processing request at the data storage node.

3. The method according to claim 2, after the data storage node executes and submits the structure adjustment transaction corresponding to the data processing request, further comprising: Determine the transaction execution space; The transaction information of the structure adjustment transaction is stored in the transaction execution space, and the address information of the transaction execution space is recorded in the data storage node.

4. The method according to claim 3, wherein receiving a data query request comprises: receiving a data query request for the data storage node; After receiving the data query request, the method further includes: When a transaction commit number is obtained in the data storage node, a commit time of the structure adjustment transaction corresponding to the data storage node is determined according to the transaction commit number.

5. The method according to claim 4, after receiving the data query request, further comprising: When the transaction submission number is not obtained in the data storage node, obtaining address information of the transaction execution space in the data storage node; Determine the transaction execution space according to the address information of the transaction execution space, and obtain the transaction submission number in the transaction execution space; The commit time of the structure adjustment transaction corresponding to the data storage node is determined according to the transaction commit number.

6. The method according to claim 1, wherein determining, based on the time when the data query request is received and the time when the structure adjustment transaction is submitted, whether to respond to the data query request through the initial version or the updated version comprises: In the case where it is determined that the receiving time of the data query request is before the commit time of the structure adjustment transaction, determining to respond to the data query request through the initial version of the data storage structure; or In the case where it is determined that the receiving time of the data query request is after the commit time of the structure adjustment transaction, it is determined to respond to the data query request through the updated version of the data storage structure.

7. The method according to claim 2, wherein the initial version of the data storage structure is updated according to the structure adjustment transaction to obtain the updated version of the data storage structure, comprising: According to the structure adjustment transaction, adjusting the data storage structure; The initial version of the data storage structure is updated according to the adjustment information of the data storage structure to obtain an updated version of the data storage structure.

8. The method according to claim 7, before executing and submitting the structure adjustment transaction corresponding to the data processing request, further comprising: An initial version of a data storage structure is determined, and the initial version is stored in the version storage space.

9. A distributed data storage system, comprising a write node and a read node, wherein: The write node is configured to execute and submit a structure adjustment transaction corresponding to the data processing request in response to the data processing request; The write node is further configured to update an initial version of the data storage structure according to the structure adjustment transaction to obtain an updated version of the data storage structure, wherein the initial version is a version of the data storage structure before being updated; The reading node is configured to receive a data query request, and determine whether to respond to the data query request through the initial version or the updated version according to the reception time of the data query request and the submission time of the structure adjustment transaction.

10. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

Images