A method and device for acquiring logical data

Abstract

This specification provides a method and device for obtaining logical data, which is applied to a read-only node of a cloud database system, including: obtaining the at least one redo log when the read-only node plays back at least one redo log; parsing the at least one redo log to obtain the data changed by the data change instruction corresponding to the at least one redo log; obtaining index information from the at least one redo log; and forming the index information and the data changed by the data change instruction into logical data corresponding to the at least one redo log. In this way, through the playback process of the read-only node, logical data is extracted from the redo log to complete the acquisition of logical data, and the read-write node does not need to save duplicate logs, which reduces the IO cost of the read-write node and achieves an effect similar to that of opening the binary log without opening the binary log.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of computer applications, and in particular, to a method and device for acquiring logical data. Background Art

[0002] For a cloud database system that supports one write and multiple reads, it generally includes a read-write node and multiple read-only nodes. The read-write node can change the data stored in the cloud database system, while the read-only node can only read the data stored in the cloud database system. All nodes share the same storage space, thus completing one write and multiple reads for the relational database in the shared storage space.

[0003] Read-write nodes generally make changes to data stored in the cloud database system by executing data change transactions submitted by users. In order to improve processing efficiency, read-write nodes do not directly write the changed data to the disk (i.e., shared storage space) when executing data change transactions, but write the data change instructions contained in the data change transaction to the redo log (redo log) in the shared storage space. Based on the above premise, for read-only nodes, in order to enable them to obtain the latest version of the database, they also need to synchronize the redo logs written by the read-write nodes to obtain the latest version of the database by replaying the redo logs.

[0004] In the above case, since the redo log records the physical data, that is, the redo log records which data in which physical page has been modified, it is not possible to obtain the logical meaning of the modified data by only using the redo log itself. In other words, in the related art, it is not possible to know the logical data corresponding to the modification by only using the redo log.

[0005] In the related art, the above problem is generally solved by opening the binary log. Both the binary log and the redo log are logs of the cloud database system, but the binary log is a log that records logical data, while the redo log is a log that records physical data. Although the above problem can be solved by recording the binary log, it will cause the read-write node to record two duplicate logs, which will cause the read-write node to pay double IO cost and low efficiency. Summary of the invention

[0006] In view of this, one or more embodiments of the present specification provide a method and device for acquiring logic data.

[0007] According to a first aspect of one or more embodiments of this specification, a logical data acquisition method is proposed, which is applied to a read-only node of a cloud database system. The method includes:

[0008] In the case where the read-only node plays back at least one redo log, obtaining the at least one redo log;

[0009] Parsing the at least one redo log to obtain data changed by the data change instruction corresponding to the at least one redo log;

[0010] Obtaining index information from the at least one redo log;

[0011] The index information and the data changed by the data change instruction are combined into logical data corresponding to the at least one redo log.

[0012] According to a second aspect of one or more embodiments of this specification, a logical data acquisition device is provided, which is applied to a read-only node of a cloud database system, and the device includes:

[0013] A redo log acquisition module, used for acquiring at least one redo log when the read-only node plays back at least one redo log;

[0014] A changed data acquisition module, used for parsing the at least one redo log and acquiring data changed by a data change instruction corresponding to the at least one redo log;

[0015] An index information acquisition module, used to acquire index information from the at least one redo log;

[0016] The logic data acquisition module is used to form the logic data corresponding to the at least one redo log by combining the index information and the data changed by the data change instruction.

[0017] According to a third aspect of an embodiment of this specification, a cloud database system is provided, the cloud database system comprising a read-write node and a plurality of read-only nodes, any read-only node executing the aforementioned logical data acquisition method.

[0018] According to a fourth aspect of the embodiments of this specification, there is provided a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by a processor, the aforementioned logic data acquisition method is implemented.

[0019] According to a fifth aspect of the embodiments of this specification, a computer device is provided, the computer device comprising:

[0020] processor;

[0021] a memory for storing processor-executable instructions;

[0022] The processor implements the aforementioned logic data acquisition method by running the executable instructions.

[0023] According to a sixth aspect of the embodiments of this specification, a computer program is provided, which implements the aforementioned logic data acquisition method when executed.

[0024] The present specification provides a method and device for obtaining logical data, which are applied to a read-only node of a cloud database system, and include: obtaining the at least one redo log when the read-only node plays back the at least one redo log; parsing the at least one redo log to obtain data changed by a data change instruction corresponding to the at least one redo log; obtaining index information from the at least one redo log; and combining the index information and the data changed by the data change instruction into logical data corresponding to the at least one redo log.

[0025] In this way, through the playback process of the read-only node, logical data is extracted from the redo log to complete the acquisition of logical data. The read-write node does not need to save duplicate logs, which reduces the IO cost of the read-write node and achieves an effect similar to that of opening the binary log without opening the binary log. In addition, the above method is completed during the playback process, that is, the redo log obtained by the playback process of the redo log of the read-only node itself is reused, so that logical data can be obtained through the redo log and can be played back, and logical data can be extracted at a lower cost. In addition, there is no need for the read-write node to record additional binary logs, which saves storage space and improves the data writing speed.

[0026] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0028] Figure 1 It is a flowchart of a logic data acquisition method shown in this specification according to an exemplary embodiment.

[0029] Figure 2 It is a block diagram of a logic data acquisition device according to an exemplary embodiment of the present specification.

[0030] Figure 3 This is a hardware structure diagram of an electronic device where a logic data acquisition device is located according to an exemplary embodiment of this specification. DETAILED DESCRIPTION

[0031] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0032] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0033] Next, the background on which the method provided in this specification is based is described.

[0034] The method provided in this specification is for a cloud database (generally a relational database) with one-write-multiple-read capabilities. In order to realize the rapid processing of read and write instructions, the cloud database generally includes a read-write node and multiple read-only nodes. The read-write node is the master node, which is used to execute the data change instructions in the data change transaction submitted by the user to change the database according to the user's instructions; the read-only node is used to execute data reading instructions.

[0035] In order to ensure writing efficiency in the database, when a data change instruction is received, the data change is not immediately synchronized to the disk. Instead, the data change instruction is temporarily written to the redo log. In other words, when the read-write node receives a data change instruction, it will first write the data change instruction to the redo log, and then execute the data change instruction. After writing to the redo log, it will return a successful write to the user, thereby improving writing efficiency (the speed of returning to the user).

[0036] In some cases, some data change instructions are only synchronized in the redo log, and the corresponding data modifications are not persisted in the shared storage space. In this case, in order to update the latest version of the database, the read-only node needs to obtain the redo log written by the read-write node in the shared storage space and replay (replay means executing the corresponding data change instructions included in the redo log in sequence) the redo log to obtain the latest version of the database, thereby ensuring the synchronization of data on each node.

[0037] After a basic introduction to the various parts of the database system, the requirements in the related technologies will be explained.

[0038] A redo log is a physical log that records the data changed by each data change instruction. In other words, the redo log only records the physical location of each modification on the page. However, for a relational database, knowing only the physical location of each modification does not tell us which table, row, or column the data at that location corresponds to, as well as the logical meaning of these tables / rows / columns (this type of data is called logical data).

[0039] This situation makes it impossible for read-only nodes to perceive logical modifications. The redo log composed of physical data is a physical log with partial logical meaning. Through the redo log, you can only know what was modified at which specific location, but not the meaning of this location and the significance of such modification. Therefore, logical modifications cannot be fully perceived only through the redo log.

[0040] In order to make each node aware of the logical modification, the related technology saves the binary log (Binary Log) at the same time as the redo log. Both the binary log and the redo log are used to save the logs of the modifications made by the read-write nodes to the database. The difference between the two is that the binary log represents the modification through logical data (the binary log can be regarded as the user's SQL sequence), while the redo log represents the modification through physical data (it should be noted that in the related technology, the method of using binary logs to synchronize data between multiple nodes, that is, the method of replaying binary logs by read-only nodes to obtain the latest version of the database is called native replication, and the method of using redo logs to synchronize data between multiple nodes is called physical replication).

[0041] Although this method of recording two logs can enable each node to perceive logical database modifications, it records two copies of data with essentially identical content (one is physical data and the other is logical data), which doubles the storage and data writing costs and affects data writing efficiency.

[0042] In order to solve the above problems, we first consider whether it is possible to save only binary logs to reduce costs and make the logs more readable for maintenance personnel. However, for some database engines (such as InnoDB), redo logs are necessary data for the database engine, and binary logs are logs that can be recorded or not. Existing database engines cannot support the practice of not saving redo logs.

[0043] Considering further that each node is made aware of logical modifications so that maintenance personnel can view database modifications, and in a cloud database system, read-write nodes are responsible for modifying the database. Therefore, it is easy to think of in related technologies that the purpose of making maintenance personnel aware of database modifications can be achieved by modifying the database nodes (read-write nodes), that is, the aforementioned use of binary logs to enable each node to perceive logical modifications (of course, this can also be achieved by modifying the structure of the irrevocable redo log so that the redo log includes more logical information, but this requires significant code modifications, which is very cumbersome and error-prone).

[0044] Although in the database system, only read-write nodes have the right to modify the database, read-only nodes will replay the redo logs to complete data synchronization, so the purpose of sensing database system modifications can be achieved through the logical changes in the data replay process of the read-only nodes. Although in some cases some read-only nodes only replay part of the redo logs, due to the large number of read-only nodes, the data replayed by all read-only nodes can always cover most of the data in the database.

[0045] Therefore, it is considered that logical data can be acquired during the playback of the read-only node.

[0046] Furthermore, during the playback of the read-only node, the redo log can be obtained, and the redo log is essentially the same as the content recorded in the binary log. Therefore, by parsing the redo log, the logical data corresponding to the redo log can be obtained.

[0047] As for how to analyze it specifically, we should consider that logical data should include the logical meaning and content of the changed data. As for the logical meaning, it can be represented by the index corresponding to the changed data, and the redo log already includes some simple indexes, which can be directly used as the index of the changed data; as for the content, the redo log includes what content of the physical page is modified, then according to the arrangement of data in the relational database and the information in the redo log, the changed data can be restored, so that the acquisition of logical data can be completed.

[0048] In other words, the present specification provides a method and device for obtaining logical data, which is applied to a read-only node of a cloud database system, including: obtaining the at least one redo log when the read-only node plays back the at least one redo log; parsing the at least one redo log to obtain data changed by a data change instruction corresponding to the at least one redo log; obtaining index information from the at least one redo log; and combining the index information and the data changed by the data change instruction into logical data corresponding to the at least one redo log.

[0049] In this way, through the playback process of the read-only node, logical data is extracted from the redo log to complete the acquisition of logical data. The read-write node does not need to save duplicate logs, which reduces the IO cost of the read-write node and achieves an effect similar to that of opening the binary log without opening the binary log. In addition, the above method is completed during the playback process, that is, the redo log obtained by the playback process of the redo log of the read-only node itself is reused, so that logical data can be obtained through the redo log and can be played back, and logical data can be extracted at a lower cost. In addition, there is no need for the read-write node to record additional binary logs, which saves storage space and improves the data writing speed.

[0050] Furthermore, on this basis, external modules can access the database through read-only nodes and subscribe to logical data obtained during playback on read-only nodes, so that external modules can perceive more readable logical data, thereby realizing the perception of database modifications.

[0051] Next, a logic data acquisition method shown in this specification will be described in detail.

[0052] like Figure 1 As shown, Figure 1 This is a flowchart of a method for obtaining logical data according to an exemplary embodiment of the present specification, comprising the following steps:

[0053] Step 101: When a read-only node plays back at least one redo log, obtain the at least one redo log.

[0054] It should also be noted that this method is applied to read-only nodes of the cloud database system.

[0055] In step 101, the redo logs obtained in the process of replaying data by the read-only node are used, and these redo logs are used as the basis for obtaining logical data.

[0056] Next, the meaning of each term in step 101 will be described in detail.

[0057] First, we need to explain the format of the redo log. Redo logs are generally divided into compact type and redundant type. The contents recorded in the two types are basically the same, but they differ in length. The compact type is a newer type and its length is shorter.

[0058] Although the redo log is a physical log, it also has logical significance. Generally, a redo log file includes multiple redo logs. The format of each redo log is as follows: generally including the log mark (SingleRecord Flag), log type (Log Type), tablespace ID (SpaceID), page number in the tablespace (Page Number) and log record data (Payload).

[0059] The log type varies according to the instruction, for example, it can include insert log, delete log and update log, etc. Different types of logs can be stored in the same redo log file. The data recorded in the log varies according to the log type. The specific differences are described below and will not be repeated here.

[0060] The cloud database system supports one write and multiple reads, including one read-write node and multiple read-only nodes. In order to synchronize the changes to physical pages in the read-write nodes, the read-only nodes will obtain and replay the redo logs to obtain the latest version of the database. The specific description of these processes is as above and will not be repeated here.

[0061] Related Technology The process of replaying a read-only node generally includes three stages. The first stage is the read stage, in which the log reading thread (Redo Log Async Reader, of course, it can also be other threads, and the log reading thread is used as an example here) in the read-only node reads the redo log from the shared storage space to the memory buffer (AsyncRead Buffer) of the read-only node.

[0062] The first stage can be executed when the latest version of the data to be read is not persisted, or it can be executed by each read-only node when it finds that the redo log of the read-write node has been newly added. For the latter, the read-only node and the read-write node can synchronize the log sequence number (LSN) through a communication thread. The LSN is used to represent the location where the latest redo log is recorded. When the LSN of the read-write node increases, it indicates that the read-write node has newly written a redo log. At this time, the read-only node can replay the newly added redo log to achieve data synchronization between the read-write node and the read-only node.

[0063] The second stage is the parsing stage. The coordination thread (Apply Coordinator) parses each redo log. Each physical page corresponds to a log application thread (Redo Log Apply Worker). The coordination thread will assign each redo log to the corresponding log application thread according to the tablespace ID and page number included in each redo log.

[0064] The parsing here is different from the parsing of the redo log in step 103 below. The parsing here refers to parsing the physical page corresponding to each redo log, so as to allocate it to the corresponding log application thread. In addition, the purpose of setting multiple log application threads is to improve the efficiency of log application. If only one log application thread is used for application, the speed is slow.

[0065] The third stage is the Apply stage, that is, after all the redo logs obtained in the first stage are parsed, the coordination thread notifies all log application threads to apply each redo log in parallel and obtain the latest version of the memory page in memory.

[0066] After explaining each term involved in step 101, step 101 will be explained from different aspects.

[0067] As mentioned above, playback includes three stages. Of course, redo logs can be obtained in any of the three stages. However, considering that there are default problems in redo logs in some cases, it will save more resources to obtain redo logs in the third stage, the application stage.

[0068] Specifically, if the content written in the previous redo log and the next redo log is the same, the next redo log will omit part of the content. For example, if the previous redo log writes the string AB at a certain position on a physical page, and the next redo log writes the string ABCD at a certain position on another physical page, since AB and ABCD have the same "AB", when recording the content of the next redo log, CD will be recorded directly, and ABCD will not be recorded.

[0069] In this case, in the parsing phase, that is, before the log application thread actually applies the log, the coordination thread will complete the log through the Mismatch information. Therefore, the completed redo log can be obtained in the third phase. In other phases, after obtaining the redo log, it is necessary to repeatedly complete the redo log. It can be seen that obtaining the redo log in the third phase will improve efficiency and reduce duplication of work.

[0070] In the parsing stage, to obtain mismatch information, the physical page must be obtained first, and the physical page is generally obtained in the application stage. If the redo log is obtained in the parsing stage, it is still necessary to read the physical page in the shared storage space first (when the physical page is not in the memory), which is also a duplication of work. If the redo log is obtained in the third stage, it can also reduce duplication of work from this aspect.

[0071] In other words, step 101 includes: after the read-only node plays back at least one redo log and completes the at least one redo log, obtaining the at least one redo log.

[0072] That is to say, obtaining the complete redo log in the application phase of playback can reduce duplication of work and improve efficiency. Specifically, obtaining in the playback phase can be done before or after the application phase begins, and the specific application timing is not limited here.

[0073] After explaining step 101 from the execution timing of step 101 , step 101 will be further explained from the type of redo log.

[0074] There are redo logs for data change instructions (including data insertion, deletion, and update instructions), as well as redo logs for transaction operations. The commit, rollback, and savepoint operations of transactions also generate corresponding logs. The processing methods for different types of logs are different.

[0075] For insert and update type data change instructions, there are generally only two types of redo logs: compact and redundant. The contents of the two logs are basically the same, and the timing of generation is also the same. Both can be used to extract logical data. The processing method for these two logs is to extract the required data from the corresponding position according to the type of redo log.

[0076] For the delete type data change instruction in the data change instruction, there are four types of redo logs, namely, the delete mark type compact format, redundant format redo log, delete recovery type compact format, redundant format redo log. For the delete mark type and delete recovery type, unlike insertion and update, deletion is not deleted immediately after receiving the data change instruction. Instead, when receiving the data change instruction, the clustered index (i.e. primary key index) corresponding to the data to be deleted is marked for deletion first, and the delete mark type redo log is recorded at this time. After a period of time, when deletion is possible, the data corresponding to the clustered index with the delete mark will be deleted to complete the recovery of the space that needs to be deleted.

[0077] In the above case, when the clustered index is marked with a deletion mark, the data is considered deleted. Originally, it would be sufficient to record only the deletion mark type of redo log, but when the data is actually deleted, that is, when the corresponding space is reclaimed, the redo log should also be recorded, otherwise the data cannot be restored through the redo log, so it is necessary to record two types of redo logs. For the two types of redo logs, the deletion mark type of redo log is recorded when the data is considered deleted. This redo log can better reflect the true status of the data, while the deletion recovery type of log will be delayed in comparison.

[0078] Therefore, in order to improve the real-time and accuracy of logical data acquisition, it is selected to parse logical data only through logs of the delete mark type. In other words, step 101 includes: when the type of the redo log played back by the read-only node is the delete mark type, at least one redo log of the delete mark type is obtained; the delete mark type is used to indicate that the redo log is generated when the cluster index is marked with a delete mark. When the type of the redo log played back by the read-only node is the delete recovery type, the redo log of the delete recovery type is not obtained; the delete recovery type is used to indicate that the redo log is generated when the data corresponding to the cluster index with a delete mark is deleted.

[0079] How to process the logs of the transaction operation type will be explained below, and will not be described here. It should be noted that the logs of the transaction operation type still need to be obtained.

[0080] Step 103: parse the at least one redo log and obtain data changed by the data change instruction corresponding to the at least one redo log.

[0081] Step 105: Obtain index information from the at least one redo log.

[0082] Next, step 103 and step 105 will be described collectively.

[0083] The process of steps 103 and 105 is the process of extracting logical data from the physical redo log. First of all, it should be noted that logical data includes the location of the data (and further may include the meaning of the location) and how the data is changed, so both need to be extracted separately.

[0084] It should be noted that the order of steps 103 and 105 is not limited in this specification.

[0085] First, the specific implementation method of step 103 is described.

[0086] For delete and insert redo logs, since the smallest unit of relational database operations is a row, insert and delete often insert or delete a whole row. In this case, the changed data to be extracted by delete and insert is a whole row of data, and a row includes multiple columns, so it is necessary to first restore the data of each column of this row, and then combine the data of each column into the data of the whole row.

[0087] For update type redo logs, sometimes the update only changes one or several columns in a row. Obtaining the changed data can be obtaining data of an entire row or obtaining data of one or several columns in a row (ie, the changed data).

[0088] As for the specific recovery method, the redo log generally includes the length of each column field, non-empty information, etc., and then the data recorded in the redo log can be split into multiple blocks through these two records, and the empty columns in the middle of multiple blocks of data can be filled with non-empty information, so as to obtain a whole row of data through the data of each column.

[0089] It should be noted that the above only shows a method of extracting changed data by parsing redo logs, and the specific implementation method is not limited to this method, and other methods may also be used, and this specification does not limit this.

[0090] Next, the specific implementation method of step 105 is described.

[0091] For the redo log corresponding to the data change instruction, the redo log will contain some logical information, such as tablespace ID and page number, etc. If the maintenance personnel only need logical data at the tablespace ID level, then the tablespace ID can be obtained only by parsing the redo log.

[0092] In some cases, the maintenance personnel actually only know the tablespace ID and do not know which specific table the ID represents. In some cases, the maintenance personnel also want to know which specific row the modification is made to. In this case, they need to know the records in the redo log and physical page.

[0093] Specifically, the tablespace ID and page number contained in the redo log can be determined through the redo log, thereby determining the physical page where the changed data is located, and then extracting the index identifier (Index ID) of the row of data (that is, the row where the changed data is located) from the index header (Index Header) of the physical page, and then determining the index (Index) object and table (table) object of the corresponding row from the index dictionary, thereby determining the primary key index and secondary index of the changed data, and obtaining complete index information of the changed data.

[0094] In other words, step 105 includes: obtaining the physical page where the data changed by the data change instruction is located from the at least one redo log; determining the index identifier corresponding to the data changed by the data change instruction from the index header of the physical page, and determining the index information of the data changed by the data change instruction based on the index identifier; the index information includes the primary key index and secondary index of the data changed by the data change instruction.

[0095] The captured row data is captured from the primary key index, which contains complete row data, while the secondary index only contains certain columns of data.

[0096] Step 107: Combining the index information and the data changed by the data change instruction into logical data corresponding to the at least one redo log.

[0097] Through steps 103 and 105, the changed data is parsed from the redo log to find out the specific table and location (index information, representing the location in the table), and how to change the data is determined. When both are obtained, they can be used as logical data corresponding to the redo log.

[0098] In addition, it should be noted that the redo logs mentioned above also have logs for transaction operations. Specifically, there are redo logs corresponding to transaction commits, redo logs corresponding to transaction rollbacks, and redo logs corresponding to transactions that set save points. For these three, since they are not operations that change the database, there is no need to analyze the logical data in these redo logs. However, these redo logs indicate which of the current redo logs have been executed and which have not yet issued transaction commit instructions (it is possible that the transaction has not yet been committed, and the corresponding data change instructions have been written to the redo logs), so these redo logs still need to be analyzed to better analyze the redo logs for data change instructions.

[0099] For the redo log corresponding to the transaction commit operation, it generally includes the transaction number, and other redo logs for data change instructions also include the transaction number. Therefore, only the redo log corresponding to the transaction number that has recorded the commit operation is the redo log that needs to be analyzed. Other redo logs are redo logs that have not been committed. These logs do not need to be analyzed now, and will be analyzed after the corresponding transactions are committed.

[0100] In other words, the method further includes: in the case where the at least one redo log obtained includes a redo log corresponding to a transaction commit instruction, determining a commit transaction number included in the redo log corresponding to the transaction commit instruction. In the at least one redo log, determining a change transaction number included in the redo log corresponding to the data change instruction. According to the commit transaction number and the change transaction number, determining a valid redo log from the at least one redo log, and determining logical data based on the valid redo log.

[0101] Among them, the valid redo logs are redo logs with the same change transaction number and any submitted transaction number, and the logical data needs to be determined based on these redo logs.

[0102] For the rollback operation of a transaction, the rollback operation does not directly record the redo log. Instead, the compensation operation is performed by reversely applying the rollback log (Undo Log), and the compensation operation will also record the corresponding redo log. Therefore, the redo log corresponding to the rollback operation of the transaction can be analyzed based on the above method (regarded as the redo log for the data change instruction).

[0103] For savepoint operations, redo logs are generated when adding savepoints. Such redo logs do not contain logical meaning and need not be analyzed. When rolling back to a savepoint, similar to the rollback operation, compensation operations are also performed by reflecting the application rollback logs. Compensation operations generate corresponding redo logs, which can be processed in the same way as the redo logs for data change instructions.

[0104] In addition, it should be noted that in order to allow maintenance personnel to see database modifications, they can also subscribe to the logical data generated on each read-only node through a subscription module, so that maintenance personnel can know the logical data corresponding to the modification of the physical page.

[0105] In other words, after the logical data is composed, the method further includes: sending the composed logical data to the subscription module. The logical data sent to the subscription module may be the logical data corresponding to all redo logs, or may be all or part of the logical data corresponding to part of the redo logs. The specific data sent is the result of the redo log acquisition range of the read-only node, the range of parsed logical data, and the modification range subscribed by the subscription module.

[0106] Since in the related technology, if the binary log is not enabled, the read-only node cannot perceive the logical data, the above method provides the read-only node with the ability to perceive the logical data, and through the subscription module, the maintenance personnel can obtain the logical data from the read-only node, thereby realizing the perception of database changes.

[0107] The above method breaks the original scheme that logical data can only be obtained through read-write nodes, so that logical data can be obtained through read-only nodes, and read-write nodes do not need to record binary logs, reducing the read-write pressure of read-write nodes. And the scheme is executed while the read-only node is playing back, so that the scheme is implemented at the lowest cost.

[0108] Corresponding to the embodiments of the aforementioned method, this specification also provides embodiments of a device and a terminal to which it is applied.

[0109] like Figure 2 As shown, Figure 2 This is a block diagram of a logical data acquisition device according to an exemplary embodiment of the present specification, which is applied to a read-only node of a cloud database system. The device includes:

[0110] The redo log acquisition module 210 is used to acquire at least one redo log when the read-only node plays back at least one redo log.

[0111] The changed data acquisition module 220 is used to parse the at least one redo log and acquire the data changed by the data change instruction corresponding to the at least one redo log.

[0112] The index information acquisition module 230 is used to acquire index information from the at least one redo log.

[0113] The logical data acquisition module 240 is used to combine the index information and the data changed by the data change instruction into the logical data corresponding to the at least one redo log.

[0114] In an optional implementation, the redo log acquisition module 210 is specifically configured to: acquire the at least one redo log after the read-only node plays back the at least one redo log and completes the at least one redo log.

[0115] In an optional embodiment, the redo log acquisition module 210 is specifically used for: in the case where the redo log type played back by the read-only node is a deletion mark type, obtaining at least one redo log of the deletion mark type; the deletion mark type is used to indicate that the redo log is generated when the clustered index is marked with a deletion mark; in the case where the redo log type played back by the read-only node is a deletion recovery type, not obtaining a redo log of the deletion recovery type; the deletion recovery type is used to indicate that the redo log is generated when the data corresponding to the clustered index with the deletion mark is deleted. In an optional embodiment, the index information acquisition module 230 is specifically used for: obtaining the physical page where the data changed by the data change instruction is located from the at least one redo log; determining the index identifier corresponding to the data changed by the data change instruction from the index header of the physical page, and determining the index information of the data changed by the data change instruction according to the index identifier; the index information includes the primary key index and secondary index of the data changed by the data change instruction.

[0116] In an optional embodiment, the device also includes: a valid redo log determination module 240 (not shown in the figure), which is used to: in a case where the at least one redo log obtained includes a redo log corresponding to a transaction commit instruction, determine the commit transaction number included in the redo log corresponding to the transaction commit instruction; in the at least one redo log, determine the change transaction number included in the redo log corresponding to the data change instruction; determine the valid redo log from the at least one redo log according to the commit transaction number and the change transaction number, and determine the logical data based on the valid redo log.

[0117] In an optional implementation, the device further includes: a logic data sending module 250 (not shown in the figure), which is used to send the composed logic data to the subscription module after composing the logic data.

[0118] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0119] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0120] like Figure 3 As shown, Figure 3 A hardware structure diagram of a computer device in which the logic data acquisition device of the embodiment is located is shown, and the device may include: a processor 1010, a memory 1020 for storing processor executable instructions, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0121] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits. The processor implements the aforementioned logic data acquisition method by running the executable instructions.

[0122] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0123] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0124] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0125] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0126] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0127] The embodiments of the present specification also provide a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the aforementioned logic data acquisition method is implemented.

[0128] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0129] In addition, the present specification also provides a computer program, which implements the aforementioned logic data acquisition method when executed.

[0130] In addition, this specification also provides a cloud database system, which includes a read-write node and a plurality of read-only nodes, and any read-only node executes the aforementioned logical data acquisition method.

[0131] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0132] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A logical data acquisition method, applied to a read-only node of a cloud database system, the method comprising: In the case where the read-only node plays back at least one redo log, obtaining the at least one redo log; Parsing the at least one redo log to obtain data changed by the data change instruction corresponding to the at least one redo log; Obtaining index information from the at least one redo log; the index information indicates the table where the changed data is located and its position in the table; The index information and the data changed by the data change instruction are combined into logical data corresponding to the at least one redo log.

2. The method according to claim 1, wherein when the read-only node plays back at least one redo log, obtaining the at least one redo log comprises: After the read-only node plays back at least one redo log and completes the at least one redo log, the at least one redo log is obtained.

3. The method according to claim 1, wherein when the read-only node plays back at least one redo log, obtaining the at least one redo log comprises: When the type of the redo log played back by the read-only node is a delete mark type, obtaining at least one redo log of the delete mark type; The delete mark type is used to indicate that the redo log is generated when a clustered index is marked with a delete mark; In the case where the redo log type played back by the read-only node is a delete recovery type, a redo log of the delete recovery type is not obtained; the delete recovery type is used to indicate that the redo log is generated when data corresponding to a clustered index with a delete mark is deleted.

4. The method according to claim 1, wherein obtaining index information from the at least one redo log comprises: Obtaining, from the at least one redo log, a physical page where the data changed by the data change instruction is located; Determine the index identifier corresponding to the data changed by the data change instruction from the index header of the physical page, and determine the index information of the data changed by the data change instruction based on the index identifier; the index information includes the primary key index and secondary index of the data changed by the data change instruction.

5. The method according to claim 1, further comprising: In a case where the at least one redo log obtained includes a redo log corresponding to a transaction commit instruction, determining a commit transaction number included in the redo log corresponding to the transaction commit instruction; In the at least one redo log, determining a change transaction number included in the redo log corresponding to the data change instruction; According to the commit transaction number and the change transaction number, a valid redo log is determined from the at least one redo log, and logical data is determined based on the valid redo log.

6. The method according to any one of claims 1 to 5, after forming the logic data, the method further comprises: The composed logical data is sent to the subscription module.

7. A logical data acquisition device, applied to a read-only node of a cloud database system, comprising: A redo log acquisition module, used for acquiring at least one redo log when the read-only node plays back at least one redo log; A changed data acquisition module, used for parsing the at least one redo log and acquiring data changed by a data change instruction corresponding to the at least one redo log; An index information acquisition module, used to acquire index information from the at least one redo log; the index information indicates the table where the changed data is located and its position in the table; The logic data acquisition module is used to form the logic data corresponding to the at least one redo log by combining the index information and the data changed by the data change instruction.

8. A cloud database system, comprising a read-write node and a plurality of read-only nodes, wherein any read-only node executes the logical data acquisition method according to any one of claims 1 to 6.

9. A computer device, comprising: processor; a memory for storing processor-executable instructions; The processor implements the logic data acquisition method according to any one of claims 1 to 6 by running the executable instructions.

10. A computer-readable storage medium, wherein computer instructions are stored on the computer-readable storage medium, and when the computer instructions are executed by a processor, the logic data acquisition method according to any one of claims 1 to 6 is implemented.

11. A computer program product, wherein when the computer program product is executed, the method for acquiring logical data according to any one of claims 1 to 6 is implemented.

Images