Data management method and database system

Abstract

Disclosed are a data management method and a database system, relating to the technical field of data storage. Compared with situations in which when logs are generated on master nodes, slave nodes directly replay the logs generated on the master nodes to synchronize data in secondary caches of the slave nodes, in the present application, when a data page in a secondary cache of a slave node is requested to be accessed, the slave node reads, from a log buffer, in response to an access request, logs of said data page for replay, so as to obtain a latest version of said data page. The method can reduce the log replay load of the slave node when data update operations of the master node are frequent, thereby guaranteeing the performance of the slave node.

Description

Data management method and database system

[0001] The present application claims priority from the Chinese patent application No. 202411314742.1 filed on September 19, 2024, and entitled "Data management method and database system", the whole content of the above-mentioned application is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of data storage, in particular to a data management method, a database system, a cluster, a storage medium and a program product. BACKGROUND

[0003] A database system of storage-computation separation generally includes a computing node and a shared storage cluster, and data transmission is performed between the computing node and the shared storage cluster through a network. In order to reduce the time delay caused by data transmission across the network, the database system based on shared storage sets a secondary cache in addition to the memory for the computing node, wherein the memory is a primary cache, which is usually used to store hot data, and the secondary cache is usually used to store warm data with lower access frequency than the hot data. In this way, when the data to be read is not hit in the memory of the computing node, the computing node can further read the data in the secondary cache, thereby accelerating the reading efficiency of the data.

[0004] In order to ensure the high availability of the database system, the computing node usually includes a master node and a slave node. The master node is used to process read and write operations on data, and the slave node can also undertake part of the read operations on data. In order to realize the data synchronization of the master node and the slave node, when the data in the secondary cache of the slave node is updated by the master node, the slave node needs to synchronize the data in the secondary cache of the slave node by replaying the logs generated by the master node. However, when the data update operation of the master node is frequent, the log replay pressure of the slave node will also become large, which affects the performance of the slave node. SUMMARY

[0005] The present application provides a data management method and a database system, which are used to solve the problem that frequent updating of data in the secondary cache of the slave node affects the performance of the slave node.

[0006] In a first aspect, the present application provides a data management method. The method is applied to a database system, the database system comprising a master node, a slave node and a shared storage cluster. The master node and the slave node are both computing nodes deployed with a database instance, and the shared storage cluster comprises at least one storage node. The slave node comprises a memory and a second-level cache. The method comprises: after the master node sends a log of a data page to the shared storage cluster for storage, the slave node acquires the log of the data page from the shared storage cluster, and stores the log of the data page to a log cache area of the slave node. The log of the data page is used to record a modification operation of the data page. When the slave node receives an access request related to the data page, in a case that the data page has been cached to the second-level cache of the slave node, the slave node responds to the access request, reads the log of the data page from the log cache area of the slave node, and replays the log of the data page to obtain an updated data page, and the slave node feeds back the access request according to the updated data page.

[0007] Compared with the case that the log is generated on the master node, the slave node directly replays the log generated by the master node to synchronize the data in the second-level cache of the slave node. In the first aspect of the present application, when the data page in the second-level cache of the slave node is requested to be accessed, the slave node responds to the access request, reads the log of the data page requested to be accessed from the log cache area, and replays the log of the data page, so as to reduce the log replay pressure of the slave node in the case that the data update operation on the master node is frequent, and provide a guarantee for the performance of the slave node.

[0008] In a possible implementation, after the slave node responds to the access request, reads the log of the data page from the log cache area, and replays the log of the data page to obtain an updated data page, the method further comprises: the slave node updates the data page in the second-level cache of the slave node. In the case that the slave node replays the log of the data page to obtain an updated data page, the slave node updates the data page in the second-level cache of the slave node. In this way, when the access request for the data page is received again, the updated data page can be directly obtained to feed back the request, and the feedback efficiency of the access request is improved.

[0009] In a possible implementation, the method further comprises: when a log elimination condition is met, the slave node deletes part of the logs from the log cache area. The log elimination condition is any one of the following conditions: the capacity of the logs stored in the log cache area reaches a preset capacity threshold; the remaining space of the log cache area reaches a preset space threshold; or the time interval between two adjacent elimination behaviors reaches a preset elimination period. In this way, the slave node timely cleans up part of the logs in the log cache area, so that the log cache area can provide sufficient storage space to store new logs.

[0010] In a possible implementation, the method further includes: before the slave node deletes the part of logs from the log cache area, if the data page corresponding to the part of logs has been cached to the secondary cache of the slave node, the slave node first updates the data page corresponding to the part of logs in the secondary cache of the slave node according to the part of logs. In this way, the data page in the secondary cache of the slave node is ensured to be updated in time.

[0011] In a possible implementation, the slave node deleting the part of logs from the log cache area includes: the slave node deleting, from the log cache area, the part of logs whose entering order into the log cache area is earlier than that of other logs in the log cache area. In this way, the slave node can clean up the long-stored logs in time, and improve the resource utilization of the storage resource of the log cache area.

[0012] In another possible implementation, the part of logs deleted from the log cache area by the slave node includes a first log and a second log, where the first log is the earliest log entering the log cache area, and the second log includes all other logs of the data page corresponding to the first log in the log cache area. In this implementation, the logs of the same data page can be deleted at the same time, thereby helping to further update the data page to a new version according to all the logs of the same data page at one time.

[0013] In a possible implementation, the method further includes: the master node performing a modification operation on the data page; the master node generating a log of the data page, the log of the data page being used to record the modification operation on the data page; and the master node sending the log of the data page to the shared storage cluster. In this way, the slave node is ensured to be able to obtain the log from the shared storage cluster and replay the log to obtain the updated data page.

[0014] In a second aspect, the present application provides a database system. The database system includes a master node, a slave node and a shared storage cluster. The master node and the slave node are both computing nodes deployed with a database instance. The shared storage cluster includes at least one storage node, and the slave node includes an internal memory and a secondary cache. After the master node sends the log of the data page to the shared storage cluster, the slave node is configured to: obtain the log of the data page from the shared storage cluster, and store the log of the data page to a log cache area of the slave node. The log of the data page is used to record the modification operation on the data page. When the slave node receives an access request related to the data page, in the case that the data page has been cached to the secondary cache of the slave node, the slave node is further configured to: in response to the access request, read the log of the data page from the log cache area of the slave node, and replay the log of the data page to obtain an updated data page. The slave node is further configured to: feed back the access request according to the updated data page.

[0015] In a possible implementation, the slave node is further configured to update the data page in the slave node secondary cache.

[0016] In another possible implementation, the slave node is further configured to delete the part of logs from the log cache area when the log eviction condition is met.

[0017] In another possible implementation, the slave node is specifically configured to delete the part of logs from the log cache area according to the order in which the logs enter the log cache area, wherein the order in which the part of logs enter the log cache area is earlier than that of other logs in the log cache area.

[0018] In another possible implementation, before the slave node deletes the part of logs from the log cache area, if the data page corresponding to the part of logs to be deleted has been cached in the secondary cache of the slave node, the slave node is further configured to update the data page corresponding to the part of logs in the secondary cache of the slave node according to the part of logs.

[0019] In another possible implementation, the part of logs includes a first log and a second log, wherein the first log is the earliest log that enters the log cache area, and the second log includes all other logs of the data page corresponding to the first log in the log cache area.

[0020] In another possible implementation, the master node is configured to perform a modification operation on a data page, generate a log of the data page, and send the log of the data page to the shared storage cluster.

[0021] In a third aspect, the present application provides a computing device cluster. The computing device cluster includes at least one computing device, and the computing device includes a memory and a processor. The memory is configured to store computer instructions, and the processor is configured to execute the computer instructions to implement the method in the first aspect or any possible implementation manner of the first aspect.

[0022] In a fourth aspect, the present application provides a computer readable storage medium. The storage medium stores computer programs or instructions, and when the computer programs or instructions are executed by a processing device, the method in the first aspect or any possible implementation manner of the first aspect is implemented.

[0023] In a fifth aspect, the present application provides a computer program product. The computer program product includes computer programs or instructions, and when the computer programs or instructions are executed by a processing device, the method in the first aspect or any possible implementation manner of the first aspect is implemented.

[0024] The beneficial effects of the second aspect to the fifth aspect above can refer to the description of the first aspect or any one of the implementation manners of the first aspect, and will not be repeated here. On the basis of the implementation manners provided in the above aspects, the application can be further combined to provide more implementation manners. BRIEF DESCRIPTION OF DRAWINGS

[0025] FIG. 1 is a structural schematic diagram of a database system provided by the application;

[0026] FIG. 2A is a structural schematic diagram of a computing device provided by the application;

[0027] FIG. 2B is a structural schematic diagram of another computing device provided by the application;

[0028] FIG. 3 is a flow schematic diagram of a data management method provided by the application;

[0029] FIG. 4 is an example diagram of a first data management method provided by the application;

[0030] FIG. 5 is an example diagram of a second data management method provided by the application;

[0031] FIG. 6 is a structural schematic diagram of a computing device cluster provided by the application. DETAILED DESCRIPTION

[0032] At present, in a storage-computing separation database, in order to realize data synchronization of the master node and the slave node, when the data in the secondary cache of the slave node is updated by the master node, the slave node needs to synchronize the data in the secondary cache of the slave node by replaying the log generated by the master node. However, when the data update operation of the master node is frequent, the log replay pressure of the slave node will also become large, affecting the performance of the slave node.

[0033] Specifically, in the prior art, the master node records the modification operation of the data page by using a log. In order to realize data synchronization of the slave node and the master node, when the master node stores the generated log into a shared storage cluster, the slave node will obtain the above log from the shared storage cluster to perform replay, so as to update the data page in the memory and the secondary cache of the slave node. In this prior art, all data page modification operations of the master node will be replayed in the slave node. As long as the master node modifies the data page, the slave node needs to flush the data page in the secondary cache of the slave node once. In the case that the data page modification operation of the master node is frequent, this will cause a large number of data writes in the secondary cache of the slave node, affecting the service life of the secondary cache of the slave node.

[0034] Based on this, the application provides a data management method. The method can be applied to a database system including a master node, a slave node and a shared storage cluster, wherein the master node and the slave node are both computing nodes deployed with a database instance, and the shared storage cluster includes at least one storage node, and the slave node includes an internal memory and a secondary cache. In the method, a log cache area is established in the slave node to cache logs of data pages generated by the master node, and when a data page in the secondary cache of the slave node is requested to be accessed, the slave node reads the log of the data page requested to be accessed from the log cache area for playback in response to the access request, so as to obtain the latest version of the data page requested to be accessed. Compared with the prior art, the data management method provided by the application can reduce the number of times of playback of the log of the data page requested to be accessed on the slave node, reduce the write amount of the secondary cache of the slave node, and provide a guarantee for the performance of the slave node.

[0035] The data management method provided by the application can be applied to the database system shown in FIG. 1, which is a structural schematic diagram of a database system provided by the application. As shown in FIG. 1, the database system 100 includes a computing node 110 and a shared storage cluster 120. The computing node 110 and the shared storage cluster 120 can transmit data through a network 130. Exemplarily, the network 130 can be a communication network based on a remote direct memory access (RDMA) technology. The network 130 described above can be a wired communication or a wireless communication. The wired communication can be an Ethernet, an optical fiber, and various peripheral component interconnect express (PCIe) buses arranged in the data migration system for connecting the computing node 110 and the shared storage cluster 120, etc. The wireless communication can be an Internet, a wireless fidelity (WIFI), an ultra wide band (UWB) technology, etc.

[0036] The computing node 110 can include a master node 111 and a slave node 112. The master node 111 is configured to process read and write operations on data, and the slave node 112 can process partial read operations on data. The master node 111 and the slave node 112 can be internally provided with a memory and a second-level cache. The memory is a first-level cache and is configured to store hot data. The second-level cache is configured to store warm data with a lower access frequency than the hot data. In this way, when the data to be read is not hit in the memory of the computing node 110, the computing node 110 can further read the data from the second-level cache, thereby accelerating the read efficiency of the data and reducing the latency caused by the transmission of the data across the network 130. The second-level cache can be a solid state disk (SSD), but should not be construed as a limitation to the present application, for example, the second-level cache can also be a hard disk drive (HDD) and the like. In some possible cases, the second-level cache can also be referred to as an extended buffer pool (EBP). The master node 111 and the slave node 112 can each be a computing device on which a database instance is deployed. The computing device can include, but is not limited to, a server, a virtual machine (VM), an elastic cloud server (ECS), a container, and the like. For example, the master node 111 and the slave node 112 can each be a server on which a database instance is deployed. In some possible cases, the master node 111 can also be referred to as a master device, and the slave node 112 can also be referred to as a slave device or a read-only node, which are not limited in the present application.

[0037] The shared storage cluster 120 can include at least one storage node 121. The at least one storage node 121 can communicate in a wired manner or in a wireless manner. For related descriptions of the communication manner, refer to the related description in the foregoing, which will not be described here. The storage node 121 can include, but is not limited to, a server, a virtual machine (VM), an elastic cloud server (ECS), a container, and the like. For example, the storage node 121 can be a server. It should be noted that the storage nodes 121 in the shared storage cluster 120 can have the same hardware configuration or different hardware configurations, which are not limited in the present application.

[0038] In some possible scenarios, the database system 100 can further include client devices 140 (e.g., client device 1, client device 2, client device 3). A user can use the client device 140 to initiate a request for reading or writing data to the database system 100, the database system 100 receives and processes the request, and feeds back to the user through the client device 140. The client device 140 is a terminal device, including but not limited to a personal computer, a server, a mobile phone, a tablet computer, and the like.

[0039] The database system provided by the present application is described above in combination with FIG. 1, and a computing device provided by the present application is described below in combination with the accompanying drawings, which can be used to implement the computing node and the storage node in FIG. 1.

[0040] FIG. 2A is a structural schematic diagram of a computing device provided by the present application. As shown in FIG. 2A, the computing device 200 includes a communication interface 214, a processor 211, and a memory 212. The communication interface 214 is configured to communicate with devices located outside the computing device 200. For example, a user inputs an access request for a data page to the computing device 200 through the communication interface 214 to obtain data in the data page, and the computing device 200 obtains a processing result (e.g., the data page) based on the received access request. The communication interface 214 can be an input / output (I / O) interface. In some possible examples, the user can also use the client device 140 to input the access request for the data page to the computing device 200.

[0041] The processor 211 is the operation core and control core of the computing device 200, and can include a central processing unit (CPU), a specific integrated circuit, other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. In actual applications, the computing device 200 can also include multiple processors. The processor 211 can include one or more processor cores. An operating system and other software programs are installed in the processor 211, so that the processor 211 can access the memory 212 and various peripheral component interconnect express (PCIe) devices.

[0042] The processor 211 is connected to the memory 212 through a bus 215. The bus 215 can be a double data rate (DDR) bus or other type of bus. The memory 212 is the main memory of the computing device 200. The memory 212 is typically used to store various running software in the operating system and the like. In order to improve the access speed of the processor 211, the memory 212 needs to have the advantage of fast access speed. In a conventional computer device, a dynamic random access memory (DRAM) is usually used as the memory 212. In addition to the DRAM, the memory 212 can also be other random access memories, such as a static random access memory (SRAM) and the like. In addition, the memory 212 can also be a read only memory (ROM). For the read only memory, for example, it can be a programmable read only memory (PROM), an erasable programmable read only memory (EPROM) and the like. The embodiment does not limit the number and type of the memory 212.

[0043] Optionally, in order to store the data persistently, the computing device 200 is also provided with a data storage system 213. The data storage system 213 can be located outside the computing device 200 (as shown in FIG. 2A) and exchange data with the computing device 200 through a network. Optionally, the data storage system 213 can also be located inside the host, such as the data storage system 213 exchanges data with the processor 211 through the bus 215. At this time, the data storage system 213 can be a hard disk.

[0044] In order to reduce the time delay caused by the transmission of data across the network, a secondary cache in addition to the memory can also be provided in the computing device. The cache is used to store warm data to shorten the time delay of hitting data. FIG. 2B is a structural schematic diagram of another computing device provided by the present application. As shown in FIG. 2B, the memory 212 of the computing device 200 includes the memory and the secondary cache. FIG. 2B is only an example provided by the embodiment of the present application. In some possible examples, the computing device 200 can also be provided with more levels of cache, which is not limited by the present application. According to the actual application needs, the foregoing secondary cache can be implemented by using the SSD or the HDD, which is not limited by the present application.

[0045] The database system provided by the present application is described above in combination with FIG. 1, FIG. 2A and FIG. 2B. The data management method provided by the present application is described in detail below in combination with the content shown in FIG. 1, FIG. 2A and FIG. 2B.

[0046] FIG. 3 is a flow diagram of a data management method provided by the present application, which can be applied to the database system described in FIG. 1, including the following steps S310-S380.

[0047] S310, the master node performs a modification operation on the data page.

[0048] The modification operation on the data page can include but is not limited to adding data (or inserting data) to the data page, deleting data, updating data. For example, the master node can execute an insert command to add data to the data page, execute a delete command to delete data in the data page, and execute an update command to update data in the data page.

[0049] It should be noted that in the embodiments of the present application, the data page is used to record data and is a storage form of data in the database system. Those skilled in the related art should understand that the implementation of the technical solution of the present application is not limited to the storage form of data. Data can be stored in any form other than data pages. In other words, the data page is the data itself.

[0050] The master node can execute the modification operation on the data page after obtaining the modification command of the data page. The master node can obtain the modification command of the data page in various ways. For example, the master node can obtain the modification command of the data page based on a user-initiated modification request of the data page. For another example, the master node can also obtain the modification command of the data page based on a deletion request of the log.

[0051] The modification operation performed by the master node on the data page can include one or more of the following: adding data (or inserting data) to the data page, deleting data, updating data, etc. For example, the modification operation performed by the master node on the data page includes deleting first data in the data page. For another example, the modification operation performed by the master node on the data page includes deleting first data in the data page and modifying second data in the data page to third data, etc.

[0052] S320, the master node generates a log of the data page.

[0053] The log of the data page is used to record a modification operation of the data page. The modification operation can include one or more of the following: adding data to the data page, deleting data, updating data. For example, the log of the data page is used to record deletion of first data in the data page. For another example, the log of the data page is used to record deletion of the first data in the data page and updating second data in the data page to third data.

[0054] The log of the data page can include an identification of the data page on which the modification operation is performed, a timestamp of performing the modification operation, and the modification operation performed. In some possible scenarios, the master node can further generate a log sequence number (LSN) for the log of the data page, and the log corresponding data page can be determined according to the log sequence number.

[0055] In some possible scenarios, the log of the data page can be a redo log of the data page, and the type of the log is not limited in the present application.

[0056] S330, the master node sends the log of the data page to the shared storage cluster.

[0057] Correspondingly, the shared storage cluster performs S330A. S330A includes: the shared storage cluster receives and stores the log of the data page sent by the master node.

[0058] S340, the slave node obtains the log of the data page from the shared storage cluster, and stores the log of the data page to the log cache area of the slave node.

[0059] After the master node sends the log of the data page to the shared storage cluster, the slave node can obtain the log of the data page from the shared storage cluster. For example, the slave node can send a request for obtaining the log of the data page to the shared storage cluster, and receive the log of the data page sent by the shared storage cluster based on the request. The slave node stores the log to the log cache area.

[0060] For example, the slave node can set aside a space in its memory as the log cache area, or set aside a space in its second-level cache as the log cache area, or separately set aside a storage space as the log cache area.

[0061] S350, the slave node receives an access request related to the data page.

[0062] Specifically, the access request related to the data page is a request for accessing the data page. It should be understood by those skilled in the art that any request involving access or operation on data in a database can be regarded as an access request related to the data page where the data is located.

[0063] In one possible implementation, the access request is a user initiated data access instruction. The data access instruction can be any Structured Query Language (SQL) statement supported by the database system to process data. For example, the user initiated data access instruction is a "select" instruction to query specified data in a table, and the table data is stored in the database system in the form of data pages. Upon receiving the "select" instruction, the slave node determines the data page storing the specified data according to the "select" instruction, and the data page storing the specified data is the data page to be accessed.

[0064] In another possible implementation, the access request can also be an instruction carrying an identifier of the data page, and the identifier of the data page includes but is not limited to a name, a number, and a code of the data page.

[0065] The access request is a user initiated data query instruction, and the access request can also be an identifier of the data page used to determine the data page.

[0066] S360, in a case where the data page has been cached to the secondary cache of the slave node, the slave node reads the log of the data page from the log cache area in response to the access request, and replays the log of the data page to obtain the updated data page.

[0067] The slave node obtains the data page to be accessed indicated by the access request from the memory of the slave node, and in a case where the data page to be accessed indicated by the access request is not hit in the memory of the slave node, the slave node further obtains the data page to be accessed from the secondary cache of the slave node. In a case where it is determined that the data page has been cached to the secondary cache of the slave node, the slave node reads all the logs of the data page from the log cache area in response to the access request, and replays all the logs of the data page to obtain the updated data page.

[0068] In a possible embodiment, the slave node determines the logs of a data page according to the identifiers of the data pages included in the logs. For example, the log cache area includes log Z1, log Z2, log Z3, log X1 and log X2, and log Y1. Each of the log Z1, log Z2 and log Z3 includes a page identifier Z indicating a data page z, each of the log X1 and log X2 includes a page identifier X indicating a data page x, and the log Y1 includes a page identifier Y indicating a data page y. In this case, the slave node can determine all the logs corresponding to a data page to be accessed according to the identifiers of the data pages included in each log in the log cache area. For example, when the slave node receives an access request related to the data page z, the slave node can determine, according to the page identifier Z, that the logs of the data page z to be accessed include Z1, Z2 and Z3, which are referred to as Z1 to Z3.

[0069] In the embodiments of the present application, the name of a log consists of a letter label and a number label, where logs with the same letter label correspond to the same data page. For logs with the same letter label, the number label in the name of the log represents the version of the log, and the larger the number label, the newer the version of the log. In other words, among logs with the same letter label, the log with the larger number label is generated later than the log with the smaller number label.

[0070] After the slave node determines all the logs of a data page from the log cache area, the slave node can replay all the logs of the data page to obtain an updated data page by using the following process. Specifically, after the slave node obtains a data page from the secondary cache and reads all the logs of the data page from the log cache area, the slave node can replay all the logs of the data page to obtain an updated data page.

[0071] For example, the slave node obtains the data page z from the secondary cache and obtains the data in the data page z. The slave node determines, by using the method described above, that all the logs of the data page z are the log Z1 to the log Z3 from the log cache area. The slave node replays the log Z1 to the log Z3 to obtain an updated data page z.

[0072] S370, the slave node feeds back the access request according to the updated data page.

[0073] The slave node can feed back the access request based on the specific content of the access request.

[0074] In a possible case, the access request is a "select" instruction of a user-initiated query table to specify data, and the slave node returns the specified data to the user after reading the specified data from the updated data page.

[0075] In a possible scenario, the access request is specifically for querying a processing result based on the specified data, and the slave node reads the specified data from the updated data page and processes the specified data, and then returns the processing result to the user.

[0076] S380, the slave node updates the data page in the slave node secondary cache.

[0077] In S360, the slave node obtains the updated data page, i.e., the new version of the data page, by replaying the log of the data page. In S380, the slave node updates the old version of the data page stored in the secondary cache to the new version of the data page.

[0078] Optionally, after updating the data page in the slave node secondary cache, the slave node can also delete the log of the data page that has been replayed in step S360. For example, after updating the data page y in the secondary cache to its new version based on the old version of the data page y in the secondary cache and the log Y1 of the data page y in the log cache, the slave node can delete the log Y1 in the log cache.

[0079] In this application, the slave node also periodically or non-periodically evicts logs in the log cache, so as to have free space in the log cache to cache new logs. Specifically, the slave node is configured with a preset log eviction condition, and when the log eviction condition is met, the slave node deletes part of the logs from the log cache. The operation of the slave node deleting part of the logs from the log cache is also referred to as the slave node performing an eviction behavior.

[0080] In a possible embodiment, the log eviction condition is that the capacity of the logs stored in the log cache reaches a preset capacity threshold. The preset capacity threshold can be the size of the log, or the percentage of the log capacity in the total capacity of the log cache. For example, the total storage capacity of the log cache is 1 megabyte (1 MB), and the preset capacity threshold is 0.8 MB. When the capacity of the logs stored in the log cache is equal to or greater than 0.8 MB, the slave node performs an eviction behavior, i.e., deletes part of the logs from the log cache.

[0081] In another possible embodiment, the log eviction condition is that the remaining space of the log cache reaches a preset space threshold. The preset space threshold can be the size of the remaining storage space, or the percentage of the remaining storage space in the total storage space of the log cache. For example, the total storage space of the log cache is 1 MB, and the preset space threshold is 0.2 MB. When the remaining space of the log cache is equal to or less than 0.2 MB, the slave node performs an eviction behavior, i.e., deletes part of the logs from the log cache.

[0082] In another possible embodiment, the log eviction condition is that a time interval between two adjacent log eviction operations reaches a preset eviction period. In this embodiment, the slave node performs a log eviction operation every time interval, and the time interval between two adjacent log eviction operations is also referred to as an eviction period. For example, if the preset eviction period is 1 day, the slave node performs a log eviction operation every day.

[0083] In this application, the slave node monitors the storage of logs in the log cache area, and triggers a log eviction operation when the log eviction condition is met. Specifically, when the slave node detects that the log eviction condition is met, the slave node first determines the part of logs to be deleted, and then deletes the part of logs.

[0084] In a possible embodiment, the slave node deletes, from the log cache area, the part of logs that enter the log cache area earlier than other logs in the log cache area according to the order in which the logs enter the log cache area. The number of the part of logs to be deleted can be any number or capacity size set by a person skilled in the art according to business needs. For example, in the case where the configuration of the slave node about the log eviction operation is to delete 20 logs each time, when the log eviction condition is met, the slave node determines, according to the order in which the logs enter the log cache area, the logs that are in the first 20 positions in the log cache area as the part of logs to be deleted, and deletes the first 20 logs.

[0085] In a possible embodiment, the slave node determines, according to the order in which the logs enter the log cache area, the part of logs to be deleted in each log eviction operation with data pages as the granularity. Specifically, when the log eviction condition is met, the slave node determines the part of logs to be deleted, which includes a first log and a second log. The first log is the log that enters the log cache area earliest, and the second log includes all other logs of a data page corresponding to the first log in the log cache area. For example, when the log eviction condition is met, the log cache area stores logs X1, X2, Y1, Z1, X3, and Z2 in the order in which the logs enter the log cache area, then the slave node determines the log X1 that enters the log cache area earliest as the log to be deleted, and further determines, because the log X1 is a log of a data page x, all other logs X2 and X3 of the data page x in the log cache area as the logs to be deleted. Finally, the slave node deletes the logs X1, X2, and X3 in this log eviction operation.

[0086] In the present application, before the slave node deletes the determined part of logs to be deleted from the log cache area, if the data page corresponding to the part of logs has been cached to the secondary cache of the slave node, the slave node will first update the data page corresponding to the part of logs in the secondary cache of the slave node according to the part of logs. For example, in the above embodiment of determining the part of logs to be deleted in each eviction behavior with data page as the granularity, before the slave node deletes logs X1, X2 and X3 from the log cache area, the slave node will first update the data page x in the secondary cache of the slave node to the new version by replaying the logs X1, X2 and X3.

[0087] The data management method provided by the present application is described in detail above in combination with the drawings, and the data management method provided by the present application is further described below in combination with specific examples.

[0088] In this example, the secondary cache of the slave node caches data page x, data page y and data page z. The slave node can update the data page stored in the secondary cache in different ways, which are described below respectively.

[0089] FIG. 4 is an example diagram of a first data management method provided by the present application, in which the slave node obtains the updated data page z in response to an access request about the data page z. Specifically, as shown in FIG. 4, after the slave node receives the access request for the data page z by using the business thread, the slave node reads the logs (i.e. logs Z1 to Z3) of the data page z from the log cache area based on the access request, and then obtains the updated data page z by replaying the logs Z1 to Z3. In addition, the slave node updates the data page z in the secondary cache of the slave node by using the updated data page z.

[0090] FIG. 5 is an example diagram of a second data management method provided by the present application, in which the slave node updates the data page corresponding to the part of logs to be deleted in the secondary cache before deleting the part of logs when the eviction condition is met. Specifically, as shown in FIG. 5, the slave node determines the part of logs to be deleted as logs Z1, Z2 and Z3, and the slave node obtains the updated data page z by replaying the logs Z1, Z2 and Z3 of the data page.

[0091] Compared with the data in the secondary cache of the slave node being updated by the master node, the slave node directly replays the log generated by the master node to synchronize the data in the secondary cache of the slave node. In the first aspect of the present application, the slave node reads the log of the data page from the log cache area of the slave node in response to the access request, and replays the log of the data page to obtain the updated data page. The slave node replays the log of the data page to obtain the updated data page in response to the access request, which can reduce the number of times of replaying the log of the slave node to obtain the updated data page in the case of frequent data update operations of the master node, reduce the log replay pressure of the slave node, and provide guarantee for the performance of the slave node.

[0092] It can be understood that, in order to realize the functions in the above-mentioned embodiments, the slave node includes corresponding hardware structures and / or software modules for performing various functions. Those skilled in the art should easily realize that, in combination with the units and method steps of the examples described in the embodiments disclosed in the present application, the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is realized in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0093] The embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a notebook computer, or a smart phone.

[0094] The present application also provides a computing device cluster, which can include at least one computing device. Each of the at least one computing device can be connected through a network. For example, FIG. 6 is a structural schematic diagram of a computing device cluster provided by the present application. As shown in FIG. 6, the computing device cluster includes three computing devices (such as computing device 200A, computing device 200B, and computing device 200C). The computing device 200A, the computing device 200B, and the computing device 200C can be connected through a network. The computing device 200A can be used to realize the function of the master node in the foregoing method embodiments, the computing device 200B can be used to realize the function of the slave node in the foregoing method embodiments, and the computing device 200C can be used to realize the function of the shared storage cluster in the foregoing method embodiments. In this way, the computing device cluster can constitute the database system 100 described in FIG. 1.

[0095] On the hardware side, the computing device 200A can include a processor 211a, a memory 212a, a communication interface 214a, and a bus 215a. The processor 211a, the memory 212a, and the communication interface 214a can be connected with each other through the bus 215a. For more details about the computing device 200A, please refer to FIG. 2A and FIG. 2B above, which will not be repeated here. The instructions stored in the memory 212a can implement the functions of the master node in the foregoing method embodiments.

[0096] On the hardware side, the computing device 200B can include a processor 211b, a memory 212b, a communication interface 214b, and a bus 215b. The processor 211b, the memory 212b, and the communication interface 214b can be connected with each other through the bus 215b. For more details about the computing device 200B, please refer to FIG. 2A and FIG. 2B above, which will not be repeated here. The instructions stored in the memory 212b can implement the functions of the slave node in the foregoing method embodiments.

[0097] On the hardware side, the computing device 200C can include a processor 211c, a memory 212c, a communication interface 214c, and a bus 215c. The processor 211c, the memory 212c, and the communication interface 214c can be connected with each other through the bus 215c. For more details about the computing device 200C, please refer to FIG. 2A and FIG. 2B above, which will not be repeated here. The instructions stored in the memory 212c can implement the functions of the shared storage cluster in the foregoing method embodiments.

[0098] It should be understood that the functions of the computing device 200A shown in FIG. 6 can be completed by multiple computing devices. Similarly, the functions of the computing device 200B and the computing device 200C can also be completed by multiple computing devices.

[0099] The embodiments of the present application also provide another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to the connection mode of the computing device cluster shown in FIG. 6. The difference is that the memory in one or more computing devices in the computing device cluster can store the same instructions for executing the data management method.

[0100] In some possible implementation manners, the memory in one or more computing devices in the computing device cluster can also respectively store partial instructions for executing the data management method. In other words, the combination of one or more computing devices can collectively execute the instructions for executing the data management method.

[0101] The memories in different computing devices in the computing device cluster can store different instructions for executing part of the functions of the database system. That is, the instructions stored in the memories in different computing devices can implement one or more functions deployed in the master node and the slave node.

[0102] The steps of the method in the embodiments can be implemented by hardware, or by a combination of software and hardware. The software instructions can be composed of one or more pieces of software modules. The software modules can be stored in any suitable storage medium or device, such as a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk, a floppy disk, a CD-ROM, or any other forms of storage mediums well known to one skilled in the art. An exemplary storage medium is coupled to the processor, such that the processor can read information from, and write information to, the storage medium. Of course, the storage medium can be a component of the processor. The processor and the storage medium can be located in an ASIC. The ASIC can be located in a computing device. Of course, the processor and the storage medium can also be present in a network device or a terminal device as discrete components.

[0103] The present application also provides a chip system comprising a processor for implementing the functions of the master node and / or the slave node in the above method. In a possible design, the chip system further comprises a memory for storing program instructions and / or data. The chip system can be composed of a chip, or can include a chip and other discrete components.

[0104] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer programs or instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments are performed. The computer can be a general purpose computer, a special purpose computer, a computer network, a network device, a user equipment or other programmable apparatus. The computer programs or instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer programs or instructions can be transferred from one website site, computer, server or data center to another website site, computer, server or data center through wired or wireless manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center and the like that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, a magnetic tape; or an optical medium, such as a digital video disc (digital video disc, DVD); or a semiconductor medium, such as a solid state drive (solid state drive, SSD).

[0105] The embodiments of the present application also provide a computer program product containing instructions. The computer program product can be a software or program product containing instructions, which can run on a computing device or be stored in any available medium. When the computer program product runs on at least one computing device, the at least one computing device is caused to perform the method provided by any of the above embodiments.

[0106] The embodiments of the present application also provide a computer readable storage medium. The computer readable storage medium can be any available medium that can be accessed by a computing device or a data storage device such as a data center and the like that contains one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state drive) and the like. The computer readable storage medium includes instructions, which instruct the computing device to perform the method provided by any of the above embodiments.

[0107] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A data management method characterized by, The method is applied to a database system, the database system comprising a master node, a slave node and a shared storage cluster, wherein the master node and the slave node are both computing nodes deployed with a database instance, and the shared storage cluster comprises at least one storage node, and the slave node comprises a memory and a secondary cache; the method comprises: After the master node sends a log of a data page to the shared storage cluster for storage, the slave node acquires the log of the data page from the shared storage cluster and stores the log of the data page to a log cache area of the slave node; wherein the log of the data page is used to record a modification operation of the data page; When the slave node receives an access request related to the data page, in a case that the data page has been cached to the secondary cache of the slave node, the slave node reads the log of the data page from the log cache area of the slave node in response to the access request, and replays the log of the data page to obtain an updated data page; The slave node feeds back the access request according to the updated data page.

2. The method of claim 1, wherein, After the slave node reads the log of the data page from the log cache area in response to the access request, and replays the log of the data page to obtain the updated data page, the method further comprises: The slave node updates the data page in the secondary cache of the slave node.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: When a log elimination condition is met, the slave node deletes part of logs from the log cache area.

4. The method of claim 3, wherein, The log elimination condition is any one of the following conditions: The capacity of the logs stored in the log cache area reaches a preset capacity threshold; The remaining space of the log cache area reaches a preset space threshold; Or The time interval between two adjacent elimination behaviors reaches a preset elimination period.

5. The method according to claim 3 or 4, characterized in that, Before the slave node deletes the part of logs from the log cache area, the method further comprises: If the data page corresponding to the part of logs has been cached to the secondary cache of the slave node, the slave node updates the data page corresponding to the part of logs in the secondary cache of the slave node according to the part of logs.

6. The method according to any one of claims 3-5, characterized in that, The slave node deleting the part of logs from the log cache area comprises: The slave node deletes the part of logs from the log cache area according to the order in which the logs enter the log cache area, and the order in which the part of logs enter the log cache area is earlier than that of other logs in the log cache area.

7. The method according to any one of claims 3-5, characterized in that, The part of logs comprises a first log and a second log, wherein the first log is the earliest log entering the log cache area, and the second log comprises all other logs of the data page corresponding to the first log in the log cache area.

8. The method according to any one of claims 1 to 7, characterized in that, The method further comprises: The master node performs a modification operation of a data page; The master node generates a log of the data page, the log of the data page being used to record the modification operation of the data page; The master node sends the log of the data page to the shared storage cluster for storage.

9. A database system, characterized in that The database system comprises a master node, a slave node and a shared storage cluster, wherein the master node and the slave node are both computing nodes deployed with a database instance, and the shared storage cluster comprises at least one storage node, and the slave node comprises a memory and a secondary cache; the slave node is configured to: after the master node sends a log of a data page to the shared storage cluster for storage, acquire the log of the data page from the shared storage cluster, and store the log of the data page to a log cache area of the slave node, wherein the log of the data page is used to record a modification operation of the data page; when the slave node receives an access request related to the data page, in a case that the data page has been cached to the secondary cache of the slave node, read the log of the data page from the log cache area of the slave node, and replay the log of the data page to obtain an updated data page; feed back the access request according to the updated data page.

10. The system of claim 9, wherein the slave node is further configured to update the data page in the secondary cache of the slave node.

11. The system of claim 9 or 10, wherein, the slave node is further configured to delete part of logs from the log cache area when a log eviction condition is met.

12. The system of claim 11, wherein, the log eviction condition is any one of the following conditions: a capacity of the logs stored in the log cache area reaches a preset capacity threshold; a remaining space of the log cache area reaches a preset space threshold; or a time interval between two adjacent eviction behaviors reaches a preset eviction period. the slave node is further configured to, before the slave node deletes the part of logs from the log cache area, if a data page corresponding to the part of logs has been cached to the secondary cache of the slave node, update the data page corresponding to the part of logs in the secondary cache of the slave node according to the part of logs.

13. The system of claim 11 or 12, wherein, the slave node is specifically configured to delete part of logs from the log cache area according to an order in which the logs enter the log cache area, and the order in which the part of logs enter the log cache area is earlier than that of other logs in the log cache area.

14. The system of any one of claims 11-13, wherein, the part of logs comprises a first log and a second log, wherein the first log is a log that enters the log cache area earliest, and the second log comprises all other logs of a data page corresponding to the first log in the log cache area.

15. The system of any one of claims 11-13, wherein, the master node is configured to:

16. The system of any one of claims 9-15, wherein, perform a modification operation of a data page; generate a log of the data page, wherein the log of the data page is used to record the modification operation of the data page; send the log of the data page to the shared storage cluster for storage. the computing device cluster comprises at least one computing device, and each computing device comprises a processor and a memory; 17. A cluster of computing devices, characterized in that, the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method in any one of claims 1-8. ​ 18. A computer-readable storage medium, characterized in that, The computer readable storage medium includes computer instructions; when the computer instructions run in a computing device, the computing device executes the method in any one of claims 1-8.

19. A computer program product, characterised in that, When the computer program product runs in a computing device, the computing device executes the method in any one of claims 1-8.

Images