Data recovery method, computing device, and storage medium

Abstract

The embodiment of the present application provides a data recovery method, a computing device and a storage medium. In the embodiment of the present application, data is written to a local memory node, record information for recording a write operation corresponding to the written data is generated and stored according to the write operation; the written data in the local memory node is written to a remote memory node; first positioning information is generated and stored, and the first positioning information is used for positioning the written data that has been written to the remote memory node; in the case that the computing node is abnormal, the written data in the local memory node that has not been written to the remote memory node is recovered according to the first positioning information and the record information. Therefore, the written data in the remote memory node is retained, and the life cycle of the remote memory node is maintained. The life cycle of the remote memory node is effectively utilized, and the data recovery amount is reduced.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data recovery method, computing device, and storage medium. Background Technology

[0002] Database memory separation architecture is a cloud-native architecture that evolved from cloud-native database storage-compute separation. In this architecture, local memory and separated memory are logically managed uniformly. However, when a compute node crashes, the local memory is also cleared when the compute node restarts, resulting in an incomplete state in the separated memory. The separated memory needs to restart along with the compute node, relying on the storage state for crash recovery. This recovery method wastes the independent lifecycle of the separated memory and the data recovery process is slow. Summary of the Invention

[0003] This application provides a data recovery method, computing device, and storage medium that can effectively utilize data in isolated memory and perform data recovery relatively quickly.

[0004] This application provides a data recovery method, comprising: writing data to a local memory node; generating and storing record information for recording the write operation according to the corresponding write operation; writing the written data in the local memory node to a remote memory node; generating and storing first positioning information, the first positioning information being used to locate the written data that has been written to the remote memory node; and recovering the written data in the local memory node that has not been written to the remote memory node according to the first positioning information and the record information in the event of a computing node malfunction.

[0005] This application embodiment also provides a computing device, including: a memory and a processor; the memory is used to store a computer program; the processor executes the computer program to: write data to a local memory node; generate and store record information for recording the write operation according to the corresponding write operation; write the written data in the local memory node to a remote memory node; generate and store first positioning information, the first positioning information being used to locate the written data already written to the remote memory node; and, in the event of a computing node malfunction, recover the written data in the local memory node that has not been written to the remote memory node according to the first positioning information and the record information.

[0006] This application also provides a computer-readable storage medium storing a computer program, which, when executed by one or more processors, causes the one or more processors to perform the steps in the above-described method.

[0007] This application also provides a computer program product, including a computer program or instructions, wherein when the computer program is executed by a processor, it causes the processor to perform the steps in the above-described method.

[0008] This application embodiment also provides a data recovery system, including: a first device and a second device; the first device sends a data write request to the second device; the second device, according to the received write request, writes data to a local memory node, generates and stores record information for recording the write operation according to the corresponding write operation; writes the written data in the local memory node to a remote memory node; generates and stores first positioning information, the first positioning information being used to locate the written data already written to the remote memory node; in the event of a computing node anomaly, recovers the written data in the local memory node that has not been written to the remote memory node according to the first positioning information and the record information.

[0009] In this embodiment, data is written to the local memory node, and record information for recording the write operation is generated and stored according to the corresponding write operation; the written data in the local memory node is written to the remote memory node; first positioning information is generated and stored, which is used to locate the written data that has been written to the remote memory node; in the event of a computing node malfunction, the written data in the local memory node that has not been written to the remote memory node is recovered according to the first positioning information and the record information.

[0010] In the event of a compute node failure, based on the initial location information and recorded information, data written to the local memory node that has not yet been written to the remote memory node is recovered. This eliminates the need to recover data already written to the remote memory node, thus preserving the written data in the remote memory node and maintaining its lifecycle. This effectively utilizes the lifecycle of the remote memory node, reduces the amount of data recovery, and significantly improves data recovery efficiency. Attached Figure Description

[0011] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0012] Figure 1 A schematic flowchart illustrating a data recovery method as an exemplary embodiment of this application;

[0013] Figure 2 A schematic diagram illustrating data recovery as an exemplary embodiment of this application;

[0014] Figure 3This is a schematic diagram of the structure of a data recovery system as an exemplary embodiment of this application;

[0015] Figure 4 A schematic flowchart illustrating a data recovery method as an exemplary embodiment of this application;

[0016] Figure 5 A schematic diagram of the structure of a data recovery apparatus provided in an exemplary embodiment of this application;

[0017] Figure 6 This is a schematic diagram of the structure of a computing device provided for an exemplary embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] As mentioned earlier, when a computing node, such as a CPU (central processing unit), crashes, its local memory and discrete memory are cleared upon restarting. While existing technologies can restore the data to a consistent state, this approach is ineffective in accelerating data recovery using data already stored in discrete memory in a discrete memory architecture, and the data in discrete memory needs to be reset.

[0020] Based on this, embodiments of this application provide a data recovery method, computing device, and storage medium, which can effectively utilize data in separate memory and perform data recovery relatively quickly.

[0021] The data recovery process will be described in detail below with reference to the method embodiments.

[0022] Figure 1 This is a flowchart illustrating a data recovery method according to an exemplary embodiment of this application. The method 100 provided in this embodiment is executed by a computing device, such as a server. The method 100 includes the following steps:

[0023] 101: Write data to the local memory node, and generate and store record information for recording the write operation based on the corresponding write operation.

[0024] 102: Write the data to be written in the local memory node to the remote memory node.

[0025] 103: Generate and store the first location information.

[0026] The first location information is used to locate the data that has been written to the remote memory node.

[0027] 104: In the event of a compute node malfunction, recover the write data in the local memory node that was not written to the remote memory node based on the first location information and the record information.

[0028] It should be noted that the execution entity in this application embodiment can more specifically be a physical host of a cloud server, etc. The corresponding architecture can be a storage-separated and memory-separated architecture. Specifically, for memory separation, corresponding memory is deployed on different physical hosts. For a local memory node, the memory on other physical hosts can be considered a remote memory node. Furthermore, for a local memory node, it also has a corresponding local computing node, such as a CPU. This computing node is not separated. Based on this, a corresponding memory-separated database can be implemented.

[0029] The following is a detailed explanation of the above steps:

[0030] 101: Write data to the local memory node, and generate and store record information for recording the write operation based on the corresponding write operation.

[0031] In this context, a local memory node refers to memory deployed on a local physical host. Furthermore, if a corresponding compute node, such as a GPU, is also deployed on that local physical host, it can also be considered a local compute node.

[0032] A write operation can refer to a specific action performed on data, such as modifying data (or transaction) A to data (or transaction) B. This write operation can include data modification operations, data storage, or write operations.

[0033] For example, a user can send an SQL (Structured Query Language) statement to a database server via a terminal, such as a computer, to modify data. Upon receiving the statement, the database server can parse it and invoke the corresponding execution engine to modify the data. The execution engine can then modify the data in local memory using a transaction manager on the local physical host. This modification process is then recorded as a log, documenting the modification operation, such as changing data A to data B.

[0034] Specifically, data is written to local memory nodes. Based on the corresponding write operation, log information for recording the write operation is generated and stored, including: receiving data modification requests, modifying the corresponding data in the corresponding memory page of the local memory node according to the modification request; generating modification operation logs for the corresponding data modification operations, and storing the modification operation logs.

[0035] For example, as described above, a user can send a data modification request via computer. This request may include the SQL statement used for data modification, or the SQL statement itself may be the data modification request. As described above, such as... Figure 2 As shown, the transaction manager 204 on the local physical host receives the instruction to modify the data. Based on this instruction, it modifies the data in the corresponding memory page in local memory, i.e., it executes step 210: modifying the data in the memory page. Then, a corresponding redo log is generated to record the modification operation. This log is then stored. Multiple memory pages 2011 are stored in the local buffer pool 2012 (PUB, i.e., local memory or a local memory node) in the local compute node 201.

[0036] It should be noted that the database in this application embodiment can be a relational database. Furthermore, a memory page refers to data corresponding to a given address.

[0037] The storage of modification operation logs includes: storing the modification operation logs in a log cache, and then storing the modification operation logs in a storage device through the log cache.

[0038] For example, as mentioned above, such as Figure 2 As shown, on the local physical host, the generated redo logs are stored in the log buffer 2014 in the local compute node 201, i.e., step 211: writing the modification operation log to the log buffer 2014. Then, through the log buffer 2014, the redo logs are written to the storage device 203 (Persistent Shared Storage) to complete the transaction commit, i.e., step 213: transaction commit to the storage device 203, or more specifically, it can be written to the write-ahead log 2031 in the storage device 203.

[0039] In addition, after successfully storing the above logs, storage device 203 can return an acknowledgment message to log cache 2014 to ensure that the logs are successfully written to storage device 203.

[0040] 102: Write the data to be written in the local memory node to the remote memory node.

[0041] In this context, a remote memory node can refer to memory deployed on other physical hosts. For example... Figure 2 Remote memory node 202 shown.

[0042] For example, as mentioned above, such as Figure 2 As shown, the modified memory page 2011 (which can be multiple memory pages) in the local cache pool 2012 is written to the corresponding cache pool in the remote memory node 202.

[0043] Specifically, writing data from a local memory node to a remote memory node includes: writing the corresponding memory page in the local memory node containing the data to a local linked list; and writing the corresponding memory page to the remote memory node through the local linked list.

[0044] Among them, the local linked list is located in such a way as Figure 2 The local computing node 201 shown.

[0045] For example, as mentioned above, such as Figure 2 As shown, the modified memory page is added to the local linked list (FLUSH_LBP, also known as the local dirty page list) 2013 in the local compute node 201 through the local cache pool 2012, that is, step 212: write the modified memory page to the local linked list 2013. The modified memory page 2011 is then written to the remote memory node 202 through the local linked list 2013, more specifically, to the remote cache pool (RemoteBuffer Pool) 2021 in the remote memory node 202.

[0046] Furthermore, writing to a remote memory node can be triggered via a local intermediate component (LFT Daemon). For example... Figure 2 As shown, according to the preceding text, step 214 is executed through the local linked list 2013: triggering the component, that is, triggering the local intermediate component 2015 to write the modified memory page 2011 into the remote cache pool 2021 in the remote memory node 202. That is, step 215 is executed: writing to the remote memory node, to the remote cache pool 2021.

[0047] It should be noted that data writing to remote memory nodes is possible even without intermediate components.

[0048] Additionally, such as Figure 2As shown, step 223 can also be executed to the transaction manager 204 via the local linked list 2013: sending the transaction result, i.e., confirming the modification transaction. The transaction manager 204 then returns the confirmation to the user's computer, informing the user of the transaction confirmation. At this point, the log cache 2014 has completed the transaction commit.

[0049] 103: Generate and store the first location information.

[0050] The first location information refers to information used to locate data that has been written to a remote memory node, such as a checkpoint. This checkpoint indicates that all data prior to this point has been stored.

[0051] For example, as described above, on the local physical host, when writing modified memory pages from the local memory node to the remote memory node, an L1 checkpoint can be set after a preset number of memory pages have been written to the remote memory node. This checkpoint indicates that all data prior to this point has been stored on the remote memory node. This L1 checkpoint can be updated as the write process progresses until a database error occurs and data recovery is required.

[0052] Specifically, generating and storing first location information for locating data written to a remote memory node includes: determining the first location information based on the memory pages written to the remote memory node; and storing the first location information in the remote memory node.

[0053] For example, as described above, when writing modified memory pages from the local memory node to the remote cache pool in the remote memory node, an L1 checkpoint can be set after a preset number of memory pages have been written to the remote cache pool in the remote memory node. This L1 checkpoint indicates that all data prior to this point has been stored on the remote memory node. This L1 checkpoint can be updated as the write process progresses until a database error occurs and data recovery is required. Figure 2 As shown, an L1 checkpoint can be set via the local linked list 2013 and then stored in the first location information cache (checkpoints) 2022 in the remote memory node 202. Alternatively, the local intermediate component 2015 can be triggered by the local linked list 2013 to set an L1 checkpoint and then stored in the first location information cache (checkpoints) 2022 in the remote memory node 202. That is, step 216 is executed: store the first location information in the first location information cache (checkpoints) 2022.

[0054] 104: In the event of a compute node malfunction, recover the write data in the local memory node that was not written to the remote memory node based on the first location information and the record information.

[0055] Among them, the computing node is located locally, that is, the local computing node: the local computing node is deployed on the local physical host.

[0056] Compute node failure refers to an abnormality in the local compute node, such as a power outage or a crash in the corresponding CPU software.

[0057] For example, as mentioned earlier, when the local CPU crashes, such as after a power outage, it is restarted upon power restoration. After the CPU restarts, it obtains the L1 checkpoint from the remote memory node and the stored redo log. Then, based on the L1 checkpoint, it determines the memory pages that have been written to the remote memory node, and then uses the redo log to find the operations corresponding to subsequent memory pages that have not been written to the remote memory node, and performs data recovery. The specific recovery process is similar to the method described above, so it will not be repeated here. It will only be noted that the recovery process is similar to... Figure 2 As shown, starting from step 210, the data in the memory page is modified.

[0058] It should be understood that the recovery process for stored data is similar, so it will not be elaborated further.

[0059] It's important to note that for better data recovery, memory pages are assigned corresponding identifiers, such as IDs. Logs can also have corresponding identifiers, such as IDs. Furthermore, the log identifiers can correspond to the memory page identifiers. Therefore, the ID of a memory page can be determined based on the checkpoint, and then the subsequent redo log entries for that ID can be searched for data recovery.

[0060] Specifically, based on the first location information and the record information, the write data in the local memory node that has not been written to the remote memory node is recovered, including: reading the first location information in the remote memory node and reading the modification operation log in the storage device; determining the unwritten write data based on the first location information, and recovering the unwritten data in the corresponding memory page in the local memory node based on the modification operation log, so that the recovered write data is ready to be written to the remote memory node.

[0061] For example, as mentioned above, such as Figure 2As shown, when the local CPU crashes, such as after a power outage, it is restarted upon power restoration. After restarting, the CPU reads the L1 checkpoint stored in the first location information cache 2022 of the remote memory node 202, and reads the redo log in the storage device 203, specifically from the write-ahead log 2031 in the storage device 203. Then, based on the L1 checkpoint, it determines the ID of the memory page 2011 that has been written to the remote memory node 202, and then searches for the subsequent redo logs for that ID. Based on the redo logs, it finds the operation corresponding to the memory page 2011 that has not been written to the remote memory node 202, and performs data recovery. The specific recovery process is similar to the method described above, so it will not be repeated here. It is only noted that the recovery process is as follows... Figure 2 As shown, starting from step 210, the data in memory page 2011 is modified until it is stored in remote cache pool 2021 in remote memory node 202.

[0062] It's important to note that L1 checkpointing enables detached memory (i.e., remote memory nodes) to have independent recovery capabilities. Detached memory can independently undergo crash recovery and continue to be used, greatly accelerating the crash recovery and performance recovery process.

[0063] Since the data corresponding to the memory pages in the remote cache pool will be stored in the storage device, if the data has already started to be stored in the storage device and the local computing node is abnormal, the above method can not only restore the data that has not been written to the remote memory node, but also restore the data that has not been written to the storage device.

[0064] Within the remote memory node, data has already begun to be stored on the storage device. If the remote memory node malfunctions, data recovery is also required. The specific recovery methods are as follows:

[0065] Specifically, the method 100 further includes: storing the written data in the remote memory node to a storage device; generating and storing second location information, the second location information being used to locate the written data that has been stored in the storage device; and, in the event of an anomaly in the remote memory node, recovering the written data in the remote memory node that has not been written to the storage device based on the record information and the second location information.

[0066] The second location information refers to information used to locate data that has been written to the storage device. For example, a checkpoint is similar to the first location information and will not be elaborated further.

[0067] For example, as described above, stored memory pages are written to the storage device via a remote memory node. During the storage process, an L2 checkpoint is set to indicate that all data prior to that point has been stored on the device. This L2 checkpoint can be updated as the writing process progresses until a remote memory node in the database fails and data recovery is required. During recovery, the local compute node needs to obtain the corresponding redo logs and L2 checkpoints to recover the written data from the remote memory node that has not yet been written to the storage device.

[0068] The process of storing write data from a remote memory node to a storage device includes: writing the corresponding memory page in the remote memory node containing the write data to a remote linked list; and storing the corresponding memory page to the storage device via the remote linked list.

[0069] For example, as mentioned above, such as Figure 2 As shown, memory pages in the remote cache pool 2021 of the remote memory node 202 can be added to the remote linked list (FLUSH_RBP, remote dirty page list) 2023. Alternatively, memory page 2011 can be added to the remote linked list 2023 by triggering the local intermediate component 2015, i.e., executing step 217: writing memory page 2011 from the remote memory node to the remote linked list 2023. Since the local intermediate component 2015 also executed step 215, this component knows the information of memory page 2011 in the remote cache pool, such as its address and data, so step 217 can be executed through this component. Further details are omitted. The data in memory page 2011 is stored in data 2033 in storage device 203 through the remote linked list 2023. This completes the persistent storage of the data.

[0070] Or, such as Figure 2 As shown, data can also be stored in storage device 203 by triggering the remote intermediate component (HFTDaemon) 2024 in the remote memory node 202. Through the remote linked list 2023, the remote intermediate component 2024 is triggered to write the data corresponding to memory page 2011 to storage device 203, i.e., step 218: triggering the component to write data. The remote intermediate component 2024 then writes the corresponding data to data 2033 in storage device 203, i.e., step 219: writing data to data 2033 in storage device 203 is executed through the remote intermediate component.

[0071] The process of generating and storing the second location information includes: determining the second location information based on memory pages already written to the storage device; and storing the second location information in the storage device.

[0072] For example, as mentioned above, such as Figure 2 As shown, when writing memory pages 2011 from remote memory node 202 to storage device 203, an L2 checkpoint can be set after a preset number of memory pages 2011 have been written to storage device 203. This indicates that all data prior to this point has been stored on the device. This L2 checkpoint can be updated as the writing process progresses until a remote memory node in the database malfunctions and data recovery is required. The L2 checkpoint can be set via remote linked list 2024 and then stored in the second location information (checkpoints) 2032 in storage device 203. Alternatively, the remote linked list 2024 can trigger the remote intermediate component 2024 to set the L2 checkpoint and then store it in the second location information (checkpoints) 2032 in storage device 203. That is, step 220 is executed: storing the second location information in the second location information (checkpoints) 2032 in storage device 203.

[0073] Specifically, based on the recorded information and the second location information, the write data in the remote memory node that has not been written to the storage device is recovered, including: reading the modification operation log in the storage device and the second location information; determining the unwritten write data based on the second location information, and recovering the unwritten data in the corresponding memory page in the local memory node based on the modification operation log, so that the recovered write data is ready to be stored in the storage device.

[0074] Among them, remote memory node failure can refer to a single remote memory node failure (such as power failure) or a combination of local computing node failure and remote memory node failure, that is, both failures occur simultaneously.

[0075] For example, as mentioned above, such as Figure 2 As shown, when a remote memory node crashes, such as after a power outage, it is restarted upon power restoration. After restarting, the local CPU reads the L2 checkpoint stored in the second location information 2032 in storage device 203, and reads the redo log in storage device 203, specifically from the write-ahead log 2031 in storage device 203. Then, based on the L2 checkpoint, it determines the ID of the memory page 2011 that has been written to storage device 203, and then searches for the subsequent redo logs for that ID. Based on the redo logs, it finds the operation corresponding to the memory page 2011 that has not been written to storage device 203, and performs data recovery. The specific recovery process is similar to the method described above, and will not be repeated here. It is only noted that the recovery process is as follows... Figure 2As shown, starting from step 210, the data in memory page 2011 is modified until it is stored in data 2033 of storage device 203.

[0076] It should be noted that after successfully writing data to storage device 203, the corresponding log in storage device 203 can be deleted. For example, ... Figure 2 As shown, the remote intermediate component 2024 triggers the local intermediate component 2015 to delete the corresponding log, such as the redo log, in the write-ahead log 2031 of the storage device 203. Then, the local intermediate component 2015 deletes the corresponding log in the write-ahead log 2031 of the storage device 203. That is, step 221: triggering the component to delete the write-ahead log, and step 222: deleting the write-ahead log are executed.

[0077] Therefore, the L2 checkpoint described above can be used to recover data when a remote memory node fails.

[0078] Figure 3 This is a schematic diagram of the structure of a data recovery system provided as an exemplary embodiment of this application. Figure 3 As shown, the system 300 may include a first device 301 and a second device 302.

[0079] The first device 301 can be a device with certain computing capabilities, capable of sending data to the second device 302 and receiving data sent by the second device 302. The basic structure of the first device 301 may include at least one processor. The number of processors may depend on the configuration and type of the device with certain computing capabilities. The device with certain computing capabilities may also include memory, which can be volatile, such as RAM, or non-volatile, such as read-only memory (ROM), flash memory, etc., or may include both types simultaneously. The memory typically stores an operating system (OS), one or more applications, and may also store program data. In addition to the processing unit and memory, the device with certain computing capabilities also includes some basic configurations, such as a network interface card (NIC) chip, an I / O bus, a display component, and some peripheral devices. Optionally, some peripheral devices may include, for example, a keyboard, a stylus, etc. Other peripheral devices are well known in the art and will not be described in detail here. Optionally, the first device 301 can be a smart terminal, such as a mobile phone, desktop computer, laptop, tablet computer, etc.

[0080] The second device 302 refers to a device that can provide computing processing services in a network virtual environment, and may refer to a device that uses the network for data recovery. In physical implementation, the second device 302 can be any device capable of providing computing services, responding to service requests, acquiring data, and performing data recovery; for example, it could be a cloud server, cloud host, virtual data center, or conventional server. The second device 302 mainly consists of a processor, hard disk, memory, system bus, etc., similar to a general computer architecture.

[0081] Specifically, the first device 301 sends a data write request to the second device.

[0082] Specifically, the second device 302 writes data to the local memory node according to the received write request, generates and stores record information for recording the write operation according to the corresponding write operation, writes the write data in the local memory node to the remote memory node, generates and stores first positioning information, which is used to locate the write data that has been written to the remote memory node, and recovers the write data in the local memory node that has not been written to the remote memory node according to the first positioning information and the record information in the event of a computing node failure.

[0083] The computing nodes are located locally.

[0084] Specifically, the second device 302 receives a data modification request, modifies the corresponding data in the corresponding memory page of the local memory node according to the modification request, generates a modification operation log for the modification operation of the corresponding data, and stores the modification operation log.

[0085] First device 301 sends a data modification request to second device 302.

[0086] Specifically, the second device 302 stores the modification operation log in the log cache, and then stores the modification operation log in the storage device through the log cache.

[0087] Specifically, the second device 302 determines the first location information based on the memory pages already written to the remote memory node; and stores the first location information in the remote memory node.

[0088] Specifically, the second device 302 reads the first location information in the remote memory node and the modification operation log in the storage device; determines the unwritten write data based on the first location information, and restores the unwritten data in the corresponding memory page in the local memory node according to the modification operation log, so that the restored write data is ready to be written to the remote memory node.

[0089] In addition, the second device 302 stores the written data in the remote memory node to the storage device; generates and stores second location information, which is used to locate the written data that has been stored in the storage device; and, in the event of an anomaly in the remote memory node, recovers the written data in the remote memory node that has not been written to the storage device based on the recorded information and the second location information.

[0090] Specifically, the second device 302 determines the second location information based on the memory pages already written to the storage device; and stores the second location information in the storage device.

[0091] Specifically, the second device 302 reads the modification operation log and the second location information in the storage device; determines the unwritten data based on the second location information, and restores the unwritten data in the corresponding memory page in the local memory node according to the modification operation log, so that the restored write data is ready to be stored in the storage device.

[0092] It should be noted that for any content not fully described in System 300, please refer to the content in Method 100 above, and for the specific implementation method, please refer to the specific implementation method of Method 100 above, which will not be repeated here.

[0093] In the data recovery scenario of this application embodiment, such as Figure 3 As shown, a user can use a first device 301, such as a computer, to execute step 310: sending a data modification request to a second device 302, such as a database server. This request may include the SQL statement used for data modification, or the SQL statement itself may be the data modification request. Upon receiving the request, the database server can parse the SQL statement in the request and call the corresponding execution engine to modify the data. The corresponding execution engine can use the transaction manager 204 on the local physical host to modify the data in local memory. Then, the modification process, i.e., the corresponding modification operation, is recorded, such as changing data A to data B.

[0094] The transaction manager 204 on the local physical host of the database server receives the instruction to modify the data. Based on this instruction, it modifies the data in the corresponding memory page in local memory, i.e., executes step 210: modify the data in the memory page. Then, a corresponding redo log is generated to record the modification operation. The generated redo log is stored in the log buffer 2014 on the local compute node 201, i.e., executes step 211: write the modification operation log to the log buffer 2014. Then, through the log buffer 2014, the redo log is written to the storage device 203, completing the transaction commit, i.e., executes step 213: transaction commit to the storage device 203, or more specifically, it can be written to the write-ahead log 2031 in the storage device 203.

[0095] The modified memory page is added to the local linked list 2013 in the local compute node 201 via the local cache pool 2012, i.e., step 212: write the modified memory page to the local linked list 2013. Step 214 is then executed via the local linked list 2013: triggering the component, i.e., triggering the local intermediate component 2015 to write the modified memory page 2011 to the remote cache pool 2021 in the remote memory node 202. Step 215 is then executed: write to the remote memory node, i.e., write to the remote cache pool 2021. Step 223 can also be executed via the local linked list 2013 to the transaction manager 204: send the transaction result, i.e., confirm the modification transaction. The transaction manager 204 then returns the confirmation to the user's computer, informing the user of the modification transaction confirmation, i.e., step 311: send the modification confirmation result. At this point, the log cache 2014 has completed the transaction commit.

[0096] When writing the modified memory page from the local memory node to the remote cache pool in the remote memory node, the local intermediate component 2015 can be triggered by the local linked list 2013 to set an L1 checkpoint, and then stored in the first location information cache (checkpoints) 2022 in the remote memory node 202. That is, step 216 is executed: store the first location information in the first location information cache (checkpoints) 2022.

[0097] When the local CPU crashes, such as after a power outage, it is restarted upon power restoration. After restarting, the CPU reads the L1 checkpoint stored in the first location information cache 2022 of the remote memory node 202, and reads the redo log in the storage device 203, specifically from the write-ahead log 2031 in the storage device 203. Then, based on the L1 checkpoint, it determines the ID of the memory page 2011 that has been written to the remote memory node 202, and then searches for the subsequent redo logs for that ID. Based on the redo logs, it finds the operations corresponding to the memory page 2011 that has not been written to the remote memory node 202, and performs data recovery.

[0098] By triggering the local intermediate component 2015 to add memory page 2011 to the remote linked list 2023, step 217 is executed: write memory page 2011 from the remote memory node to the remote linked list 2023. Through the remote linked list 2023, the remote intermediate component 2024 is triggered to write the data corresponding to memory page 2011 to the storage device 203, step 218 is executed: trigger component to write data. Through the remote intermediate component 2024, the corresponding data is written to data 2033 in the storage device 203, step 219 is executed: write data to data 2033 in the storage device 203.

[0099] When writing memory pages 2011 from remote memory node 202 to storage device 203, an L2 checkpoint can be set after a preset number of memory pages 2011 have been written to storage device 203. The L2 checkpoint can be set by triggering the remote intermediate component 2024 via the remote linked list 2024, and then stored in the second location information (checkpoints) 2032 in storage device 203. That is, step 220 is executed: storing the second location information in the second location information (checkpoints) 2032 in storage device 203.

[0100] When a remote memory node crashes, such as after a power outage, it is restarted upon power restoration. After restarting, the local CPU reads the L2 checkpoint stored in the second location information 2032 in storage device 203, and also reads the redo log in storage device 203. Based on the redo log, it locates the operation corresponding to the memory page 2011 that was not subsequently written to storage device 203, and performs data recovery.

[0101] For details not covered here, please refer to the preceding text; they will not be repeated here.

[0102] In the above embodiment, the first device 301 and the second device 302 are connected to a network. If the first device 301 and the second device 302 are connected for communication, the network standard of the mobile network can be any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G+ (LTE+), WiMax, 5G, etc.

[0103] Based on a similar inventive concept, this application provides an exemplary embodiment of a data recovery method. The method provided in this embodiment is executed by a computing device, such as a server. Figure 4 As shown, the method 400 includes the following steps:

[0104] 401: Data written to a remote memory node is stored in a storage device.

[0105] 402: Generate and store second location information.

[0106] The second location information is used to locate the data that has been written to the storage device.

[0107] 403: In the event of a remote memory node malfunction, recover the written data in the remote memory node that has not been written to the storage device based on the recorded information and the second location information.

[0108] Specifically, storing the data to be written in remote local memory to a storage device includes: writing the corresponding memory page in the remote memory node where the data to be written to a remote linked list; and storing the corresponding memory page to the storage device through the remote linked list.

[0109] Specifically, generating and storing the second location information includes: determining the second location information based on memory pages already written to the storage device; and storing the second location information in the storage device.

[0110] Specifically, based on the recorded information and the second location information, the write data in the remote memory node that has not been written to the storage device is recovered, including: reading the modification operation log in the storage device and the second location information; determining the unwritten write data based on the second location information, and recovering the unwritten data in the corresponding memory page in the local memory node based on the modification operation log, so that the recovered write data is ready to be stored in the storage device.

[0111] It should be noted that since the specific implementation methods of steps 401-403 are similar to the specific implementation methods of method 200 described above, they will not be repeated here.

[0112] For details regarding the missing information in this method 400, please refer to the previous text; further explanation will not be provided here.

[0113] Figure 5 This is a schematic diagram of the structural framework of a data recovery device provided for an exemplary embodiment of this application. The device 500 can be applied to a server. The device 500 includes: a generation module 501, a writing module 502, and a recovery module 503; the functions of each module are described in detail below:

[0114] The generation module 501 is used to write data to local memory nodes, and to generate and store record information for recording the write operation based on the corresponding write operation.

[0115] The write module 502 is used to write the write data in the local memory node to the remote memory node.

[0116] The generation module 501 is used to generate and store the first positioning information, which is used to locate the data that has been written to the remote memory node.

[0117] The recovery module 503 is used to recover write data in the local memory node that has not been written to the remote memory node, based on the first location information and the record information, in the event of a computing node failure.

[0118] The computing nodes are located locally.

[0119] Specifically, the generation module 501 includes: a modification unit, used to receive data modification requests and modify the corresponding data in the corresponding memory page of the local memory node according to the modification requests; and a generation unit, used to generate a modification operation log for the modification operation of the corresponding data and store the modification operation log.

[0120] Specifically, the generation unit is used to store the modification operation logs in the log cache, and then store the modification operation logs in the storage device through the log cache.

[0121] Specifically, the write module 502 includes: a write unit, used to write the corresponding memory page in the local memory node where the write data is located to a local linked list; and a first write unit, used to write the corresponding memory page to a remote memory node through the local linked list.

[0122] Specifically, the generation module 501 includes: a determining unit, used to determine first positioning information based on memory pages already written to a remote memory node; and a first storage unit, used to store the first positioning information in the remote memory node.

[0123] Specifically, the recovery module 503 includes: a reading unit, used to read the first location information in the remote memory node and read the modification operation log in the storage device; and a recovery unit, used to determine the unwritten data based on the first location information and to recover the unwritten data in the corresponding memory page in the local memory node based on the modification operation log, so that the recovered write data is ready to be written to the remote memory node.

[0124] In addition, the device 500 also includes: a storage module for storing the written data in the remote memory node to a storage device; a generation module 501 for generating and storing second location information, the second location information being used to locate the written data that has been stored in the storage device; and a recovery module 503 for recovering the written data in the remote memory node that has not been written to the storage device, based on the recorded information and the second location information, in the event of an anomaly in the remote memory node.

[0125] Specifically, the storage module includes: a second writing unit, used to write the corresponding memory page in the remote memory node where the write data is located to a remote linked list; and a second storage unit, used to store the corresponding memory page to a storage device through the remote linked list.

[0126] Specifically, the determining unit is used to determine the second location information based on the memory pages already written into the storage device; the first storage unit is used to store the second location information into the storage device.

[0127] Specifically, the read unit is used to read the modification operation log and the second location information in the storage device; the recovery unit is used to determine the unwritten data according to the second location information, and to recover the unwritten data in the corresponding memory page in the local memory node according to the modification operation log, so that the recovered write data is ready to be stored in the storage device.

[0128] For details not covered in this device 500, please refer to the previous text; further elaboration will not be repeated here.

[0129] The above describes Figure 5 The internal functions and structure of the device 500 shown, in one possible design, Figure 5 The structure of the illustrated device 500 can be implemented as a computing device, such as a server. Figure 6 As shown, the device 600 may include: a memory 601 and a processor 602;

[0130] Memory 601 is used to store computer programs.

[0131] The processor 602 is configured to execute a computer program for: writing data to a local memory node, corresponding to the write operation; generating and storing record information for recording the write operation; generating and storing first location information for locating the data written to the remote memory node; and, in the event of a computing node malfunction, recovering the data written to the local memory node that has not been written to the remote memory node based on the first location information and the record information.

[0132] Specifically, processor 602 is used to: receive data modification requests, modify the corresponding data in the corresponding memory page of the local memory node according to the modification requests; generate modification operation logs for the modification operations of the corresponding data, and store the modification operation logs.

[0133] Specifically, processor 602 is used to: store modification operation logs in a log cache, and then store the modification operation logs in a storage device through the log cache.

[0134] Specifically, processor 602 is used to: write the corresponding memory page in the local memory node where the data is located to the local linked list; and write the corresponding memory page to the remote memory node through the local linked list.

[0135] Specifically, processor 602 is used to: determine first location information based on memory pages already written to a remote memory node; and store the first location information in the remote memory node.

[0136] Specifically, the processor 602 is used to: read the first location information in the remote memory node and read the modification operation log in the storage device; determine the unwritten write data according to the first location information, and restore the unwritten data in the corresponding memory page in the local memory node according to the modification operation log, so that the restored write data is waiting to be written to the remote memory node.

[0137] In addition, the processor 602 is also used to: store the written data in the remote memory node to the storage device; generate and store second location information, the second location information being used to locate the written data that has been stored in the storage device; and, in the event of an anomaly in the remote memory node, recover the written data in the remote memory node that has not been written to the storage device based on the record information and the second location information.

[0138] Specifically, processor 602 is used to: write the corresponding memory page in the remote memory node where the data to be written is located to a remote linked list; and store the corresponding memory page to a storage device through the remote linked list.

[0139] Specifically, processor 602 is used to: determine second location information based on memory pages already written to the storage device; and store the second location information in the storage device.

[0140] Specifically, processor 602 is used to: read the modification operation log and the second location information in the storage device; determine the unwritten write data according to the second location information, and restore the unwritten data in the corresponding memory page in the local memory node according to the modification operation log, so that the restored write data is ready to be stored in the storage device.

[0141] In addition, embodiments of the present invention provide a computer storage medium in which a computer program, when executed by one or more processors, causes one or more processors to implement... Figure 1-2 The steps of a data recovery method are described in the method embodiment.

[0142] In addition, embodiments of the present invention provide a computer program product, including a computer program or instructions, wherein when the computer program is executed by a processor, it causes the processor to implement... Figure 1-2 The steps of a data recovery method are described in the method embodiment.

[0143] Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations appearing in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 101, 102, 103, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0144] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0145] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0146] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable multimedia data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable multimedia data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0147] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable multimedia data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0148] These computer program instructions may also be loaded onto a computer or other programmable multimedia data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0149] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0150] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0151] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data recovery method applied to a database memory-separated architecture, comprising: Receive a data modification request, and modify the corresponding data in the corresponding memory page of the local memory node according to the modification request; For each modification operation on the corresponding data, a modification operation log is generated and stored. Write the data in the local memory node to the remote memory node; The local memory node and the remote memory node are deployed on different physical hosts; Based on the memory pages already written to the remote memory node, first location information is determined, and the first location information is stored in the remote memory node. The first location information is used to locate the data already written to the remote memory node. In the event of an anomaly in the local compute node deployed on the same physical host as the local memory node, the first location information in the remote memory node and the modified operation log stored therein are read. The unwritten write data is determined based on the first location information, and the unwritten write data is restored in the corresponding memory page of the local memory node according to the modification operation log, so that the restored write data is ready to be written to the remote memory node. Data written to remote memory nodes is stored on storage devices; Generate and store second positioning information, which is used to locate data written to the storage device; In the event of an abnormal remote memory node, read the modification operation log from the storage device and the second location information; The unwritten data is determined based on the second location information, and the unwritten data is restored in the corresponding memory page of the local memory node according to the modification operation log, so that the restored data is ready to be stored in the storage device.

2. The method according to claim 1, wherein storing the modification operation log includes: The modification operation logs are stored in the log cache, and then stored in the storage device through the log cache.

3. The method according to claim 1, wherein writing the data to the remote memory node from the local memory node comprises: Write the corresponding memory page in the local memory node where the data is located to the local linked list; The corresponding memory page is written to the remote memory node through the local linked list.

4. The method according to claim 1, wherein storing the write data in the remote memory node to the storage device comprises: Write the corresponding memory page in the remote memory node where the data is located to the remote linked list; The corresponding memory pages are stored to the storage device using the remote linked list.

5. The method according to claim 1, wherein generating and storing the second location information comprises: The second location information is determined based on the memory pages already written to the storage device; The second location information is stored in the storage device.

6. A computing device, comprising: Memory, processor; The memory is used to store computer programs; The processor executes the computer program for: In a database memory separation architecture scenario, a data modification request is received, and the corresponding data in the corresponding memory page of the local memory node is modified according to the modification request; for the modification operation of the corresponding data, a modification operation log is generated and stored. Write the data in the local memory node to the remote memory node; The local memory node and the remote memory node are deployed on different physical hosts; Based on the memory pages already written to the remote memory node, first location information is determined, and the first location information is stored in the remote memory node. The first location information is used to locate the data already written to the remote memory node. In the event of an anomaly in the local compute node deployed on the same physical host as the local memory node, the first location information in the remote memory node and the modified operation log stored therein are read. The unwritten write data is determined based on the first location information, and the unwritten write data is restored in the corresponding memory page of the local memory node according to the modification operation log, so that the restored write data is ready to be written to the remote memory node. Data written to remote memory nodes is stored on storage devices; Generate and store second positioning information, which is used to locate data written to the storage device; In the event of an abnormal remote memory node, read the modification operation log from the storage device and the second location information; The unwritten data is determined based on the second location information, and the unwritten data is restored in the corresponding memory page of the local memory node according to the modification operation log, so that the restored data is ready to be stored in the storage device.

7. A computer-readable storage medium storing a computer program, which, when executed by one or more processors, causes the one or more processors to perform the steps of the method according to any one of claims 1-5.

8. A computer program product comprising a computer program or instructions, wherein, When a computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 1-5.

9. A data recovery system, comprising: The first device and the second device; The first device sends a data write request to the second device; In a database memory separation architecture scenario, the second device receives a data modification request, modifies the corresponding data in the corresponding memory page of the local memory node according to the modification request, generates a modification operation log for the modification operation of the corresponding data, and stores the modification operation log. Write the data in the local memory node to the remote memory node; The local memory node and the remote memory node are deployed on different physical hosts; based on the memory pages that have been written to the remote memory node, first location information is determined and stored in the remote memory node. The first location information is used to locate the data that has been written to the remote memory node. In the event of an anomaly in the local compute node deployed on the same physical host as the local memory node, the first location information in the remote memory node and the modified operation log stored therein are read. The unwritten write data is determined based on the first location information, and the unwritten write data is restored in the corresponding memory page of the local memory node according to the modification operation log, so that the restored write data is ready to be written to the remote memory node. Data written to remote memory nodes is stored on storage devices; Generate and store second positioning information, which is used to locate data written to the storage device; In the event of an abnormal remote memory node, read the modification operation log from the storage device and the second location information; The unwritten data is determined based on the second location information, and the unwritten data is restored in the corresponding memory page of the local memory node according to the modification operation log, so that the restored data is ready to be stored in the storage device.

Images