Method and device for ensuring persistent memory data crash consistency

By removing log operations in a persistent memory system and adopting a log-free crash recovery process, using the original verification data of the encoded stripe for data recovery, the performance degradation caused by crash consistency in the existing technology is solved, and the writing is significantly improved Performance and overall performance.

CN119025029AActive Publication Date: 2024-11-26HUAZHONG UNIV OF SCI & TECH

Patent Information

Application Number
CN202411033596.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-26
Estimated Expiration
2044-07-30

AI Technical Summary

Technical Problem

In the prior art, persistent memory system crash consistency is achieved through the logging mechanism, but it significantly increases the write operation overhead, resulting in a degradation of system performance.

Method used

Removing log operations within the system, adopting a log-free crash recovery process, scanning the encoded strips of memory space, using the original verification data to recover data, and replacing the log mechanism to ensure data consistency.

Benefits of technology

It effectively enhances the write performance of persistent memory systems, improves the system's latency and throughput, and improves overall performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025029A_ABST
    Figure CN119025029A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of computer storage, and particularly discloses a method and device for ensuring persistent memory data crash consistency. The method comprises the following steps: removing a log operation used for ensuring the consistency of memory data crash in a system; after the system crashes and is restarted, executing a log-free crash recovery process; the log-free crash recovery process is used for sequentially scanning each coding strip stored in a memory space so as to perform data recovery by using the stored original verification data of the coding strips under the condition that data crash inconsistency occurs in the coding strips. Through the method, the writing performance of the persistent memory system can be effectively enhanced, the delay and throughput of the system are remarkably improved, and the overall performance of the system is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of computer storage technology, and more specifically, to a method and device for ensuring crash consistency of persistent memory data. Background Art

[0002] Persistent Memory (PM) is a promising new memory technology that aims to provide DRAM-like performance and disk-like capacity for persistent storage systems. System applications can directly execute load / store instructions on byte-addressable persistent memory. Although persistent memory provides data persistence, the correctness of the data will be severely affected if an unexpected crash occurs, resulting in data inconsistency.

[0003] In the prior art, in order to solve the above technical defects, the traditional crash consistency method is to introduce a log mechanism to record the historical data of memory operations in the persistent memory system, especially write operations, and ensure that the data is restored to a consistent state after a system crash through functions such as log rollback. However, this log mechanism significantly increases the number of write operations, which limits the system's write performance and leads to a decrease in system performance. Summary of the invention

[0004] In view of the defects of the prior art, the purpose of this application is to enhance the write performance of the persistent memory system and improve the system performance. It aims to solve the problem that the log mechanism that ensures system crash consistency in the prior art significantly increases the write operations, which limits the system write performance and thus leads to a decrease in system performance.

[0005] To achieve the above objectives, in a first aspect, the present application provides a method for ensuring crash consistency of persistent memory data, comprising:

[0006] Remove the log operation used to ensure the consistency of memory data crash in the system;

[0007] After the system crashes and restarts, a logless crash recovery process is executed;

[0008] The logless crash recovery process is used to sequentially scan each encoding stripe stored in the memory space, so as to perform data recovery using the original verification data of the stored encoding stripe when data crash inconsistency occurs in the encoding stripe.

[0009] Optionally, the original check data includes an original checksum of an original check block and an original data block; and the specific steps of scanning the coded strips stored in the memory space include:

[0010] Reading the encoded strip into a memory buffer;

[0011] In the memory buffer, traverse all original data blocks in the coding stripe to calculate a new checksum of each original data block;

[0012] Matching each of the new checksums with the corresponding original checksums, and determining the number of data block columns with inconsistent data in the coding stripe based on the matching result;

[0013] When the number of the data block columns is greater than zero and not greater than the number of the check block columns in the coding stripe, decoding each original check block in the check block column to obtain a plurality of new data blocks;

[0014] Data recovery is performed on the coded strip based on the multiple new data blocks and the original checksum of each of the original data blocks.

[0015] Optionally, the performing data recovery on the coded stripe based on the multiple new data blocks and original checksums of the original data blocks includes:

[0016] Determining a checksum for each of the new data blocks;

[0017] Matching the checksum of each of the new data blocks with each of the original checksums;

[0018] In the case where there is a successfully matched new data block, the successfully matched new data block replaces the corresponding original data block in the coding strip;

[0019] Encoding the successfully matched new data block to obtain a corresponding new check block;

[0020] In the case that the new check block does not match the original check block, the new check block is used to replace the corresponding original check block in the coding stripe to complete data recovery of the coding stripe.

[0021] Optionally, after matching each of the new checksums with the corresponding original checksums and determining the number of data block columns with inconsistent data in the coding stripe based on the matching result, the method further includes:

[0022] When the number of data block columns with inconsistent data in the coding stripe is zero, determining a new check block based on all original data blocks in the coding stripe;

[0023] In the case that the new check block does not match the original check block, the new check block is used to replace the corresponding original check block in the coding stripe to complete data recovery of the coding stripe.

[0024] Optionally, after matching each of the new checksums with the corresponding original checksums and determining the number of data block columns with inconsistent data in the coding stripe based on the matching result, the method further includes:

[0025] In the case where the number of data block columns is greater than the number of check block columns in the coding stripe, a warning message is popped up;

[0026] A next coding stripe of the coding stripe is scanned to recover data of the next coding stripe.

[0027] Optionally, after removing the log operation for ensuring the consistency of memory data crash, the method further includes:

[0028] After receiving a data block update request for the coding stripe, cache the data block update request and update data of at least one target data block in a memory buffer; the target data block is a data block in the coding stripe that needs to be updated by the data block update request;

[0029] In response to the data block update request, in the memory buffer, a data block difference value corresponding to each of the target data blocks is calculated according to a target byte cache line granularity, and a corresponding check block difference value is determined based on each of the data block difference values;

[0030] Performing a first operation of asynchronously writing updated data of a data block, so as to asynchronously write updated data of each target data block to a corresponding memory address at the target byte cache line granularity in the order of data block addresses;

[0031] A first memory barrier instruction is called to make the first operation persistent.

[0032] Optionally, after calling the first memory barrier instruction to persist the first operation, the method further includes:

[0033] Determine a corresponding new check block based on each check block difference according to the target byte cache line granularity;

[0034] Performing a second operation of asynchronously writing updated data of the check block, so as to asynchronously write the data of each of the new check blocks to the corresponding memory address according to the target byte cache line granularity;

[0035] A second memory barrier instruction is called to make the second operation permanent.

[0036] In a second aspect, the present application provides a device for ensuring crash consistency of persistent memory data, including:

[0037] The log removal module is used to remove log operations in the system to ensure the consistency of memory data crashes;

[0038] A data recovery module, used to perform a logless crash recovery process after the system crashes and restarts;

[0039] The logless crash recovery process is used to sequentially scan each encoding stripe stored in the memory space, so as to perform data recovery using the original verification data of the stored encoding stripe when data crash inconsistency occurs in the encoding stripe.

[0040] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0041] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0042] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0043] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art:

[0044] The present application provides a method and device for ensuring crash consistency of persistent memory data. By introducing a log-free crash recovery process, the log operation that ensures crash consistency of memory data in the system is removed. After the system crashes and restarts, the log-free crash recovery process is executed. The process will scan each coding strip stored in the memory space in turn, so that when data crash inconsistency occurs in the coding strip, the original verification data of the stored coding strip is used to recover the data. In this way, the system completely uses verification data to replace logs, and reuses multiple reliability redundant data already in the existing system to ensure data crash consistency. This can effectively enhance the write performance of the persistent memory system, significantly improve the system's latency and throughput, and improve the overall performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is one of the flow charts of the method for ensuring crash consistency of persistent memory data provided by an embodiment of the present application;

[0046] Figure 2This is the second flow chart of the method for ensuring crash consistency of persistent memory data provided by an embodiment of the present application;

[0047] Figure 3 This is one of the flowcharts of data update in the method for ensuring crash consistency of persistent memory data provided by an embodiment of the present application;

[0048] Figure 4 This is the second flowchart of data update in the method for ensuring persistent memory data crash consistency provided by an embodiment of the present application;

[0049] Figure 5 It is a structural diagram of a device for ensuring crash consistency of persistent memory data provided by an embodiment of the present application;

[0050] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of objects. For example, a first memory barrier instruction and a second memory barrier instruction are memory barrier instructions used to distinguish different functions rather than to describe a specific order of memory barrier instructions.

[0053] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0054] In the description of the embodiments of the present application, unless otherwise specified, “multiple” means two or more than two. For example, multiple new data blocks refer to two or more new data blocks, etc.

[0055] For ease of understanding, the relevant technical terms involved in the embodiments of the present application are explained and described below.

[0056] (1) Coding strip

[0057] The original stored data is divided into several data blocks, and then one or several check blocks are generated by performing specific encoding on these data blocks. These data blocks and the check blocks form a stripe, which can also be described as a coded stripe.

[0058] Among them, data blocks are actual data units in the storage or transmission process. They contain specific data information that users need to save or transmit. Each data block contains data of a certain size; check blocks are redundant data used to verify the integrity of data blocks. They are usually calculated on data blocks through certain algorithms (such as cyclic redundancy check, etc.).

[0059] (2) Crash consistency

[0060] Crash consistency is a very important concept in file systems, database systems, and other systems that require persistent data storage. It means that after a system crash, power outage, or other unforeseen failure, the system can ensure data consistency and integrity, and can recover to a legal and operational state, ensuring that data will not be damaged, lost, or inconsistent after the system is restarted or restored.

[0061] (3) Memory barrier instructions

[0062] Memory barrier instructions can also be called memory fences, memory barriers, memory barriers, etc. They are a type of synchronization barrier instructions. They are a synchronization point in the CPU or compiler's random memory access operations, so that all read and write operations before this point can be executed before the operations after this point can be started.

[0063] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0064] Figure 1 This is one of the flow charts of the method for ensuring crash consistency of persistent memory data provided by the embodiment of the present application, such as Figure 1 As shown, it includes: step 110 and step 120.

[0065] Wherein, step 110 is to remove the log operation in the system for ensuring the consistency of memory data crash;

[0066] Step 120, after the system crashes and restarts, execute a logless crash recovery process;

[0067] The logless crash recovery process is used to sequentially scan each encoding stripe stored in the memory space, so as to recover data using the original check data of the stored encoding stripe when data crash inconsistency occurs in the encoding stripe.

[0068] It can be understood that the system described in the embodiment of the present application is a persistent memory storage system.

[0069] Specifically, the log operation described in the embodiment of the present invention refers to the log operation in the original system used to ensure the crash consistency of memory data, mainly including log addition and deletion operations and log write operations in the data update process.

[0070] The log-free crash recovery process described in the embodiment of the present application is a program flow for restoring data to a consistent state after a system crash when the log mechanism is completely removed.

[0071] In an embodiment of the present application, in step 110, the log operations in the system used to ensure the crash consistency of memory data are first removed, that is, the log addition and deletion operations of the data update process in the original system, as well as the log write operations, are removed to reduce log overhead.

[0072] It should be noted that directly removing the log will affect the correctness of the data. In order not to affect the reliability and crash consistency of the system in the absence of logs, a log-free crash recovery process is introduced in the embodiment of the present application.

[0073] Further, in an embodiment of the present application, in step 120, after the system crashes and restarts, the logless crash recovery process is executed, under which each coding strip stored in the memory space is scanned in sequence according to the storage address of the memory space. During the scanning process, if data crash inconsistency is found in a certain coding strip, the original check data of the coding strip can be used to decode and calculate the parity check and combine with other consistent data in the coding strip to restore the inconsistent data in the coding strip to its previous consistent state, thereby realizing data crash consistency recovery.

[0074] The method for ensuring the crash consistency of persistent memory data in the embodiment of the present application introduces a log-free crash recovery process, removes the log operation that ensures the crash consistency of memory data in the system, and executes the log-free crash recovery process after the system crashes and restarts. The process will scan each coding strip stored in the memory space in turn, so that when data crash inconsistency occurs in the coding strip, the original verification data of the stored coding strip is used to recover the data. In this way, the system completely uses verification data to replace logs, and reuses multiple reliability redundant data already in the existing system to ensure data crash consistency, which can effectively enhance the write performance of the persistent memory system, significantly improve the system's latency and throughput, and improve the overall performance of the system

[0075] Based on the content of the above embodiment, as an optional embodiment, the original check data includes the original checksum of the original check block and the original data block; the specific steps of scanning the coded strips stored in the memory space include:

[0076] Read the encoded strip into the memory buffer;

[0077] In the memory buffer, all the original data blocks in the encoded stripe are traversed to calculate a new checksum of each original data block;

[0078] Matching each new checksum with the corresponding original checksum, and determining the number of data block columns in the encoding stripe having inconsistent data based on the matching result;

[0079] When the number of data block columns is greater than zero and not greater than the number of check block columns in the coding stripe, decoding each original check block in the check block column to obtain a plurality of new data blocks;

[0080] Data recovery is performed on the coded strip based on the multiple new data blocks and the original checksum of each original data block.

[0081] It should be noted that a data block column refers to a column storage data composed of one or more data blocks in a coding stripe, and each data block has its corresponding stored data block column. Similarly, a check block column is also a column storage data composed of one or more check blocks. In addition, for a single check block storage system, such as a storage system based on a redundant array of independent disks (RAID), its coding stripe contains multiple data blocks and one check block, so its coding stripe contains multiple columns of data blocks and a single column of check block; and for a multi-check block storage system, such as a storage system based on erasure coding technology, its coding stripe can contain multiple data blocks and multiple check blocks, so its coding stripe can contain multiple columns of data blocks and multiple columns of check block.

[0082] Specifically, in an embodiment of the present application, in order to ensure that the system is not unable to obtain location information of potential crash inconsistencies due to lack of log records, an optimistic speculative recovery technique is proposed.

[0083] More specifically, during the execution of the logless crash recovery process, starting from the logical head of the configured full address space, each coded stripe stored in the memory space is scanned in sequence, with a coded stripe (a series of data blocks and corresponding check blocks) as a unit. During the scanning process, for any coded stripe, this process first reads the coded stripe into the memory buffer, and then traverses all the original data blocks in the coded stripe in the memory buffer to calculate whether the checksum of its internal persistent record matches the data block content.

[0084] Specifically, according to each original data block, according to the calculation method of its original checksum, such as using bit-wise XOR operation, cumulative summation or cyclic redundancy check, the original data block is operated to calculate the new checksum of each original data block.

[0085] Furthermore, in an embodiment of the present application, the new checksum of each original data block is matched with the original checksum stored corresponding to each original data block to determine whether the two are consistent, and based on the matching result of each original data block, it can be determined which data blocks have data inconsistencies, and then the number of data block columns with inconsistent data in the coding stripe can be determined based on the data block column where the data blocks with inconsistent data are located. Then, based on the size relationship between the number of data block columns and the number of check block columns in the coding stripe, it is determined whether the data inconsistency of the data blocks exceeds the repair capability.

[0086] Furthermore, in an embodiment of the present application, if it is determined that the number of inconsistent data block columns is greater than zero and not greater than the number of check block columns in the coding stripe, it indicates that there is data block crash inconsistency, and the check data can be used for data recovery, and the inconsistency of the system data blocks does not exceed the repair capability. At this time, each original check block in the check block column can be further decoded and calculated using different paths, wherein each check block corresponds to a decoding calculation path, and the corresponding data block of the decoding calculation of each original check block is obtained, thereby obtaining multiple new data blocks.

[0087] In an embodiment of the present application, the multiple new data blocks obtained by the above decoding and the original checksum of each original data block in the check data are used to perform re-encoding calculation to obtain a new check block, thereby completing the data recovery process for the coded stripe.

[0088] The method of the embodiment of the present application, by fully reusing the check blocks and checksums in the system and utilizing scanning recovery, completely replaces the functions of traditional logs, effectively enhances the write performance of the system, and at the same time, can detect all crash inconsistencies and repair inconsistent data blocks within its repair capability, thereby improving the overall performance of the system.

[0089] Based on the content of the above embodiment, as an optional embodiment, data recovery is performed on the coded strip based on the original checksum of the multiple new data blocks and each original data block, including:

[0090] Determine the checksum of each new data block;

[0091] Matching the checksum of each new data block with each original checksum;

[0092] If there is a successfully matched new data block, the successfully matched new data block replaces the corresponding original data block in the coding strip;

[0093] Encode the new data block that matches successfully to obtain the corresponding new check block;

[0094] In the case that the new check block does not match the original check block, the new check block is used to replace the corresponding original check block in the coding stripe to complete the data recovery of the coding stripe.

[0095] Specifically, in an embodiment of the present application, after a plurality of new data blocks are obtained by decoding and calculating each original check block, each new data block can be processed according to the original checksum calculation method, such as by using a bit-wise XOR operation, cumulative summation, or cyclic redundancy check, to obtain a checksum for each new data block.

[0096] Furthermore, in an embodiment of the present application, the checksum of each new data block is matched with each original checksum to determine whether there is a checksum among the new data blocks that is consistent with the original checksum. If there is a new data block that has been successfully matched, it means that the checksum of the new data block is consistent with the original checksum. At this time, the new data block that has been successfully matched replaces the corresponding original data block in the coding strip according to the corresponding spatial address, thereby realizing the recovery of the data block data in the coding strip.

[0097] In an embodiment of the present application, after comparing and matching the checksum of each new data block with each original checksum, if there is no new data block that has been successfully matched, that is, the checksum of each new data block is different from each original checksum, and there is no new data block that has passed the check, then this indicates that the total number of crash inconsistencies on the data blocks and check blocks exceeds the repair capability, and a warning message will pop up.

[0098] Furthermore, in the embodiment of the present application, the system will continue to scan the next coding strip of the coding strip, and perform data recovery or throw a warning for the next coding strip in the same manner as described above. Thus, by judging whether there is mutually matching and consistent data in the checksum of each new data block and each original checksum, if not, it is determined that the total number of crash inconsistencies on the data blocks and check blocks in the system coding strip exceeds the repair capability, and a warning message is promptly popped up to the front desk to further remind the user of the recovery status of the memory data, so that the user can respond in time and improve the user experience.

[0099] Furthermore, in an embodiment of the present application, a new data block that has been successfully matched is encoded to obtain a corresponding new check block. At this time, the new check block is compared and matched with the original check block. If there is a mismatch, the new check block can be used to replace the corresponding original check block in the coding strip, thereby realizing the recovery of the check block data in the coding strip, and finally completing the data recovery of the coding strip. If each new check block matches each original check block, it means that the original check block is correct data, and there is no need to recover the data of the original check block.

[0100] The method of the embodiment of the present application compares and matches the original checksum with the checksum of the new data block to determine whether the new data block is a correct data block, and then after a successful match, a re-encoding process is performed on the successfully matched new data block to restore the check block data, and finally complete the data recovery of the entire coding strip, which can effectively achieve data crash consistency under log-free conditions.

[0101] Based on the content of the above embodiment, as an optional embodiment, after matching each new checksum with the corresponding original checksum and determining the number of data block columns with inconsistent data in the coding stripe based on the matching result, the method further includes:

[0102] When the number of data block columns with inconsistent data in the coding stripe is zero, a new check block is determined based on all original data blocks in the coding stripe;

[0103] In the case that the new check block does not match the original check block, the new check block is used to replace the corresponding original check block in the coding stripe to complete the data recovery of the coding stripe.

[0104] Specifically, in an embodiment of the present application, when it is determined that the number of data block columns with inconsistent data in a coding strip is zero, it can be explained that there is no data inconsistency in the data blocks in the coding strip, and the original data blocks are all correct data. At this time, there is no need to perform data recovery on the data of each original data block.

[0105] Furthermore, in an embodiment of the present application, to determine whether data inconsistency occurs in the original check block, all original data blocks in the coding stripe may be re-encoded and calculated to determine a new check block, and then the new check block may be compared and matched with the corresponding original check block. If the new check block does not match the original check block, it indicates that the data of the original check block is inaccurate. At this time, the corresponding original check block in the coding stripe may be replaced with the new check block to complete data recovery of the entire coding stripe.

[0106] The method of the embodiment of the present application determines that there is no data inconsistency in the original data block, compares and matches the original check block with the new check block obtained by re-encoding the original data block, and in the case of mismatch, uses the new check block to restore the check block data, and finally completes the data recovery of the entire coding strip, which can effectively achieve data crash consistency under log-free conditions.

[0107] Based on the content of the above embodiment, as an optional embodiment, after matching each new checksum with the corresponding original checksum and determining the number of data block columns with inconsistent data in the coding stripe based on the matching result, the method further includes:

[0108] If the number of data block columns is greater than the number of check block columns in the encoding stripe, a warning message will pop up;

[0109] The next encoding stripe of the encoding stripe is scanned to recover data of the next encoding stripe.

[0110] Specifically, in an embodiment of the present application, after determining the number of data block columns with inconsistent data in the coding stripe, if it is determined that the number of data block columns is greater than the number of check block columns in the coding stripe, it means that the number of data blocks with inconsistent data is too large, and the inconsistency of system data blocks exceeds the repair capability. At this time, a warning message will pop up for the user to view, informing the user that the coding stripe repair has failed.

[0111] Furthermore, in an embodiment of the present application, the system will continue to scan the next coding stripe of the coding stripe, and perform data recovery or issue a warning for the next coding stripe in the same manner as described above.

[0112] The method of the embodiment of the present application determines that the inconsistency of the system data blocks exceeds the repair capability by judging that the number of data block columns is greater than the number of check block columns in the coding stripe, and promptly pops up a warning message to the foreground to remind the user of the recovery status of the memory data, so as to facilitate the user to respond in time and improve the user experience.

[0113] Figure 2This is a second flow chart of a method for ensuring crash consistency of persistent memory data provided by an embodiment of the present application, such as Figure 2 As shown, in an embodiment of the present application, when the system unexpectedly crashes and restarts, the system enters a logless crash recovery process, which includes the following steps:

[0114] First, starting from the logical head of the configured full address space, a coding stripe (a series of data blocks and corresponding check blocks) is scanned as a unit, and a complete coding stripe is read into the memory buffer.

[0115] Then, determine whether the original checksum matches the data block: traverse all the original data blocks in the encoded stripe and calculate whether the original checksum of the internal persistent record matches the checksum recalculated from the original data block content. If they do not match, it means that there is a crash inconsistency on the original data block. Otherwise, it means that there is no crash inconsistency on the original data block.

[0116] Next, after scanning all the original data blocks of the coding stripe, if there is no crash inconsistency, that is, the number of data block columns with inconsistent data is zero, then enter the following re-encoding process; if the number of crash-inconsistent data block columns is greater than the number of check block columns, it means that an error that exceeds the repair capability has occurred. After a warning is thrown, determine whether all coding strips have been scanned. If not, continue to scan the next coding stripe in the above manner; if the number of crash-inconsistent data block columns is greater than zero and not greater than the number of check block columns, it means that the data block has crash inconsistency and is within the system's repair capability, then enter the following repair process.

[0117] Next, the re-encoding process is performed: the correct data block data is re-encoded to obtain a new check block, and the new check block is compared with the original check block to determine whether it matches. If it does not match, it means that there is a check block with inconsistent data, and the correct new check block is written back. After the operation is completed, after determining that all encoding strips have not been scanned, continue to scan the next encoding strip in the same way as above.

[0118] Then, the repair process is executed: multiple new data blocks are obtained through multi-path decoding calculations of different paths (M check blocks correspond to M decoding calculation paths), and the checksum inside the new data blocks obtained after decoding is verified with the original checksum. If any of them pass the verification, the recovery is successful, and the new data block is written back, and then the above re-encoding process is entered; if no verification passes, it means that the total number of crash inconsistencies on the data block and the check block exceeds the repair capability, and a warning message is thrown. After confirming that all the coding strips have not been scanned, continue to scan the next coding strip in the above manner.

[0119] In an embodiment of the present application, after removing the log operation used to ensure the crash consistency of memory data, since the log addition and deletion operations in the update process are removed, in order to avoid the system reliability being affected by the lack of logs, a log-free crash consistency update process is also provided.

[0120] Based on the content of the above embodiment, as an optional embodiment, after removing the log operation for ensuring the consistency of memory data crash, the method further includes:

[0121] After receiving a data block update request for a coding stripe, the data block update request and update data of at least one target data block are cached in a memory buffer; the target data block is a data block in the coding stripe that needs to be updated by the data block update request;

[0122] In response to the data block update request, in the memory buffer, the data block difference corresponding to each target data block is calculated according to the target byte cache line granularity, and the corresponding check block difference is determined based on each data block difference;

[0123] Performing a first operation of asynchronously writing updated data of a data block, so as to asynchronously write updated data of each target data block to a corresponding memory address in a target byte cache line granularity according to the data block address sequence;

[0124] A first memory barrier instruction is called to make the first operation persistent.

[0125] Specifically, the data block update request described in the embodiment of the present application refers to a data update request initiated by a user at the foreground, or a data update request initiated by a periodic data update instruction set within the system.

[0126] The target byte cache line granularity described in the embodiment of the present application refers to the smallest unit of data transmitted between the cache and the memory during the data update process. It can be specifically determined according to the cache line granularity of the computer architecture. For example, the cache line granularity of the X86 computer architecture is 64 bytes, and the target byte cache line granularity can be 64 bytes.

[0127] The first operation described in the embodiment of the present application refers to an operation of asynchronously writing updated data into a data block.

[0128] The first memory barrier instruction described in the embodiment of the present application is used to make the above-mentioned first operation persistent. Through the first memory barrier instruction, the execution command before it is not interchangeable with the execution command after it, ensuring the persistence of the first operation of asynchronously writing the updated data of the data block, thereby ordering the persistent memory write requests executed before and after it, and avoiding potential multi-crash inconsistency problems caused by concurrent updates.

[0129] Figure 3This is one of the flow diagrams of data update in the method for ensuring the crash consistency of persistent memory data provided by the embodiment of the present application, such as Figure 3 As shown, in an embodiment of the present application, after receiving a data block update request for a coding stripe, the object to be updated by the mobile phone is stored in a memory buffer, that is, the data block update request and the update data of at least one target data block to be updated are cached in the memory buffer. Then, in response to the data block update request, the target byte cache line granularity, such as a 64-byte granularity, is traversed and processed, and the update data of each target data block is calculated in the memory buffer and compared with the corresponding original data to obtain the data block difference (delta data) corresponding to each target data block. At the same time, the corresponding check block difference (deltaparity) can be obtained by calculating the finite field of the erasure code using each data block difference.

[0130] Further, in an embodiment of the present application, a first operation of asynchronously writing updated data of a data block is performed, so that updated data of each target data block is asynchronously written to a corresponding memory address step by step in a target byte cache line granularity in accordance with the data block address sequence. That is, according to the asynchronous persistent write mechanism, based on the data block address sequence, the system can sequentially and step by step update the updated data of each target data block to the corresponding memory address one by one in a 64-byte cache line granularity until the update processing of each target data block is completed.

[0131] Furthermore, in an embodiment of the present application, a memory barrier instruction request is initiated, and a first memory barrier instruction is called to persist the operation of asynchronously writing updated data into a data block, so as to ensure the persistence of all new data block asynchronous write requests.

[0132] The method of the embodiment of the present application introduces fine-grained memory barrier technology, and initiates an additional memory barrier instruction request between the continuous update instructions of the data block and the check block at the same memory address offset, so that the memory barrier instruction orders the persistent memory write requests executed before and after it, thereby avoiding potential multi-crash inconsistency problems caused by concurrent updates. At the same time, by parallelizing encoding calculations and asynchronous persistent write requests at the minimum byte cache line granularity, each cache line can be refreshed to the persistent memory as early as possible without waiting for the encoding of the entire data block to be completed, which can further improve the instruction parallelism during the update process.

[0133] Based on the content of the above embodiment, as an optional embodiment, after calling the first memory barrier instruction to persist the first operation, the method further includes:

[0134] According to the target byte cache line granularity, determine the corresponding new check block based on each check block difference;

[0135] Performing a second operation of asynchronously writing updated data of the check block to asynchronously write data of each new check block to a corresponding memory address according to the target byte cache line granularity;

[0136] A second memory barrier instruction is called to make the second operation permanent.

[0137] Specifically, the second operation described in the embodiment of the present application refers to the operation of asynchronously writing updated data into the check block.

[0138] The second memory barrier instruction described in the embodiment of the present application is used to make the above second operation persistent. Similarly, through the second memory barrier instruction, the execution command before it can be made non-interchangeable with the execution command after it, ensuring the persistence of the operation of asynchronously writing the update data of the check block, avoiding the potential multi-crash inconsistency problem caused by concurrent updates.

[0139] Continue to refer to Figure 3 In the embodiment of the present application, during the update of the check block data, the traversal processing is performed at the target byte cache line granularity, and each check block difference and the corresponding original check block data are used to quickly calculate the corresponding new check block, and then the new check block asynchronous persistent write operation is performed, that is, the second operation of asynchronously writing the update data of the check block is performed, so as to asynchronously write the data of each new check block to the corresponding memory address step by step according to the target byte cache line granularity. That is to say, according to the asynchronous persistent write mechanism, based on the check block address sequence, the system can use the check block difference under the corresponding scale to determine the corresponding new check block data at the 64-byte cache line granularity, and then update the new check block data to the corresponding memory address, and so on, and update the calculated update data of each check block to the corresponding memory address one by one, until all check block data are updated.

[0140] Furthermore, in an embodiment of the present application, a memory barrier instruction request is initiated, a second memory barrier instruction is called, and the operation of asynchronously writing updated data to the check block is persisted to ensure the persistence of all new check block asynchronous write requests.

[0141] The method of the embodiment of the present application introduces pipeline encoding and persistent writing technology to ensure that the data blocks and check blocks at the same offset involved in each update process will not have crash inconsistency problems at the same time, thereby improving the reliability and efficiency of data updates in the persistent memory system.

[0142] Figure 4 This is the second flow chart of data update in the method for ensuring the crash consistency of persistent memory data provided by the embodiment of the present application, such as Figure 4 As shown, Figure 4(a) is a schematic diagram of a traditional memory data update process based on a log mechanism, and (b) is a schematic diagram of a crash-consistent data update process without logs in an embodiment of the present application. Figure 4 It can be seen that, taking a single check block as an example, the original traditional technology first uniformly performs encoding calculations and then writes back the calculation results. The pipeline encoding and persistent writing technology of this application advances part of the calculation operations originally performed after the memory barrier to the window before the memory barrier to reduce the blocking effect of the memory barrier and hide the update delay. In addition, the encoding calculations and asynchronous persistent write requests are parallelized at a minimum granularity of 64 bytes, so that each cache line can be refreshed to the persistent memory as early as possible without waiting for the encoding of the entire data block to be completed, greatly improving the update efficiency of memory data.

[0143] In some embodiments, considering that the crash recovery process of the present application requires time-consuming operations such as scanning the global space, which will affect the recovery of the foreground service, a read-time recheck strategy is further used to provide a system instant recovery (Instant Recovery) function in a state where reliability is temporarily degraded. The specific process is as follows:

[0144] Step S1, firstly start multiple independent background threads to perform parallel scanning and repair between coding strips.

[0145] Step S2: When a read request is made to the front desk, the system reads the corresponding data block and checksum, and performs checksum matching in advance to prevent potential crash inconsistency. If a match occurs, the data is returned; if a mismatch occurs, it means that an error has occurred in the current data block, and the system waits for the backend to repair the error.

[0146] Step S3, when there is a write request access on the foreground, the above-mentioned normal log-free crash consistency update process is performed.

[0147] Step S4, after waiting for the global scan of the logless crash recovery process to be completed, the reliability assurance level of the system is restored to the state before the crash.

[0148] It should be noted that in the recovery process of traditional technology, in addition to recording the content of the updated area, the log also records the location of these areas in order to fully detect and recover inconsistencies, while the log-free method of the present application does not record this information and cannot perform normal detection and recovery. Therefore, during the crash recovery process, the present application uses an optimistic speculative recovery method introduced above to effectively scan the entire persistent memory space and restore it to a data consistent state. This method reuses checksums and checksum redundant decoding to achieve comprehensive detection and recovery of potential inconsistencies. In addition, speculative recovery can also achieve instant recovery services through temporary reliability degradation.

[0149] In the update process, because one update process may involve updating the data area and the check area at the same time, it may cause crash inconsistency problems in multiple areas. Therefore, in the update process, this application uses fine-grained memory barrier instructions to impose a stricter write order to prevent concurrent updates to multiple areas in a single update. By separating the concurrent updates of data blocks and check blocks at the same offset value, it is ensured that the repair capability of the check meets the potential inconsistency range. Secondly, in order to address the problem of additional memory barriers introduced in the update to limit concurrent updates, this application proposes pipeline encoding and persistent write technology, which reduces the blocking effect of memory barriers and hides update delays by refreshing and encoding calculations early, so that the CPU can perform foreground calculations while asynchronous persistent write requests in background execution are submitted to persistent memory in parallel, thereby improving the parallelism of instruction execution.

[0150] In addition, this application executes real workloads by building a transactional key-value storage engine on top of the system to further evaluate the effect. Test results on a multi-channel server platform deployed with real persistent memory devices show that this application can reduce the flow of writing to persistent memory by 33-43%, significantly improve the latency and throughput of the system, and increase the throughput of update transactions by up to 3.4 times.

[0151] The following is a description of an apparatus for ensuring crash consistency of persistent memory data provided by the present invention. The apparatus for ensuring crash consistency of persistent memory data described below and the method for ensuring crash consistency of persistent memory data described above can refer to each other.

[0152] Figure 5 is a schematic diagram of the structure of a device for ensuring crash consistency of persistent memory data provided by an embodiment of the present application, such as Figure 5 As shown, including:

[0153] The log removal module 510 is used to remove the log operation in the system to ensure the consistency of memory data crash;

[0154] A data recovery module 520, configured to execute a logless crash recovery process after the system crashes and restarts;

[0155] The logless crash recovery process is used to sequentially scan each encoding stripe stored in the memory space, so as to perform data recovery using the original verification data of the stored encoding stripe when data crash inconsistency occurs in the encoding stripe.

[0156] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0157] The device for ensuring crash consistency of persistent memory data in an embodiment of the present application introduces a log-free crash recovery process, removes the log operation that ensures crash consistency of memory data in the system, and executes the log-free crash recovery process after the system crashes and restarts. The process will scan each coding strip stored in the memory space in turn, so that when data crash inconsistency occurs in the coding strip, the original verification data of the stored coding strip is used to recover the data. In this way, the system completely uses verification data to replace logs, and reuses multiple reliability redundant data already in the existing system to ensure data crash consistency, which can effectively enhance the write performance of the persistent memory system, significantly improve the system's latency and throughput, and improve the overall performance of the system.

[0158] Based on the method in the above embodiment, the embodiment of the present application provides an electronic device, such as Figure 6 As shown, the electronic device may include: a processor (Processor) 610, a communication interface (Communications Interface) 620, a memory (Memory) 630 and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the method in the above embodiment.

[0159] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0160] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0161] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0162] It is understandable that the processor in the embodiment of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0163] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0164] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0165] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0166] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for ensuring crash consistency of persistent memory data, It is characterized in that include: Remove the log operation used to ensure the consistency of memory data crash in the system; After the system crashes and restarts, a logless crash recovery process is executed; The logless crash recovery process is used to sequentially scan each encoding stripe stored in the memory space, so as to perform data recovery using the original verification data of the stored encoding stripe when data crash inconsistency occurs in the encoding stripe.

2. The method for ensuring crash consistency of persistent memory data according to claim 1, It is characterized in that The original check data includes an original check block and an original checksum of an original data block; and the specific steps of scanning the coded strips stored in the memory space include: Reading the encoded strip into a memory buffer; In the memory buffer, traverse all original data blocks in the coding stripe to calculate a new checksum of each original data block; Matching each of the new checksums with the corresponding original checksums, and determining the number of data block columns with inconsistent data in the coding stripe based on the matching result; When the number of the data block columns is greater than zero and not greater than the number of the check block columns in the coding stripe, decoding each original check block in the check block column to obtain a plurality of new data blocks; Data recovery is performed on the coded strip based on the multiple new data blocks and the original checksum of each of the original data blocks.

3. The method for ensuring crash consistency of persistent memory data according to claim 2, It is characterized in that The performing data recovery on the coded stripe based on the multiple new data blocks and the original checksum of each of the original data blocks comprises: Determining a checksum for each of the new data blocks; Matching the checksum of each of the new data blocks with each of the original checksums; In the case where there is a successfully matched new data block, the successfully matched new data block replaces the corresponding original data block in the coding strip; Encoding the successfully matched new data block to obtain a corresponding new check block; In the case that the new check block does not match the original check block, the new check block is used to replace the corresponding original check block in the coding stripe to complete data recovery of the coding stripe.

4. The method for ensuring crash consistency of persistent memory data according to claim 2, It is characterized in that After matching each of the new checksums with the corresponding original checksums and determining the number of data block columns with inconsistent data in the coding stripe based on the matching result, the method further includes: When the number of data block columns with inconsistent data in the coding stripe is zero, determining a new check block based on all original data blocks in the coding stripe; In the case that the new check block does not match the original check block, the new check block is used to replace the corresponding original check block in the coding stripe to complete data recovery of the coding stripe.

5. The method for ensuring crash consistency of persistent memory data according to claim 2, It is characterized in that After matching each of the new checksums with the corresponding original checksums and determining the number of data block columns with inconsistent data in the coding stripe based on the matching result, the method further includes: In the case where the number of data block columns is greater than the number of check block columns in the coding stripe, a warning message is popped up; A next coding stripe of the coding stripe is scanned to recover data of the next coding stripe.

6. The method for ensuring crash consistency of persistent memory data according to any one of claims 1 to 5, It is characterized in that After removing the log operation for ensuring the consistency of memory data crash, the method further includes: After receiving a data block update request for the coding stripe, cache the data block update request and update data of at least one target data block in a memory buffer; the target data block is a data block in the coding stripe that needs to be updated by the data block update request; In response to the data block update request, in the memory buffer, a data block difference value corresponding to each of the target data blocks is calculated according to a target byte cache line granularity, and a corresponding check block difference value is determined based on each of the data block difference values; Performing a first operation of asynchronously writing updated data of a data block, so as to asynchronously write updated data of each target data block to a corresponding memory address at the target byte cache line granularity in the order of data block addresses; A first memory barrier instruction is called to make the first operation persistent.

7. The method for ensuring crash consistency of persistent memory data according to claim 6, It is characterized in that After calling the first memory barrier instruction to persist the first operation, the method further includes: Determine a corresponding new check block based on each check block difference according to the target byte cache line granularity; Performing a second operation of asynchronously writing updated data of the check block, so as to asynchronously write the data of each of the new check blocks to the corresponding memory address according to the target byte cache line granularity; A second memory barrier instruction is called to make the second operation permanent.

8. A device for ensuring crash consistency of persistent memory data, It is characterized in that include: The log removal module is used to remove log operations in the system to ensure the consistency of memory data crashes; A data recovery module, used to perform a logless crash recovery process after the system crashes and restarts; The logless crash recovery process is used to sequentially scan each encoding stripe stored in the memory space, so as to perform data recovery using the original verification data of the stored encoding stripe when data crash inconsistency occurs in the encoding stripe.

9. An electronic device, It is characterized in that include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Asyncrhonous consistent snapshots in persistent memory stores

    CN104854566A

  • Storage system control

    CN110431538A

  • Secure NVM system and data recovery method after crash thereof

    CN111079201A

  • Write-ahead style logging in a persistent memory device

    CN111480149A

  • Method for ensuring data crash consistency in secure nonvolatile memory and processor

    CN112540931A

Cited By

  • Data processing method and device in assembly line, medium and program product

    CN119883382A

  • Data Processing Method, Device, Medium and Program Product in Pipeline

    CN119883382B

  • Data recovery method and device of operating system and storage medium

    CN121433583A

  • Storage-array-based data storage method, and related apparatus

    WO2026170759A1