Data management method and device, electronic equipment, medium and product

Abstract

The invention discloses a data management method and device, electronic equipment, a medium and a product, and relates to the technical field of data processing, the data management method comprises the following steps: adding a generation identifier in an address mapping relation, because the generation identifier added in the address mapping relation can distinguish different data modification versions, the data modification versions can be distinguished by the generation identifier added in the address mapping relation; in this way, only the address mapping relation corresponding to the modified data can be stored instead of the full-amount address mapping relation, so that the technical problem that the address mapping relation occupies too much memory space is solved, and the technical effect of effectively increasing the mapping relation storage number is achieved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to data management methods, devices, electronic devices, media and products. Background Art

[0002] With the improvement of data processing technology and the expansion of data application scenarios, the amount of data is currently growing rapidly. In this context, the complexity of data has increased significantly, bringing great challenges to data management.

[0003] In the related art, after modifying the data, the modified data is stored in a new physical address, and a mapping relationship between the logical address and the physical address is established, and data of different historical versions can be rolled back through the mapping relationship. However, as the amount of data increases and / or the number of modifications increases, the number of mapping relationships increases, which in turn causes the mapping relationships to occupy too much memory space. Summary of the invention

[0004] The present application provides a data management method, device, electronic device, medium and product to at least solve the problem of excessive memory space occupation in the related art.

[0005] The present application provides a data management method, including: receiving a data modification request, the data modification request including original data, an original logical address, an original physical address, and modification information; modifying the original data according to the data modification request through the modification information to obtain modified data, determining a target physical address, and storing the modified data in the target physical address; determining a target generation identifier corresponding to the modified data, creating a target address mapping relationship according to the target generation identifier, the original logical address, and the target physical address, the target generation identifier indicating the generation of the data modification request; creating a target snapshot according to the target generation identifier and the target address mapping relationship, the target snapshot being used to restore the original data.

[0006] The present application also provides a data management device, including: a receiving module, used to receive a data modification request, the data modification request includes original data, original logical address, original physical address, and modification information; a modification module, used to modify the original data according to the data modification request and the modification information, obtain the modified data, determine the target physical address, and store the modified data in the target physical address; a creation module, used to determine the target generation identifier corresponding to the modified data, create a target address mapping relationship according to the target generation identifier, the original logical address, and the target physical address, the target generation identifier represents the generation of the data modification request; a generation module, used to create a target snapshot according to the target generation identifier and the target address mapping relationship, the target snapshot is used to restore the original data.

[0007] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any one of the above-mentioned data management methods when executing the computer program.

[0008] The present application also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned data management methods are implemented.

[0009] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned data management methods when executed by a processor.

[0010] Through the present application, since the generation identifier added in the address mapping relationship can distinguish different data modification versions, it is possible to store only the address mapping relationship corresponding to the modified data instead of storing the full address mapping relationship, thereby solving the technical problem of the address mapping relationship occupying too much memory space and achieving the technical effect of increasing the number of effective mapping relationships stored. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 A schematic diagram of an application scenario of a data management method provided in an embodiment of the present application;

[0013] Figure 2 A flowchart of a data management method provided in an embodiment of the present application;

[0014] Figure 3 A schematic diagram of data modification provided in an embodiment of the present application;

[0015] Figure 4 A flowchart of a data management method provided in an embodiment of the present application;

[0016] Figure 5 A schematic diagram of a time interval matrix table provided in an embodiment of the present application;

[0017] Figure 6 A schematic diagram of the address mapping relationship provided in the embodiment of the present application;

[0018] Figure 7 A flowchart of a data management method provided in an embodiment of the present application;

[0019] Figure 8 A schematic diagram of data rollback provided in an embodiment of the present application;

[0020] Fig. 9 A schematic diagram of data rollback provided in an embodiment of the present application;

[0021] Fig.10 A schematic diagram of the structure of a data management device provided in an embodiment of the present application;

[0022] Fig.11 A schematic diagram of the structure of a data management device provided in an embodiment of the present application;

[0023] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0026] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0027] In the related art, for data management of the storage system, after modifying part of the data in the source volume, a full address mapping relationship corresponding to the data in the source volume is generated, and the full address mapping relationship is stored for data rollback. As the amount of data increases and / or the number of modifications increases, the number of stored address mapping relationships increases rapidly, resulting in excessive memory space being occupied. In order to avoid the impact of too many stored mapping relationships on normal data reading and writing, it is necessary to clean up the address mapping relationships in a timely manner, resulting in the number of address mapping relationships stored at the same time being limited. For example, even if only 1% of the data in the source volume is modified, since the location of the modification cannot be accurately located in the related art, the address mapping relationship corresponding to the full amount of data still needs to be stored, resulting in excessive memory space being occupied.

[0028] This application generates an address mapping relationship based on the physical address involved in the data modification, only stores the address mapping relationship corresponding to the modified data, and distinguishes the address mapping relationship by generation identification. When the data is rolled back, the generation identification can accurately distinguish different data modification records, thereby accurately locating the data that needs to be rolled back, without having to store the full address mapping relationship in the memory, thereby solving the problem of excessive memory space occupation in the related technology.

[0029] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0030] In conjunction with the specific application environment architecture or the specific hardware architecture on which the execution of the data management method depends, the specific application environment architecture or the specific hardware architecture is described herein. Figure 1 , Figure 1 The schematic diagram of the application scenario of the data management method is as follows: the original data is modified to obtain the modified data, and the address mapping relationship corresponding to the original data is stored. The modified data is used for current use, and the address mapping relationship is used to roll back the original data.

[0031] Combined with the scenario example, the scenario of developing new functions, fixing system vulnerabilities, or correcting system errors requires modifying historical data to obtain modified data. In the process of using modified data, if data corruption, system failure, or compatibility problems occur, they can be solved by rolling back the data to historical data.

[0032] Figure 2 A flow chart of a data management method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, an embodiment of the present application provides a data management method, and the method is described in detail as follows:

[0033] S201. Receive a data modification request, where the data modification request includes original data, an original logical address, an original physical address, and modification information.

[0034] The original data is the data before modification. The modification information is the specific modification requirement, such as deleting specified data or adding specified data. The original logical address is the address for accessing the original data, and the original physical address is the address for storing the original data.

[0035] For example, the logical address is a virtual address for accessing data, which provides a unified access interface and shields the underlying physical storage details. Applications read and write data through logical addresses without having to perceive the actual storage location of the data. Logical addresses simplify file system design. The physical address is the actual location of the data on the physical storage medium. The logical address and the physical address are associated through an address mapping relationship. The address mapping relationship can accurately determine the physical address corresponding to each logical address, so that the data stored under the physical address can be accessed through the logical address.

[0036] S202. According to the data modification request, modify the original data by modifying the information to obtain modified data, determine the target physical address, and store the modified data at the target physical address.

[0037] Exemplarily, the original data is modified according to the specific modification requirements in the modification information to obtain the modified data. If the original data is overwritten by the modified data, that is, the original data is deleted, the original data cannot be rolled back when it is needed later. Saving the original data and the modified data at the same time can realize the subsequent data rollback.

[0038] Next, combine Figure 3 Explain the data modification.

[0039] Figure 3 This is a schematic diagram of data modification provided in the embodiment of the present application. Figure 3 As shown, the original data is stored at the original physical address, and the original data at the original physical address can be accessed through the original logical address. The original data is modified to obtain modified data, which is stored at the target physical address without changing the original logical address. After the modification is completed, the modified data at the target physical address is accessed through the original logical address, while the original data is stored at the original physical address, and the original data at the original physical address can be accessed only when performing data rollback.

[0040] S203, determine the target generation identifier corresponding to the modified data, and create a target address mapping relationship according to the target generation identifier, the original logical address, and the target physical address, where the target generation identifier indicates the generation of the data modification request.

[0041] The generation identifier indicates the generation corresponding to each data modification, and the generation identifier is used to distinguish different data modification versions, and each generation identifier corresponds to a data version.

[0042] Optionally, by maintaining a generation identification table, the generation identification table includes the generation identification corresponding to each data modification, so that the specified modification record can be quickly queried. Each time the data is rolled back, the corresponding modification record is queried from the generation identification table according to the specified generation identification.

[0043] Exemplarily, the target address mapping relationship records the updated address mapping relationship, refer to Figure 3 The address mapping relationship before the data is modified is the mapping relationship between the original logical address and the original physical address, and the address mapping relationship after the data is modified is the mapping relationship between the original logical address and the target physical address. By adding a target generation identifier based on the mapping relationship between the original logical address and the target physical address, the target address mapping relationship can be accurately distinguished from other address mapping relationships.

[0044] In combination with the scenario example, when the data is initially created, the generation identifier is set to 000, when the data is modified for the first time, the corresponding generation identifier is set to 001, when the data is modified for the second time, the corresponding generation identifier is set to 002, and so on. The generation identifier can accurately determine the generation of the corresponding data modification.

[0045] Exemplarily, the target address mapping relationship is generated only according to the changed physical address, and is distinguished by a generation identifier to avoid excessive memory space occupation caused by a full update of the address mapping relationship.

[0046] S204: Create a target snapshot according to the mapping relationship between the target generation identifier and the target address.

[0047] The target snapshot is used to restore the original data.

[0048] Among them, a snapshot is a mirror image of the data state at a certain point in time in the storage system. Its core principle is based on metadata version control rather than full data copy. After the snapshot is created, the data state at a certain point in time is "frozen", and subsequent modifications do not affect the snapshot content.

[0049] Exemplarily, the data state in the present application is an address mapping relationship, and the target snapshot is used to record the target generation identifier and the target address mapping relationship. The address mapping relationship can be quickly queried through the snapshot, so that data can be rolled back quickly. By adding the target generation identifier information to the snapshot, multiple snapshots can be accurately distinguished.

[0050] Combined with the scenario example, a target snapshot is generated at time T3 to record the target address mapping relationship at time T3. Subsequent data modifications at time T3 will not change the target snapshot. A new snapshot is generated based on the new data modification at time T3, and the new snapshot coexists with the target snapshot. Multiple snapshots can be rolled back to the corresponding address mapping relationship.

[0051] Based on the above implementation, by adding a target generation identifier to a snapshot, multiple snapshots can be accurately distinguished, so that a specified snapshot can be accurately determined when data is rolled back.

[0052] The data management method provided by the embodiment of the present application receives a data modification request, the data modification request includes original data, original logical address, original physical address, and modification information; according to the data modification request, the original data is modified by the modification information to obtain the modified data, the target physical address is determined, and the modified data is stored in the target physical address; the target generation identifier corresponding to the modified data is determined, and a target address mapping relationship is created according to the target generation identifier, the original logical address, and the target physical address, the target generation identifier represents the generation of the data modification request; according to the target generation identifier and the target address mapping relationship, a target snapshot is created, and the target snapshot is used to restore the original data. In the above scheme, since the generation identifier added in the address mapping can distinguish different data modification versions, it is possible to store only the address mapping relationship corresponding to the modified data instead of storing the full address mapping relationship, thereby solving the technical problem that the address mapping relationship occupies too much memory space and achieving the technical effect of increasing the number of valid mapping relationship storage.

[0053] Based on any of the above embodiments, Figure 4 , the detailed process of data management is explained.

[0054] Figure 4 A flow chart of a data management method provided in an embodiment of the present application. Figure 4 As shown, the method includes:

[0055] S401. Receive a data modification request, where the data modification request includes original data, an original logical address, an original physical address, and modification information.

[0056] It should be noted that the execution process of S401 refers to S201 and will not be repeated here.

[0057] S402: Determine the data size of the modified data.

[0058] Exemplarily, the clear data size is used to accurately determine the number of physical blocks required to store the modified data, thereby accurately determining the target physical address.

[0059] In combination with a scenario example, for example, if the data size is 12KB and the physical block size is 4KB, three physical blocks are required.

[0060] S403: Determine a free block bitmap, where the free block bitmap includes a plurality of free states corresponding to a plurality of blocks, where the free state is free or not free.

[0061] Exemplarily, the free block bitmap is a binary metadata structure that records the free state of each physical block in the storage pool. If the free state is not free, it means that the corresponding physical block is occupied, and if the free state is free, it means that the corresponding physical block can be allocated.

[0062] Optionally, after obtaining the free block bitmap, a version check is performed on the free block bitmap to check whether the free block bitmap is the latest version. If it is not the latest version, the free block bitmap is updated to improve the accuracy of allocating physical blocks.

[0063] A feasible implementation method can determine the target physical address through the following method, including: scanning and processing the free block bitmap according to the data size to determine multiple free blocks, and the free states of the multiple free blocks are all free; determining a block linked list, the block linked list includes multiple physical addresses corresponding to the multiple blocks; and determining multiple target physical addresses corresponding to the multiple free blocks according to the block linked list.

[0064] Exemplarily, the total space corresponding to the plurality of free blocks should be sufficient to store the modified data corresponding to the data size. It can be understood that allocating free blocks on demand according to the data size can avoid waste of free blocks and ensure that the free blocks are sufficient to store the modified data.

[0065] Optionally, the allocation method of multiple free blocks is determined according to the application scenario of the modified data. For example, for modified data that is mainly read and written sequentially, multiple continuous free blocks are determined to reduce the seek time for accessing the modified data, thereby improving the access speed. For modified data that is frequently read and written randomly, multiple discrete free blocks are determined to improve storage utilization.

[0066] Combined with the scenario example, the free blocks are not occupied. The free blocks are obtained by scanning to avoid the problem of storage failure caused by storing modified data in occupied blocks.

[0067] Exemplarily, a block linked list is a data structure that records the physical addresses of multiple physical blocks and is used to quickly allocate and recycle physical blocks. The block linked list is queried to obtain multiple target physical addresses corresponding to multiple free blocks.

[0068] In this feasible implementation, the block linked list records the physical address in advance, and the target physical address can be quickly obtained through the block linked list, thereby improving the efficiency of data management.

[0069] A feasible implementation method can store the modified data to the target physical address through the following method, including: generating an encryption key corresponding to the modified data; encrypting the modified data using the encryption key to obtain encrypted data; and storing the encrypted data to the target physical address.

[0070] Optionally, an encryption key is generated by a random generator or derived based on a user password.

[0071] Optionally, encryption keys and encrypted data are stored separately to avoid the risk of single point leakage and improve data security.

[0072] Optionally, the encryption key can be split into multiple shards and stored in multiple distributed nodes through distributed storage, which can avoid the risk of single point leakage and improve data security.

[0073] Optionally, encryption processing is performed using a symmetric encryption algorithm or an asymmetric encryption algorithm to obtain encrypted data.

[0074] Optionally, encrypt each free block independently to avoid the problem of excessive performance overhead caused by full disk encryption.

[0075] Optionally, if an interruption occurs during the encryption process, such as a hardware failure, the terminal is rolled back to the state before encryption and the encryption process is performed again.

[0076] Combined with the scenario example, the encrypted data is obtained by encrypting the modified data, and the encrypted data is stored to allow the user range who can access the data.

[0077] In this feasible implementation, the encrypted data is stored in the target physical address and can be obtained by unauthorized personnel in a timely manner. Since the data has been encrypted, it can be prevented from being directly accessed, thereby improving the security of the data.

[0078] S404: Determine a global generation identification table, where the global generation identification table includes a plurality of allocated historical generation identifications.

[0079] Exemplarily, the allocated historical generation identifiers are used generation identifiers, each historical generation identifier corresponds to a data modification operation, and the global generation identifier table is a central metadata table in the storage system that records all allocated generation identifiers.

[0080] Optionally, the global generation identification table can be stored in a single point centralized manner or distributedly across nodes. The global generation identification table maintains data consistency across nodes and cycles.

[0081] Exemplarily, the historical generation identifier is unique, that is, any two historical generation identifiers in the global generation identifier table are different.

[0082] Combined with the scenario example, the storage system includes multiple logical addresses. When the data under each logical address is modified, the corresponding generation identifier is assigned. The global generation identifier table stores the historical generation identifier assigned to each logical address to avoid conflicts with any historical generation identifier when allocating the generation identifier.

[0083] S405: Determine a target generation identifier according to the historical generation identifier, where the target generation identifier is different from any historical generation identifier.

[0084] Optionally, the target generation identifier is assigned in a monotonically increasing and globally unique manner. For example, if the generation identifier with the largest value among multiple historical generation identifiers is 019, then the target generation identifier is 020.

[0085] Exemplarily, each generation identifier is ensured to be a unique generation identifier in a globally unique manner.

[0086] Optionally, the global generation counter is incremented through atomic operations to obtain a unique and incrementing target generation identifier to avoid generating multiple identical generation identifiers at the same time, thereby improving the accuracy of data management.

[0087] Combined with the scenario example, the target generation identifier determined based on the historical generation identifier can avoid conflicts between the target generation identifier and any historical generation identifier, thereby ensuring the uniqueness of each generation identifier to improve the reliability of data management.

[0088] Based on the above implementation method, the target generation identifier is determined through the global generation identifier table to ensure the uniqueness of the target generation identifier, so as to accurately distinguish the target address mapping relationship from other address mapping relationships through the target generation identifier to avoid conflicts, thereby improving the reliability of data management.

[0089] In a feasible implementation, the data management method further includes: updating the global generation identification table through the target generation identification to obtain an updated generation identification table; determining multiple distributed storage nodes; and sending the updated generation identification table to the multiple distributed storage nodes.

[0090] Exemplarily, each distributed storage node holds the same generation identification table to avoid conflicts in newly generated generation identifications caused by version divergence. In addition, each distributed storage node holds the same generation identification table so that repair can be performed in the event of a single point failure.

[0091] Combined with the scenario example, by sending the updated generation identification table to multiple distributed storage nodes, any distributed storage node can allocate a generation identification according to the updated generation identification table, thereby avoiding generation identification conflicts. Accordingly, after any distributed storage node updates the generation identification table, it is synchronized to other distributed storage nodes.

[0092] In this feasible implementation, by sending an updated generation identification table to distributed storage nodes, the uniqueness of generation identifications across nodes can be ensured, thereby improving the reliability of data management.

[0093] A feasible implementation method, the data management method also includes: determining multiple child-generation identifiers corresponding to the original logical address, the multiple child-generation identifiers including the target generation identifier; arranging the multiple child-generation identifiers in descending order according to a chain structure to obtain a child-generation identifier list; generating a sub-bitmap index based on the child-generation identifier list, the sub-bitmap index including multiple bits, each bit corresponding to a child-generation identifier.

[0094] Exemplarily, the multiple sub-generation identifiers are generation identifiers corresponding to multiple data modification records corresponding to the original logical address.

[0095] Optionally, multiple child generation identifiers are determined from a global generation identifier table.

[0096] Exemplarily, arranging multiple child sub-identifiers in descending order according to a chain structure can reduce the complexity of traversal, thereby quickly locating a specified child sub-identifier.

[0097] Optionally, the child sub-identity list is a one-way or two-way linked list, and the head node points to the latest version. The head node points to the latest version to improve traversal efficiency.

[0098] Optionally, the list of child generation identifiers is mapped to a binary bitmap, where each bit indicates whether a child generation identifier exists. For example, assuming that the global generation range is 001-005, and the child generation identifiers are [001, 003, 005], that is, the data modification records corresponding to 001, 003, and 005 are modification records of the data under the original logical address, and the data modification records corresponding to 002 and 004 are modification records of the data under other logical addresses, and the bitmap is 10101 (5 bits) in binary representation. It can be understood that by compressing to a binary representation, memory usage can be reduced.

[0099] Combined with the scenario example, the generation identification list related to each logical address is managed separately. When the data is rolled back, the specified historical data can be quickly located from the generation identification list corresponding to the specified logical address according to the logical address specified for data rollback, thereby improving the efficiency of data rollback.

[0100] In this feasible implementation, through the collaborative design of chain descending arrangement and sub-bitmap index, an efficient, lightweight and scalable solution is provided for multi-version management of logical addresses, which can improve the efficiency of data rollback.

[0101] S406, obtaining the current metadata writing speed, the current processor usage rate, the current memory usage rate, and the time interval matrix table.

[0102] Exemplarily, the current metadata write speed, the current processor usage, and the current memory usage are all current performance parameters. The metadata write speed indicates the update frequency of metadata within a unit time. The processor usage is used to characterize the load of computing resources. The memory usage is used to characterize the load of memory resources. The duration interval matrix table is used to map the duration intervals that match the current performance, and the current performance is sufficient to support the generation of snapshots according to the matching duration intervals.

[0103] Optionally, the duration interval table is generated by pre-calculation.

[0104] Optionally, recalculation is performed periodically to update the duration interval table.

[0105] In the related art, the snapshot generation frequency is a fixed value. The snapshot generation frequency is low, and the snapshot covers fewer mapping relationships. When it is necessary to roll back to any historical data, there may be no corresponding snapshot, resulting in rollback failure.

[0106] Combined with the scenario example, multiple current performance parameters are obtained, and the maximum snapshot generation frequency supported by the current performance can be determined based on the current multiple performance parameters, and the current actual snapshot generation frequency is determined based on the maximum supported snapshot generation frequency. It can be understood that the current actual snapshot generation frequency is less than or equal to the maximum supported snapshot generation frequency, which can ensure that the generation of snapshots does not occupy too many system resources, so that the system can operate normally while the snapshot is generated, thereby improving the reliability of the system.

[0107] S407: Determine a target duration interval according to the current metadata writing speed, the current processor usage, the current memory usage, and the duration interval matrix table.

[0108] Combined with the scenario example, the shorter the interval, the greater the frequency of snapshot generation, the more snapshots need to be generated in a period, and the greater the pressure on system resources. The target interval determined based on real-time performance is the shortest interval supported by system resources. When the actual interval for generating snapshots is greater than the target interval, the generation of snapshots will not affect the normal operation of the system.

[0109] Optionally, the current metadata writing speed, the current processor usage, and the current memory usage are normalized to obtain corresponding normalized indicators, and the target time interval is obtained by matching the normalized indicators from the time interval matrix table. It can be understood that the normalization process can eliminate the dimension difference, thereby improving the matching accuracy.

[0110] Optionally, a corresponding weight is determined for each performance parameter, and each performance parameter is corrected according to the weight to obtain a correction result, and the target time interval is matched from the time interval matrix table according to the correction result. It can be understood that the contribution of each performance parameter can be accurately reflected through the correction process, thereby improving the accuracy of matching.

[0111] Optionally, multiple parameter ranges are set for each performance parameter in the duration interval matrix table, and the corresponding target duration interval is determined according to which parameter range the performance parameter falls into.

[0112] Next, combine Figure 5 The duration interval matrix table is explained.

[0113] Figure 5 Schematic diagram of the time interval matrix table provided in the embodiment of the present application. Figure 5 As shown, for example, if the current metadata writing speed is in range B, the current processor usage is in range E, and the current memory usage is in range H, then the target duration interval is determined to be interval B.

[0114] S408: Create a target snapshot at each target time interval according to the mapping relationship between the target generation identifier and the target address.

[0115] Exemplarily, the target address mapping relationship is a mapping relationship corresponding to the modified data, and recording the target address mapping relationship through the target snapshot can reduce the memory usage of the target snapshot.

[0116] Optionally, add a timestamp of when the target snapshot was generated to the target snapshot. The timestamp is the time when the target snapshot was generated.

[0117] Combined with the scenario example, by adding timestamp information, you can quickly locate the snapshot at a specified time according to the timestamp when rolling back data, thereby improving the positioning speed and thus improving the efficiency of data rollback.

[0118] Based on the above implementation methods, by creating target snapshots according to target time intervals, the snapshot generation frequency can be increased without affecting the normal operation of the system, thereby covering more historical data and performing data rollback for more historical data.

[0119] A feasible implementation method can determine the target duration interval by the following method, including: matching the matching duration interval from the duration interval matrix table according to the current metadata writing speed, the current processor usage rate, and the current memory usage rate; calculating and processing according to the current metadata writing speed, the current processor usage rate, and the current memory usage rate to obtain the theoretical metadata writing speed; if a preset number of consecutive historical metadata writing speeds are all less than the theoretical metadata writing speed, the sum of the matching duration interval and the preset value is determined as the target duration interval; if a preset number of consecutive historical metadata writing speeds are all greater than the theoretical metadata writing speed, the difference between the matching duration interval and the preset value is determined as the target duration interval; if there are no preset number of consecutive historical metadata writing speeds that are all less than the theoretical metadata writing speed, and there are no preset number of consecutive historical metadata writing speeds that are all greater than the theoretical metadata writing speed, the matching duration interval is determined as the target duration interval.

[0120] Exemplarily, the current metadata writing speed is the acquired data, and the theoretical metadata writing speed is the subsequent calculated data obtained by calculation.

[0121] Exemplarily, the target duration interval is dynamically corrected. For example, the theoretical metadata write speed is calculated to be M. Then, if the actual metadata write speed is less than the theoretical metadata write speed M for consecutive preset times, the snapshot duration interval is increased by the preset duration; if the metadata write speed is greater than the theoretical metadata write speed M for consecutive preset times, the snapshot duration interval is reduced by the preset duration, and so on. The same applies to processor usage and memory usage.

[0122] In this feasible implementation, by dynamically adjusting the snapshot frequency, the snapshot frequency can be made consistent with the real-time resources of the system, thereby avoiding increasing the pressure on the system.

[0123] A feasible implementation method can create a target snapshot through the following method, including: determining the encoding length based on multiple historical generation identifiers; determining a blank field based on the encoding length, and adding a blank field to the target address mapping relationship; adding the target generation identifier in the blank field to obtain a target snapshot.

[0124] In the related art, during the process of generating a snapshot, data reading and writing are set to a silent state, that is, data reading and writing are locked until the snapshot generation is completed, so as to avoid conflicts between data reading and writing during the process of generating the snapshot and the snapshot.

[0125] Exemplarily, the target snapshot is generated by the generation identifier, and different snapshots can be distinguished by the generation identifier. The data write operation newly generated in the process of generating the snapshot is directed to the new physical address.

[0126] Combined with the scenario example, for example, the target generation identifier corresponding to the current target snapshot is 020, and the corresponding physical address is the target physical address. In the process of generating the target snapshot, if a new data write operation is generated, the newly generated data write operation is directed to a physical address different from the target physical address and is identified by the generation identifier 021, so as to accurately distinguish the data modified by the new data write operation from the current data.

[0127] Exemplarily, the coding length is the number of coding bits, and the coding length determined by the historical generation identifier is sufficient to accommodate the target generation identifier, ensuring that the target generation identifier can be displayed correctly. For example, the largest generation identifier among multiple historical generation identifiers can be 019 or 019653, and 019 and 019653 occupy different numbers of bits. The coding length determined based on multiple historical generation identifiers should be sufficient to display the target generation identifier.

[0128] Next, combine Figure 6 The address mapping relationship is described.

[0129] Figure 6 A schematic diagram of the address mapping relationship provided in the embodiment of the present application. Figure 6 As shown, the target address mapping relationship includes a logical address field and a physical address field. The logical address field stores the original logical address, and the physical address field stores the target physical address. A blank field is added to the target address mapping relationship, and the length of the blank field is the encoding length. The blank field is used to store the target generation identifier. After adding the target generation identifier to the blank field, a target snapshot is created according to the target address mapping relationship.

[0130] In this feasible implementation, the target snapshot created by the target generation identifier and the target address mapping relationship can be distinguished from other snapshots by the target generation identifier, and the data storage address can also be obtained by the target address mapping relationship, thereby achieving accurate data rollback.

[0131] In a feasible implementation method, the data management method also includes: determining the previous generation identifier of the target generation identifier from multiple historical generation identifiers; generating a parent version pointer based on the previous generation identifier; and adding the parent version pointer in a blank field.

[0132] Exemplarily, the parent version pointer is a shortcut link pointing to the previous generation identifier. The parent-child dependency relationship is constructed through the parent version pointer to form a global version chain.

[0133] Exemplarily, if the generation identifier is an increasing serial number, the previous generation identifier is the generation identifier with the largest serial number among multiple historical generation identifiers. If the generation identifier is a timestamp, the previous generation identifier is the generation identifier closest to the current time among multiple historical generation identifiers.

[0134] For example, chain jumps can be achieved through the parent version pointer, that is, each snapshot is a node on the chain, and the parent pointer is the chain connecting the nodes. The parent version pointer can be used to quickly locate the previous generation identifier, so that data can be rolled back quickly.

[0135] Exemplarily, the parent version pointer only stores the inheritance relationship between data versions instead of storing specific data, which can save memory space.

[0136] Combined with the scenario example, the parent version pointer can be used to clarify the data inheritance relationship. Specifically, it can quickly determine which data version any data version is modified based on, so as to quickly locate the historical data.

[0137] In this feasible implementation, a global version chain is established, and any historical version can be restored through pointer jumps to reduce the time consumption of data rollback and improve the efficiency of data rollback.

[0138] Figure 7 A flow chart of a data management method provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, an embodiment of the present application provides a data management method, and the method is described in detail as follows:

[0139] S701. Receive a data rollback request, where the data rollback request includes a specified generation identifier and a source volume identifier.

[0140] Exemplarily, the designated generation identifier is the generation identifier corresponding to the data to be rolled back indicated by the data rollback request, and the source volume identifier is the identifier of the source volume corresponding to the data to be rolled back.

[0141] Optionally, there is a one-to-one correspondence between the volume and the volume identifier, and the unique corresponding source volume can be determined by the source volume identifier.

[0142] Combined with the scenario example, the source volume corresponds to a logical address and multiple physical addresses. Different versions of data are stored under multiple physical addresses. The physical address pointed to by the logical address in the current address mapping relationship is the latest version of the data. The data rollback request is used to request the acquisition of historical data stored under other physical addresses.

[0143] S702: According to the data rollback request, determine the source volume corresponding to the source volume identifier, and obtain multiple historical snapshots of the source volume.

[0144] Exemplarily, the multiple historical snapshots are all snapshots corresponding to the physical addresses corresponding to the source volume.

[0145] Next, combine Figure 8 Describe data rollback.

[0146] Figure 8A schematic diagram of data rollback provided in an embodiment of the present application. Figure 8 As shown in the figure, the data currently displayed by the source volume is the data stored at physical address D, and the data stored at physical address A, physical address B, and physical address C are all historical data. The address mapping relationship between the logical address and physical address A, physical address B, and physical address C is recorded through the corresponding historical snapshots, and each historical snapshot corresponds to an address mapping relationship. For example, historical snapshot B records the address mapping relationship between the logical address and physical address B. Among them, except for the physical address of the data storage created for the first time, the data stored in other physical addresses are only the data that has actually been modified.

[0147] S703: Perform data rollback processing according to the specified generation identifier and multiple historical snapshots to obtain a target volume, where the target volume includes rollback data.

[0148] Exemplarily, each historical snapshot corresponds to a generation identifier, and the rollback data is data stored at a physical address corresponding to the historical snapshot corresponding to the specified generation identifier.

[0149] A feasible implementation method can perform data rollback processing through the following method: create a first clone volume and a second clone volume of the source volume, the second clone volume is used to store the newly generated input and output data after receiving the data rollback request; obtain the rollback data according to the specified generation identifier and multiple historical snapshots; store the rollback data in the first clone volume to obtain a third clone volume; merge the second clone volume and the third clone volume to obtain the target volume.

[0150] Exemplarily, the first clone volume is created consistent with the source volume, and is used to load the rollback data to generate the intermediate state after the rollback (ie, the third clone volume). The second clone volume is created as an empty volume, and receives new data generated during the data rollback operation.

[0151] Exemplarily, the second clone volume and the third clone volume are merged, and the obtained target volume includes the rollback data and the newly generated data.

[0152] Combined with the scenario example, by separating the first clone volume and the second clone volume, it is possible to avoid conflicts between data read and write operations and data rollback during the data rollback operation.

[0153] Next, combine Fig. 9 Describe data rollback.

[0154] Fig. 9 A schematic diagram of data rollback provided in an embodiment of the present application. Fig. 9As shown, the generation identifier 000 indicates the initial creation of data, corresponding to the initial data A under each logical address. Each time the data is modified, a corresponding generation identifier is assigned, such as 001, 002, 003, and 004, and a corresponding snapshot is generated for the incremental data of each modification. The generation identifier of the current version of the data is 004. If the data rollback request indicates to roll back to the data with the generation identifier 002, the data under the corresponding physical address is obtained according to the snapshot 2 corresponding to the generation identifier 002, and stored in the first clone volume to obtain the third clone volume. The third clone volume includes the modified data corresponding to the incremental generation identifier 001 based on the initial data (i.e., data B) and the data modified by the generation identifier 002 (i.e., data C).

[0155] In this feasible implementation, by separating the first clone volume and the second clone volume, there is no need to lock the data read and write operations during the data rollback operation, and there is no need to interrupt the data read and write operations, thereby ensuring business continuity.

[0156] A feasible implementation method can obtain rollback data through the following method, including: creating a logical view based on multiple historical snapshots, the logical view includes multiple address mapping relationships corresponding to the multiple historical snapshots; matching processing is performed based on the specified generation identifier and the logical view to obtain the specified physical address; and obtaining rollback data from the physical address.

[0157] Exemplarily, the logical view is a global metadata index layer that integrates address mapping relationships of multiple historical snapshots to obtain a versioned address space topology.

[0158] Optionally, the process of creating a logical view may include scanning multiple historical snapshots, extracting the address mapping relationship recorded in each historical snapshot, sorting the multiple address mapping relationships according to multiple generation identifiers corresponding to the multiple historical snapshots, obtaining a multi-version address chain, and the multi-version address chain forms a logical view.

[0159] Exemplarily, a physical address corresponding to a specified generation identifier in a logical view is determined as the specified physical address.

[0160] Optionally, the logical view is stored according to the data type in the logical view, for example, the frequently accessed address mapping relationships in the logical view are stored in the memory, and the infrequently accessed address mapping relationships in the logical view are stored in the disk to improve the access speed of the frequently accessed address mapping relationships.

[0161] In this feasible implementation, a logical view is established only based on the historical snapshot corresponding to the source volume. Compared with establishing a logical view for the full historical snapshot in the system, this can reduce the execution complexity and thus improve the efficiency of data rollback.

[0162] A feasible implementation method can perform matching processing through the following method to obtain the specified physical address, including: performing matching processing through multiple threads according to the specified generation identifier and logical view to obtain a matching result, the matching result includes the specified physical address or a miss; if the matching result is a miss, then the physical address corresponding to the generation identifier adjacent to the specified generation identifier in the logical view is determined as the specified physical address.

[0163] Exemplarily, performing matching processing simultaneously through multiple threads can improve matching efficiency compared to performing matching processing through a single thread.

[0164] Optionally, tasks are assigned to multiple threads according to the loads corresponding to the multiple threads to achieve reasonable assignment. Specifically, a range of generation identifiers matching the load is assigned to each thread. For example, thread A is assigned matching tasks corresponding to the range of generation identifiers 0000-0113, and thread B has a lower load, so thread B is assigned matching tasks corresponding to the range of generation identifiers 0114-0506.

[0165] Optionally, each thread is matched by binary search to locate the specified generation identifier.

[0166] Combined with the scenario example, if the matching result is a miss, it means that the specified generation identifier is not included in the logical view, which may be caused by data corruption and other problems. At this time, the data in the physical address corresponding to the generation identifier closest to the specified generation identifier is closest to the data that the data rollback request wants to roll back. For example, if the specified generation identifier is 075, the logical view does not include generation identifier 075, but includes generation identifier 060 and generation identifier 080, then the physical address corresponding to generation identifier 080 is determined as the specified physical address.

[0167] Optionally, if the matching result is a miss and the designated physical address is matched through an adjacent generation identifier, a corresponding prompt message is generated so that the user can clearly identify the version corresponding to the rollback data according to the prompt message.

[0168] In this feasible implementation, matching of adjacent generation identifiers can be used to cope with extreme scenarios and improve the fault tolerance of data rollback.

[0169] In a feasible implementation manner, the data management method also includes: obtaining a transaction log of the source volume, the transaction log including the metadata state and physical block reference count of the source volume before executing the data rollback; performing integrity verification on the second clone volume and the third clone volume to obtain a verification result, the verification result being verification passed or verification failed; if the verification result is verification failed, restoring the source volume through the transaction log to obtain a recovery volume; if the verification result is verification passed, merging the second clone volume and the third clone volume to obtain a target volume.

[0170] Exemplarily, the metadata state includes a generation identifier, a snapshot, and an address mapping relationship. The physical block reference count is the number of times each physical block is referenced by a snapshot (for example, any physical block is referenced by three snapshots).

[0171] Optionally, before performing the data rollback, a transaction log is generated, and the transaction log reflects the state of the source volume before the data rollback.

[0172] Optionally, check whether the address mapping relationship in the second clone volume and the third clone volume is self-consistent (for example, whether there is a repeated mapping relationship) for verification processing. Check whether the physical block reference counts of the second clone volume and the third clone volume match the physical block reference counts in the transaction log for verification processing.

[0173] Combined with the scenario example, if the verification is successful, it means that the second clone volume and the third clone volume are both valid, and the target volume is generated. If the verification is successful, it means that the second clone volume and the third clone volume are invalid and may be abnormal, so restore to the source volume to avoid abnormal data affecting user use.

[0174] In this feasible implementation, through verification processing, data rollback can be stopped when abnormal data appears, so as to avoid the abnormal data affecting user usage, thereby improving the reliability of data rollback.

[0175] In a feasible implementation manner, the data management method further includes: determining the used space of multiple historical snapshots in the source volume, and multiple timestamps corresponding to the multiple historical snapshots; if the used space is greater than or equal to a preset space size, deleting the historical snapshots from the source volume according to the multiple timestamps until the used space is less than the preset space size, and releasing the corresponding physical addresses.

[0176] Exemplarily, the used space is the memory space occupied by multiple historical snapshots. The timestamp corresponding to each historical snapshot is determined according to the time when the historical snapshot is generated. The preset space size is the upper limit of the memory space size that can be occupied by the preset historical snapshot. Exceeding the preset space size may affect the normal operation of the system.

[0177] Optionally, when deleting historical snapshots, the historical snapshots corresponding to the timestamp farthest from the current moment are deleted first.

[0178] Combined with the scenario example, the farther the historical snapshot is from the current moment, the lower the corresponding data version is and the lower the value of rollback is, so it is deleted first.

[0179] Optionally, a priority level corresponding to each snapshot is generated according to data modification information corresponding to each snapshot, and snapshots with lower priorities are deleted first.

[0180] In this feasible implementation, by periodically recycling snapshots, it is possible to avoid excessive space occupation due to unlimited increase of snapshots, thereby improving the smoothness of system operation.

[0181] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0182] Fig.10 This is a schematic diagram of the structure of the data management device provided in the embodiment of the present application. Fig.10 As shown, the embodiment of the present application further provides a data management device, the data management device 100 may include: a receiving module 101, a modifying module 102, a creating module 103, and a generating module 104, wherein:

[0183] The receiving module 101 is used to receive a data modification request, where the data modification request includes original data, original logical address, original physical address, and modification information.

[0184] The modification module 102 is used to modify the original data according to the data modification request through the modification information to obtain the modified data, determine the target physical address, and store the modified data at the target physical address.

[0185] The creation module 103 is used to determine the target generation identifier corresponding to the modified data, and create a target address mapping relationship according to the target generation identifier, the original logical address, and the target physical address. The target generation identifier indicates the generation of the data modification request.

[0186] The generating module 104 is used to create a target snapshot according to the mapping relationship between the target generation identifier and the target address, and the target snapshot is used to restore the original data.

[0187] Optionally, the receiving module 101 may execute Figure 2 S201 in the embodiment.

[0188] Optionally, the modification module 102 may execute Figure 2 S202 in the embodiment.

[0189] Optionally, the creation module 103 may execute Figure 2 S203 in the embodiment.

[0190] Optionally, the generating module 104 may execute Figure 2 S204 in the embodiment.

[0191] It should be noted that the data management device shown in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0192] In a possible implementation, the modification module 102 is specifically configured to:

[0193] Determine the data size of the modified data;

[0194] Determine a free block bitmap, the free block bitmap includes a plurality of free states corresponding to a plurality of blocks, the free state being free or not free;

[0195] Determine the target physical address based on the free block bitmap and data size.

[0196] In a possible implementation, the modification module 102 is specifically configured to:

[0197] According to the data size, the free block bitmap is scanned to determine a plurality of free blocks, and the free states of the plurality of free blocks are all free;

[0198] Determine a block linked list, the block linked list includes multiple physical addresses corresponding to multiple blocks;

[0199] According to the block linked list, multiple target physical addresses corresponding to the multiple free blocks are determined.

[0200] In a possible implementation, the creation module 103 is specifically configured to:

[0201] Determine a global generation identification table, the global generation identification table including a plurality of allocated historical generation identifications;

[0202] A target generation identifier is determined based on the historical generation identifier, and the target generation identifier is different from any historical generation identifier.

[0203] In a possible implementation, the generating module 104 is specifically configured to:

[0204] Get the current metadata writing speed, current processor usage, current memory usage, and time interval matrix table;

[0205] Determine a target time interval according to a current metadata writing speed, a current processor usage rate, a current memory usage rate, and a time interval matrix table;

[0206] At target time intervals, a target snapshot is created based on the mapping relationship between the target generation identifier and the target address.

[0207] In a possible implementation, the generating module 104 is specifically configured to:

[0208] According to the current metadata writing speed, the current processor usage rate, and the current memory usage rate, a matching time interval is obtained from the time interval matrix table;

[0209] The theoretical metadata writing speed is obtained by performing calculations based on the current metadata writing speed, the current processor usage rate, and the current memory usage rate.

[0210] If a preset number of consecutive historical metadata writing speeds are all lower than the theoretical metadata writing speed, the sum of the matching time interval and the preset value is determined as the target time interval;

[0211] If a preset number of consecutive historical metadata writing speeds are all greater than the theoretical metadata writing speed, the difference between the matching time interval and the preset value is determined as the target time interval;

[0212] If there are not a preset number of consecutive historical metadata writing speeds that are all lower than the theoretical metadata writing speed, and there are not a preset number of consecutive historical metadata writing speeds that are all higher than the theoretical metadata writing speed, then the duration interval is matched and determined as the target duration interval.

[0213] Fig.11 A schematic diagram of the structure of a data management device provided in an embodiment of the present application. Fig.10 Based on the embodiment shown, Fig.11 As shown, the data management device 110 further includes: an encryption module 105, a distribution module 106, a bitmap module 107, a first adding module 108, a second adding module 109, a rollback module 111, a verification module 112, and a recycling module 113, wherein:

[0214] The encryption module 105 is used for:

[0215] Generate an encryption key corresponding to the modified data;

[0216] The modified data is encrypted by using an encryption key to obtain encrypted data;

[0217] Store the encrypted data to the target physical address.

[0218] The distributed module 106 is used for:

[0219] The global generation identification table is updated through the target generation identification to obtain an updated generation identification table;

[0220] Determine multiple distributed storage nodes;

[0221] Send an update generation identification table to multiple distributed storage nodes.

[0222] The bitmap module 107 is used for:

[0223] Determine a plurality of child generation identifiers corresponding to the original logical address, wherein the plurality of child generation identifiers include a target generation identifier;

[0224] Arrange multiple child sub-identifiers in descending order according to the chain structure to obtain a child sub-identifier list;

[0225] A sub-bitmap index is generated according to the list of sub-generation identifiers, and the sub-bitmap index includes a plurality of bits, each bit corresponding to a sub-generation identifier.

[0226] The first adding module 108 is used for:

[0227] Determine the coding length according to multiple historical generation identifiers;

[0228] Determine the blank field according to the encoding length, and add the blank field to the target address mapping relationship;

[0229] Add the target generation ID in the blank field to get the target snapshot.

[0230] The second adding module 109 is used for:

[0231] Determine a previous generation identifier of a target generation identifier from a plurality of historical generation identifiers;

[0232] Generate a parent version pointer based on the previous generation identifier;

[0233] Add the parent version pointer in the empty field.

[0234] The rollback module 111 is used for:

[0235] receiving a data rollback request, the data rollback request including a specified generation identifier and a source volume identifier;

[0236] According to the data rollback request, determine the source volume corresponding to the source volume identifier, and obtain multiple historical snapshots of the source volume;

[0237] According to the specified generation identifier and multiple historical snapshots, data rollback processing is performed to obtain a target volume, which includes the rollback data.

[0238] In a possible implementation, the rollback module 111 is specifically configured to:

[0239] Creating a first clone volume and a second clone volume of the source volume, where the second clone volume is used to store input and output data newly generated after receiving the data rollback request;

[0240] Get rollback data based on the specified generation ID and multiple historical snapshots;

[0241] The rollback data is stored in the first clone volume to obtain a third clone volume;

[0242] The second clone volume and the third clone volume are merged to obtain a target volume.

[0243] In a possible implementation, the rollback module 111 is specifically configured to:

[0244] Creating a logical view according to the multiple historical snapshots, where the logical view includes multiple address mapping relationships corresponding to the multiple historical snapshots;

[0245] Perform matching processing according to the specified generation identifier and the logical view to obtain the specified physical address;

[0246] Get rollback data from the physical address.

[0247] In a possible implementation, the rollback module 111 is specifically configured to:

[0248] According to the specified generation identifier and logical view, matching processing is performed through multiple threads to obtain a matching result, which includes a specified physical address or a miss;

[0249] If the matching result is a miss, the physical address corresponding to the generation identifier adjacent to the specified generation identifier in the logical view is determined as the specified physical address.

[0250] The verification module 112 is used to: obtain the transaction log of the source volume, the transaction log includes the metadata state and physical block reference count of the source volume before performing data rollback; perform integrity verification on the second clone volume and the third clone volume to obtain a verification result, which is either a verification pass or a verification fail; if the verification result is a verification fail, restore the source volume through the transaction log to obtain a recovery volume; if the verification result is a verification pass, merge the second clone volume and the third clone volume to obtain a target volume.

[0251] The recycling module 113 is used to: determine the used space of multiple historical snapshots in the source volume and multiple timestamps corresponding to the multiple historical snapshots; if the used space is greater than or equal to the preset space size, delete the historical snapshots from the source volume according to the multiple timestamps until the used space is less than the preset space size, and release the corresponding physical addresses.

[0252] For the description of the features in the embodiment corresponding to the data management device, reference can be made to the relevant description of the embodiment corresponding to the data management method, which will not be described in detail here.

[0253] Fig.12 This is a schematic diagram of the structure of the electronic device provided in this application. Fig.12As shown, the electronic device 120 provided in this embodiment includes: at least one processor 1201 and a memory 1202. Optionally, the electronic device 120 further includes a communication component 1203. The processor 1201, the memory 1202 and the communication component 1203 are connected via a bus.

[0254] In the specific implementation process, at least one processor 1201 executes the computer execution instructions stored in the memory 1202, so that at least one processor 1201 executes the above-mentioned data management method embodiment.

[0255] The specific implementation process of the processor 1201 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0256] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the application may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0257] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0258] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0259] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned data management method embodiments when running.

[0260] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0261] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above data management method embodiments are implemented.

[0262] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned data management method embodiments are implemented.

[0263] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0264] The above is a detailed introduction to a data management device provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A data management method, characterized in that: include: receiving a data modification request, wherein the data modification request includes original data, an original logical address, an original physical address, and modification information; According to the data modification request, modify the original data using the modification information to obtain modified data, determine a target physical address, and store the modified data at the target physical address; Determine a target generation identifier corresponding to the modified data, and create a target address mapping relationship according to the target generation identifier, the original logical address, and the target physical address, wherein the target generation identifier indicates the generation of the data modification request; A target snapshot is created according to the target generation identifier and the target address mapping relationship, and the target snapshot is used to restore the original data.

2. The method according to claim 1, characterized in that Determining the target physical address includes: Determining the data size of the modified data; Determine a free block bitmap, the free block bitmap comprising a plurality of free states corresponding to a plurality of blocks, the free state being free or not free; A target physical address is determined according to the free block bitmap and the data size.

3. The method according to claim 2, characterized in that Determining a target physical address according to the free block bitmap and the data size includes: Scanning the free block bitmap according to the data size to determine a plurality of free blocks, wherein the free blocks are all in free status; Determine a block linked list, wherein the block linked list includes a plurality of physical addresses corresponding to a plurality of blocks; According to the block linked list, a plurality of target physical addresses corresponding to the plurality of free blocks are determined.

4. The method according to any one of claims 1 to 3, characterized in that Storing the modified data to the target physical address includes: generating an encryption key corresponding to the modified data; Encrypting the modified data using the encryption key to obtain encrypted data; The encrypted data is stored to the target physical address.

5. The method according to claim 1, characterized in that Determining a target generation identifier corresponding to the modified data includes: Determine a global generation identification table, wherein the global generation identification table includes a plurality of allocated historical generation identifications; The target generation identifier is determined according to the historical generation identifier, and the target generation identifier is different from any historical generation identifier.

6. The method according to claim 5, characterized in that The method further comprises: The global generation identification table is updated by using the target generation identification to obtain an updated generation identification table; Determine multiple distributed storage nodes; The update generation identification table is sent to the multiple distributed storage nodes.

7. The method according to claim 5, characterized in that The method further comprises: Determine a plurality of child generation identifiers corresponding to the original logical address, wherein the plurality of child generation identifiers include the target generation identifier; Arrange the multiple child sub-identifiers in descending order according to the chain structure to obtain a child sub-identifier list; A sub-bitmap index is generated according to the sub-generation sub-identification list, wherein the sub-bitmap index includes a plurality of bits, each bit corresponding to a sub-generation sub-identification.

8. The method according to claim 1, characterized in that Creating a target snapshot according to the target generation identifier and the target address mapping relationship includes: Get the current metadata writing speed, current processor usage, current memory usage, and time interval matrix table; Determine a target duration interval according to the current metadata writing speed, the current processor usage rate, the current memory usage rate, and the duration interval matrix table; At each target time interval, a target snapshot is created according to the mapping relationship between the target generation identifier and the target address.

9. The method according to claim 8, characterized in that Determining a target duration interval according to the current metadata writing speed, the current processor usage rate, the current memory usage rate, and the duration interval matrix table includes: According to the current metadata writing speed, the current processor usage rate, and the current memory usage rate, matching duration intervals are obtained from the duration interval matrix table; Calculating and processing according to the current metadata writing speed, the current processor usage rate, and the current memory usage rate to obtain a theoretical metadata writing speed; If a preset number of consecutive historical metadata writing speeds are all lower than the theoretical metadata writing speed, the sum of the matching time interval and the preset value is determined as the target time interval; If a preset number of consecutive historical metadata writing speeds are all greater than the theoretical metadata writing speed, the difference between the matching time interval and the preset value is determined as the target time interval; If there are not a continuous preset number of historical metadata writing speeds that are all lower than the theoretical metadata writing speed, and there are not a continuous preset number of historical metadata writing speeds that are all higher than the theoretical metadata writing speed, then the matching duration interval is determined to be the target duration interval.

10. The method according to claim 8 or 9, characterized in that: Creating a target snapshot according to the target generation identifier and the target address mapping relationship includes: Determine the coding length according to multiple historical generation identifiers; Determine a blank field according to the encoding length, and add the blank field to the target address mapping relationship; The target generation identifier is added to the blank field to obtain the target snapshot.

11. The method according to claim 10, characterized in that The method further comprises: Determining a previous generation identifier of the target generation identifier from the multiple historical generation identifiers; Generate a parent version pointer according to the previous generation identifier; Add the parent version pointer in the blank field.

12. The method according to claim 1, characterized in that The method further comprises: receiving a data rollback request, wherein the data rollback request includes a specified generation identifier and a source volume identifier; Determine, according to the data rollback request, a source volume corresponding to the source volume identifier, and obtain multiple historical snapshots of the source volume; Data rollback processing is performed according to the specified generation identifier and the multiple historical snapshots to obtain a target volume, where the target volume includes rollback data.

13. The method according to claim 12, characterized in that Performing data rollback processing according to the multiple historical snapshots to obtain a target volume includes: creating a first clone volume and a second clone volume of the source volume, wherein the second clone volume is used to store input and output data newly generated after receiving the data rollback request; Acquire rollback data according to the specified generation identifier and the multiple historical snapshots; The rollback data is stored in the first clone volume to obtain a third clone volume; The second clone volume and the third clone volume are merged to obtain the target volume.

14. The method according to claim 13, characterized in that Acquiring rollback data according to the specified generation identifier and the multiple historical snapshots includes: Creating a logical view according to the multiple historical snapshots, the logical view including multiple address mapping relationships corresponding to the multiple historical snapshots; Perform matching processing according to the specified generation identifier and the logical view to obtain a specified physical address; The rollback data is obtained from the physical address.

15. The method according to claim 14, characterized in that Performing matching processing according to the specified generation identifier and the logical view to obtain a specified physical address includes: According to the specified generation identifier and the logical view, matching processing is performed through multiple threads to obtain a matching result, wherein the matching result includes a specified physical address or a miss; If the matching result is a miss, the physical address corresponding to the generation identifier adjacent to the designated generation identifier in the logical view is determined as the designated physical address.

16. The method according to claim 13, characterized in that The method further comprises: Acquire a transaction log of the source volume, wherein the transaction log includes a metadata state and a physical block reference count of the source volume before performing data rollback; Performing integrity verification on the second clone volume and the third clone volume to obtain a verification result, where the verification result is verification passed or verification failed; If the verification result is that the verification fails, the source volume is restored through the transaction log to obtain a recovery volume; If the verification result is verification passed, the second clone volume and the third clone volume are merged to obtain the target volume.

17. The method according to any one of claims 12 to 16, characterized in that: The method further comprises: Determining, in the source volume, used spaces of the plurality of historical snapshots and a plurality of timestamps corresponding to the plurality of historical snapshots; If the used space is greater than or equal to a preset space size, historical snapshots are deleted from the source volume according to the multiple timestamps until the used space is less than the preset space size, and the corresponding physical addresses are released.

18. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the data management method according to any one of claims 1 to 17 when executing the computer program.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the data management method according to any one of claims 1 to 17.

20. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the data management method according to any one of claims 1 to 17 are implemented.

Images