Address mapping management method and device, equipment and storage medium

Abstract

The invention discloses an address mapping management method and device, equipment and a storage medium. The method comprises the steps of receiving a storage space management operation causing a data physical address change; according to a first mapping relation, an intermediate address corresponding to the logic address of the data is determined, and the intermediate address uniquely identifies the data; updating the second mapping relation, and mapping the intermediate address to the changed new physical address; wherein the first mapping relation is used for maintaining mapping from a logic address to an intermediate address, and the second mapping relation is used for maintaining mapping from the intermediate address to a physical address. According to the embodiment of the invention, the problem of large-scale modification of the metadata caused by the change of the data physical address can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer storage, and in particular to an address mapping management method and device, equipment and a storage medium. BACKGROUND

[0002] Currently, in a storage system, especially a distributed storage system, deduplication and compression technologies are widely used to improve storage space utilization. At the same time, as the core guarantee of data protection and business continuity, the snapshot function has also become a standard feature of the storage system.

[0003] The implementation of a snapshot usually relies on metadata mapping management technology. The system maintains a mapping relationship between a logical address (such as LBA) and a physical address. When a snapshot is created, the newly generated snapshot shares the same underlying physical data and mapping relationship with the source volume. When data changes, the system allocates a new physical location for the new data using copy-on-write or other technologies, and updates the mapping relationship of the corresponding volume, thereby realizing data separation.

[0004] In addition, in order to balance performance and efficiency, the storage system often adopts an asynchronous processing strategy. For example, data is first written in an uncompressed form to the cache and the write operation is immediately confirmed, and then compression, deduplication and other heavy operations are performed asynchronously in the background, and finally the processed data is flushed to the backend persistent storage layer.

[0005] However, the above prior art has an inherent and significant technical defect:

[0006] When the storage system performs storage space management operations such as compression, defragmentation, garbage collection, etc., these operations will change the physical location of the data content in the backend storage (i.e. the physical address changes); under the existing mapping architecture, any change in the physical address must update all the logical address mapping relationships pointing to the physical address;

[0007] This mechanism will bring disastrous metadata operation overhead in the multi-snapshot or data sharing scenario; because a piece of data may be shared by a source volume and multiple snapshots, i.e. multiple logical addresses point to the same physical address; once the physical address changes due to background management operations, the system must traverse and modify all associated logical address mapping entries; the metadata update overhead is proportional to the number of snapshots sharing the data entity (O(n) complexity); when the system is large and the number of snapshots is large, this frequent and large amount of metadata update operation will seriously consume system resources, become the performance bottleneck of the entire system, and restrict the further improvement of storage efficiency. SUMMARY

[0008] An object of the present application is to provide an address mapping management method, device, equipment and storage medium, which are used to solve the problem of large-scale modification of metadata caused by data physical address change.

[0009] In a first aspect, the embodiments of the present application provide an address mapping management method, which comprises:

[0010] receiving a storage space management operation that causes data physical address change;

[0011] determining an intermediate address corresponding to the logical address of the data according to a first mapping relationship, wherein the intermediate address uniquely identifies the data;

[0012] updating a second mapping relationship to map the intermediate address to a new physical address after the change;

[0013] The first mapping relationship is used to maintain the mapping of logical address to intermediate address, and the second mapping relationship is used to maintain the mapping of intermediate address to physical address.

[0014] In the embodiments of the present application, by introducing an intermediate address layer and establishing two-layer mapping relationship between logical address and physical address, the problem of large-scale modification of metadata caused by data physical address change is effectively solved: when the physical storage location of data changes due to compression, defragmentation and other storage space management operations, the traditional method needs to update the direct mapping of all associated logical addresses, and the metadata modification overhead is proportional to the number of logical entities; while the present method only needs to update the second mapping relationship between the intermediate address and the physical address, without touching the first mapping relationship between the logical address and the intermediate address, so that the amount of metadata modification is greatly reduced, the metadata operation overhead is greatly reduced, and the execution efficiency of the storage space management operation and the overall system performance are significantly improved.

[0015] In some possible embodiments, the storage space management operation is a data compression flush operation. Before receiving the storage space management operation causing the data physical address to change, the method further includes: in response to a write request of the data, allocating the intermediate address for the data, and updating the first mapping relationship to establish a mapping between the logical address corresponding to the request and the allocated intermediate address; and writing the uncompressed data into the cache. This implementation decouples the back-end compression operation from the front-end write path by pre-allocating a stable intermediate address for the data and establishing a logical layer mapping. When the physical address needs to be changed after data compression, only the single mapping relationship between the intermediate address and the physical address needs to be updated, without modifying the mapping of all logical entities at the upper layer, thereby reducing the metadata modification overhead from O(n) complexity proportional to the number of snapshots to constant O(1) complexity, completely solving the problem of large-scale metadata modification caused by the compression operation in the multi-snapshot scenario, and significantly improving the performance of the storage system.

[0016] In some possible embodiments, after writing the uncompressed data into the cache, the method further includes: returning a write completion response to the host; and after receiving the storage space management operation causing the data physical address to change and updating the second mapping relationship, the method further includes: compressing the uncompressed data in the cache, and flushing to the back-end storage medium according to the new physical address. This implementation achieves an optimized balance between performance and efficiency: the front end guarantees low write latency through a fast response mechanism, and the back end realizes efficient use of storage space through asynchronous processing.

[0017] In some possible embodiments, the method further includes: in response to a data read request carrying a target logical address; querying the first mapping relationship to obtain a target intermediate address corresponding to the target logical address; using the target intermediate address as an index to search for corresponding uncompressed data in the cache; and returning the found uncompressed data. This implementation significantly reduces the data read delay by avoiding the double overhead of back-end storage access and decompression calculation, effectively improves the system response performance, and is especially suitable for access scenarios with time locality characteristics, thereby realizing efficient retrieval of hot data.

[0018] In some possible embodiments, after the corresponding uncompressed data is searched for in the cache, the method further includes: if the corresponding uncompressed data is not searched for in the cache, querying a second mapping relationship to obtain a target physical address corresponding to the target intermediate address; and reading the compressed data from the target physical address in the back-end storage medium and returning the data after decompression. This implementation adopts a hierarchical reading mechanism to optimize reading performance on the premise of ensuring complete reliability of data access: uncompressed data is preferentially obtained from the cache to achieve efficient access, and if a miss occurs, reading from the back-end storage and decompression are performed to ensure data integrity. This design achieves an optimal balance between performance and reliability, and can adapt to diversified workloads and access modes.

[0019] In some possible embodiments, the storage space management operation is a defragmentation operation. The updating of the second mapping relationship, which maps the intermediate address to the changed new physical address, includes: in a defragmentation process, in response to an operation of migrating the data from the original physical address to the new physical address, mapping the intermediate address to the new physical address. This implementation improves the defragmentation efficiency by decoupling the upper-layer logic and the underlying physical storage: in the data migration process, only the mapping relationship between the intermediate address and the new physical address needs to be updated, and the first mapping relationship of all upper-layer logical entities remains completely unchanged, thereby converting the high-complexity operation of modifying n logical mappings in the traditional architecture into a low-complexity operation of modifying only one underlying mapping, greatly reducing the amount of metadata modification, and completely eliminating the relevance between metadata operations and the number of snapshots, so that the defragmentation efficiency is improved by several orders of magnitude, while the consistency and transparency of data access are perfectly maintained.

[0020] In some possible embodiments, the first mapping relationship is shared by the source volume and at least one snapshot, so as to share the data by sharing the intermediate address. This implementation realizes the instantaneous generation of snapshots and the efficient reuse of storage space by the intermediate address sharing mechanism, and relies on the decoupling architecture of the logical layer and the physical layer to only update one underlying mapping record when the physical location of the data changes, thereby greatly reducing the metadata maintenance overhead and systematically solving the performance bottleneck that the storage operation is proportional to the metadata overhead in a multi-snapshot environment.

[0021] In some possible embodiments, the method further includes: generating a globally unique identifier based on a content feature of the data, as the intermediate address of the data. This implementation, by combining the content feature and the address generation, not only realizes efficient mapping management, but also additionally obtains additional functions such as data deduplication and integrity checking, thereby further improving the comprehensive performance and reliability of the storage system.

[0022] In a second aspect, an embodiment of the present application provides an address mapping management apparatus, which comprises:

[0023] An operation receiving module is configured to receive a storage space management operation that causes a data physical address to change;

[0024] An address determining module is configured to determine, according to a first mapping relationship, an intermediate address corresponding to a logical address of the data, wherein the intermediate address uniquely identifies the data;

[0025] A mapping relationship updating module is configured to update a second mapping relationship, mapping the intermediate address to a new physical address after the change;

[0026] The first mapping relationship is used to maintain a mapping of logical addresses to intermediate addresses, and the second mapping relationship is used to maintain a mapping of intermediate addresses to physical addresses.

[0027] In a third aspect, another embodiment of the present application further provides an address mapping management device, comprising at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any address mapping management method provided by the embodiments of the present application.

[0028] In a fourth aspect, another embodiment of the present application further provides a computer storage medium, which stores a computer program, and the computer program is used to enable a computer to execute any address mapping management method provided by the embodiments of the present application.

[0029] Other features and advantages of the present application will be described in the following description and will be apparent from the description, or will be learned from the practice of the application. The purposes and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims, and by the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. Obviously, the drawings to be introduced below are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0031] Figure 1 The address mapping management method flowchart provided according to the embodiments of the present application;

[0032] Figure 2 The data writing flowchart provided according to the embodiments of the present application;

[0033] Figure 3Fig. 1 is a schematic diagram of address mapping relationship before and after data writing according to prior art;

[0034] Figure 4 Fig. 2 is a schematic diagram of address mapping relationship before and after data writing according to an embodiment of the present application;

[0035] Figure 5 Fig. 3 is a schematic diagram of data reading process according to an embodiment of the present application;

[0036] Figure 6 Fig. 4 is a schematic diagram of address mapping relationship before and after data migration according to prior art;

[0037] Figure 7 Fig. 5 is a schematic diagram of address mapping relationship before and after data migration according to an embodiment of the present application;

[0038] Figure 8 Fig. 6 is a schematic diagram of address mapping management device according to an embodiment of the present application;

[0039] Figure 9 Fig. 7 is a structural diagram of address mapping management device according to an embodiment of the present application. DETAILED DESCRIPTION

[0040] To further illustrate the technical solutions provided by the embodiments of the present application, the following will describe the embodiments in detail with reference to the accompanying drawings and specific embodiments. Although the embodiments of the present application provide the following method operation steps as shown in the embodiments or drawings, more or fewer operation steps can be included in the method based on conventional or non-creative labor. The execution order of the steps is not limited to the execution order provided by the embodiments of the present application in the logical sense. The method can be executed in sequence or in parallel when the method is executed in actual processing or by a control device according to the method order shown in the embodiments or drawings.

[0041] At present, when a storage system performs space management operations such as compression, defragmentation, garbage collection, etc., the physical storage location of data can change. Under the traditional mapping mechanism, any change of physical address must be synchronized to update all logical mapping relationships pointing to the physical address. This mechanism can cause serious performance problems in the multi-snapshot or data sharing scenario: since the same physical data can be shared by the source volume and multiple snapshots (i.e., multiple logical addresses are mapped to the same physical address), once the physical address changes, the system must traverse and update all related logical mapping entries one by one. The overhead of this metadata update operation is linearly related to the number of snapshots or logical entities sharing the data (O(n)), and in a large-scale system, when there are a large number of snapshots or high-frequency background operations, it will cause frequent and massive metadata modification load, seriously occupying system resources, forming a performance bottleneck, and limiting the improvement of the overall efficiency of the storage system.

[0042] In view of the above problems existing in the related art, the present application proposes an address mapping management method, device and storage medium.

[0043] The related technical terms involved in the address mapping management method provided by the embodiments of the present application are explained as follows:

[0044] 1) Source volume, which is a basic logical storage container for users to directly perform data read and write operations, is the original data source for creating snapshots or other derived data volumes;

[0045] 2) Snapshot, which is a data copy generated based on a source volume at a specific time point, records the data state of the source volume at that time through mechanisms such as copy-on-write (Copy-on-Write), rather than complete data content, thereby realizing efficient data protection and recovery functions;

[0046] 3) De-duplication technology, which is a technical method for eliminating redundant data storage, its core principle is: identifying the characteristics of the data block written (such as calculating the hash value), if it is found that multiple logical data blocks have the same content, only the unique instance is retained in the physical storage, and other repeated instances are replaced with a reference pointer to the physical instance, thereby significantly improving the storage space utilization;

[0047] 4) Compression technology, which is a technical method for reducing the physical storage space occupied by data by encoding and reorganizing the data through a specific algorithm under the premise of ensuring lossless or controllable loss of information, the purpose of which is to reduce storage costs, improve transmission efficiency, and improve the overall resource utilization of the storage system;

[0048] 5) Online compression, which is a data compression processing strategy, specifically refers to completing the compression operation in real time in the I / O path of data writing into the storage system before being persisted to the disk, this method may slightly increase the write delay, but can immediately reduce the data amount of the write backend storage, and is suitable for business scenarios that are sensitive to storage space and can accept certain write performance impact.

[0049] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings.

[0050] The address mapping management method in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0051] The present application provides an address mapping management method, which is applied to a storage system, such as Figure 1 As shown in the figure, the method comprises the following steps 101-103.

[0052] Step 101, receiving a storage space management operation that causes the data physical address to change.

[0053] In this step, a command to initiate or execute a storage space management operation is received, which will cause the actual physical storage location of the data to change.

[0054] The storage space management operation serves as the event source that triggers the subsequent mapping update process, including but not limited to data compression flushing operation (writing compressed data in cache to backend persistent storage medium, which usually reallocates physical space for compressed data); defragmentation operation (migrating dispersed data to new, continuous physical addresses to improve storage space continuity and access efficiency); and garbage collection, data migration, and other background management tasks that also change the physical storage location of data. The core of step 101 is to identify and respond to such events in a timely manner, providing a trigger condition for subsequent updates to the second layer mapping relationship (i.e. mapping of intermediate addresses to new physical addresses), thereby avoiding modifying a large number of upper logical mapping relationships one by one, significantly reducing metadata update overhead.

[0055] Step 102, determining the intermediate address corresponding to the logical address of the data according to the first mapping relationship.

[0056] Wherein, the intermediate address uniquely identifies the data. The first mapping relationship is used to maintain the mapping of logical addresses to intermediate addresses. After step 101 is triggered, the process proceeds to step 102. The core of this step is that the system uses the logical address of the data as the query key (Key) to retrieve and query the first mapping relationship, thereby determining the unique intermediate address corresponding to it.

[0057] The logical address (Logical Block Address, LBA) is the data addressing identifier from the perspective of the host operating system or application, which defines the data position offset in the logical volume or virtual disk, and is independent of the specific layout of the underlying physical storage. In the initial data write request, the logical address is explicitly specified and carried by the host; in the storage space management operation (such as compression flushing and defragmentation) described in step 101, although the system does not directly receive external logical addresses, it can clearly trace and determine the logical address corresponding to the data to be operated through its internal management metadata.

[0058] Specifically, the first mapping relationship constitutes a first level in the system mapping management architecture, and its core function is to establish and maintain the mapping from the logical address space to the intermediate address space. Each intermediate address is defined in this application as a globally unique, content-independent identifier, and its core role is to uniquely represent a specific data content within the entire storage system. Regardless of how the physical storage location of the data changes in the backend, or how many snapshots or logical volumes it is referenced, its corresponding intermediate address remains unchanged throughout its life cycle.

[0059] Therefore, this step is the query hub of the entire mapping management mechanism. It converts the volatile logical address that is closely coupled with the user's view into a stable intermediate address that is decoupled from the physical implementation through a lightweight query operation. This conversion is the basis for all subsequent efficient management operations, ensuring that even if the physical location of the underlying data changes due to compression, consolidation, or other operations, there is no need to backtrack and modify all upper-level logical references. Only the binding relationship between the stable intermediate address and the final physical location needs to be adjusted in the second level.

[0060] Step 103, updating the second mapping relationship to map the intermediate address to the changed new physical address.

[0061] Among them, the second mapping relationship is used to maintain the mapping of the intermediate address to the physical address. Step 103 is the key execution phase of this method to realize its core advantage (significantly reducing metadata update overhead). This step specifically allocates a new physical address for the data migrated due to storage space management operations, and then updates the second mapping relationship to establish a mapping association between the intermediate address determined in step 102 and the new physical address. The second mapping relationship constitutes a second level in the system mapping management architecture, and its core responsibility is to dynamically maintain the mapping from the stable intermediate address space to the volatile physical address space. The physical address explicitly indicates the actual storage location of the data on the specific storage medium (such as a hard disk, SSD).

[0062] In the embodiments of the present application, when the physical storage location of data is changed due to a background management task, the system does not need to modify the mapping (i.e., the first mapping relationship) of the logical address to the intermediate address held by any upper logical entity (such as a source volume, a snapshot), but only needs to modify the target physical address pointed to by the unique intermediate address in the second mapping layer. Through this layered updating mechanism, the present application successfully converts the metadata updating problem caused by physical data movement, which is extensive in scope and complicated in operation, into a local and single mapping item updating operation. Regardless of how many snapshots or logical volumes the data block is shared by, the modification amount of metadata is significantly reduced from the O(n) complexity proportional to the number of sharing in the traditional method to the constant O(1) complexity, thereby greatly reducing the metadata management burden of the system and improving the execution efficiency of the storage space management operation and the overall performance of the system.

[0063] In some possible embodiments, the first mapping relationship is shared by the source volume and at least one snapshot to share the data by sharing the intermediate address.

[0064] Specifically, when a snapshot is created, the data content of the source volume is not copied, but the mapping entries in the first mapping relationship are shared, so that the corresponding logical addresses in the source volume and the snapshot volume point to the same intermediate address. This sharing mechanism can be implemented in the following way: when a snapshot is created, an independent first mapping relationship is established for the newly generated snapshot, which is consistent with the first mapping relationship of the source volume initially, i.e., the same logical address points to the same intermediate address. When a subsequent data write operation occurs, a copy-on-write mechanism can be used to allocate a new intermediate address only for the modified data and update the mapping relationship of the corresponding volume, while the unmodified data continues to share the same physical data through the same intermediate address.

[0065] In the embodiments of the present application: first, the time and space overhead required for snapshot creation is greatly reduced, and nearly instantaneous snapshot generation is achieved; second, through the sharing of intermediate addresses, the source volume and the snapshot can transparently share the same physical data, avoiding unnecessary waste of storage space; most importantly, when the physical location of the underlying data changes due to compression, migration, or other operations, only the mapping relationship (second mapping relationship) of the intermediate address to the physical address needs to be updated, and all logical volumes (including the source volume and all related snapshots) that share the intermediate address through the first mapping relationship can automatically point to the new physical location, significantly reducing the metadata update overhead.

[0066] In some possible embodiments, the storage space management operation is a data compression flush operation. Referring to Figure 2 Before receiving the storage space management operation causing the change of the physical address of the data, the method further includes:

[0067] Step 201, in response to a data write request, allocating an intermediate address for the data;

[0068] Step 202, updating the first mapping relationship to establish a mapping between the logical address corresponding to the write request and the allocated intermediate address;

[0069] Step 203, writing the uncompressed data into the cache.

[0070] First, in response to a data write request initiated by the host for a specific logical address. To avoid the delay caused by performing compression operations on the write critical path, the data is not immediately compressed, but an intermediate address globally unique is first allocated for the data of this write. The intermediate address is the unique identity of this data within the system, which remains unchanged throughout its life cycle.

[0071] Subsequently, the first mapping relationship is immediately updated to establish a mapping between the logical address carried by the write request and the newly allocated intermediate address. This step ensures that subsequent data access through the logical address can correctly point to the corresponding intermediate address. At the same time, the uncompressed raw data is written into the cache storage area.

[0072] In the embodiments of the present application, the unique identification of the data is established by allocating an intermediate address, and the correspondence between the logical view and the internal identification is maintained by updating the first mapping relationship, and the temporary storage of data in the cache is used to achieve fast request response. This enables the subsequent real physical storage space management operations (data compression and flushing) to be executed efficiently as a background task without affecting the front-end I / O performance.

[0073] In some possible embodiments, referring to Figure 2 After writing the uncompressed data into the cache, the method further includes step 204, returning a write completion response to the host. After receiving the storage space management operation that causes the data physical address to change and updating the second mapping relationship, the method further includes step 205, compressing the uncompressed data in the cache and flushing to the back-end storage medium according to the new physical address.

[0074] After writing the uncompressed data into the cache, the method immediately performs the critical step of returning a write completion response to the host. This design is the core of the high-performance write implementation of the method. It means that the host does not need to wait for the data to go through the time-consuming compression process and finally persist to the disk. Once the data is safely stored in the cache, the system confirms the success of the write operation. This asynchronous processing mechanism completely decouples the front-end I / O path from the back-end resource-intensive operation, greatly shortens the response delay of the write operation, and significantly improves the overall throughput and user experience of the system.

[0075] In the subsequent phase, when the system initiates or receives a data compression flush storage space management operation in the background, the flow enters the physical storage processing phase. After updating the second mapping relationship and establishing a new physical address for the compressed data according to the foregoing steps, the method performs the following subsequent operation: the system reads the previously buffered uncompressed data from the cache, compresses it using the specified compression algorithm, and then writes or flushes the compressed data block to the back-end storage medium (such as a hard disk, an SSD, etc.) according to the new physical address recorded in the second mapping relationship.

[0076] In the embodiments of the present application, performance and efficiency are ingeniously balanced. The fast response mechanism guarantees low latency for front-end writing; and the background asynchronous compression and flush ensure efficient use of storage space. What is particularly important is that the intermediate address representing the unique identity of the data and the first mapping relationship connecting the user view remain unchanged throughout the process, and only the second mapping relationship needs to be updated after the physical location of the data is changed. This greatly reduces the size and complexity of the metadata operation, making the compression storage space management operation extremely efficient and lightweight.

[0077] The following specific examples clearly show the significant technical advantages brought by the method described in the present application in reducing metadata operation overhead when writing data.

[0078] In the traditional architecture before the introduction of the present method, the logical address is directly mapped to the physical address. When the physical storage location of the data changes due to an operation (for example, from physical address P old to P new), the system must update all logical mapping relationships pointing to the original physical address one by one. As shown in Figure 3 If the data block is shared by one source volume and n snapshot volumes, a total of n+1 mapping relationships in the source volume, snapshot 1, snapshot 2, and so on, need to be modified. Assuming that the metadata overhead of a single mapping relationship modification is k, the total overhead is (n+1)*k, and the complexity is O(n). The more logical entities that share the data block, the greater the metadata update operation overhead, which eventually becomes the main bottleneck of system performance.

[0079] After introducing the hierarchical mapping management method based on the intermediate address described in the present application, the metadata modification mechanism has fundamentally changed. At this time, the logical address is first mapped to a stable and unchanged intermediate address, and then mapped to the actual physical address from the intermediate address. When the physical location of the data needs to be changed, the system does not need to touch a large number of logical address to intermediate address mappings (first mapping relationship) in the upper layer, but only needs to modify the intermediate address to the new physical address mapping (second mapping relationship) once. As shown in Figure 4As shown in the middle, no matter how many snapshots exist, the modification point of metadata is only one place, i.e. updating the mapping of the compressed address (an implementation of the intermediate address) to the new physical address, and the total overhead is constant k, and the complexity is reduced to O(1).

[0080] In some possible embodiments, referring to Figure 5 , the method further comprises:

[0081] Step 501, in response to a data read request carrying a target logical address;

[0082] Step 502, querying a first mapping relationship to obtain a target intermediate address corresponding to the target logical address;

[0083] Step 503, taking the target intermediate address as an index to find corresponding uncompressed data in the cache;

[0084] Step 504, returning the found uncompressed data.

[0085] Firstly, in response to a data read request sent by a host, the request carrying a target logical address of data to be read. Then, a first mapping relationship is queried to obtain a target intermediate address corresponding to the target logical address. This step realizes the conversion from the logical address in the user's perspective to the system internal uniform identifier.

[0086] After obtaining the target intermediate address, the intermediate address is taken as a key index to find whether there is a corresponding uncompressed data copy in the cache. Since the data is first stored in the cache in an uncompressed form during the writing process, for newly written or frequently accessed data, this finding operation has a high hit probability.

[0087] If the corresponding uncompressed data is successfully found in the cache, the data is directly returned to the host.

[0088] In the embodiments of the application, the I / O operation required for reading compressed data from the back-end storage and the subsequent decompression calculation overhead are completely avoided, thereby greatly reducing the data read delay and improving the response speed and overall processing efficiency of the system. This mechanism is especially suitable for workloads with temporal locality access characteristics, so that hot data can be quickly accessed.

[0089] In some possible embodiments, referring to Figure 5 , after finding the corresponding uncompressed data in the cache, the method further comprises:

[0090] Step 505, if the corresponding uncompressed data is not found in the cache, querying a second mapping relationship to obtain a target physical address corresponding to the target intermediate address;

[0091] Step 506, read the compressed data from the target physical address in the backend storage medium, and return the decompressed data.

[0092] When the uncompressed data corresponding to the target intermediate address is not found in the cache, i.e., a cache miss, the alternative data reading path is automatically enabled to ensure the integrity and reliability of data access. Specifically, if the corresponding uncompressed data corresponding to the target intermediate address is not found in the cache, it indicates that the required data has not been cached or has been evicted. At this time, the second mapping relationship is queried to obtain the target physical address corresponding to the target intermediate address. This step realizes the conversion from stable data identifier to actual physical storage location, ensuring that even if the physical storage location of the data changes due to compression, migration, etc., the current storage location of the data can still be accurately located.

[0093] After obtaining the target physical address, the compressed data stored on the backend medium is read from the physical address, and then the decompression processing is performed on the compressed data, and the recovered original data is returned to the host.

[0094] In the embodiments of the present application, although this complete reading path increases I / O access and decompression calculation overhead compared to direct cache reading, it guarantees a 100% success rate of data access, so that high-performance reading can be provided without sacrificing data reliability and integrity. This layered reading mechanism (preferentially attempting cache access and then switching to backend storage access when failed) achieves the best balance between performance and reliability, and is suitable for various data access modes and workload scenarios.

[0095] In some possible embodiments, the storage space management operation is a defragmentation operation. The updating of the second mapping relationship, which maps the intermediate address to the changed new physical address, includes:

[0096] During the defragmentation process, in response to the operation of migrating the data from the original physical address to the new physical address, the intermediate address is mapped to the new physical address.

[0097] During the long-term operation of the storage system, frequent data writing, updating and deleting operations will cause a large number of discontinuous fragmented areas in the storage space, thereby reducing the storage space utilization and data access performance. The defragmentation operation is a background maintenance operation performed to solve this problem, which reorganizes the physical storage space through data migration, etc.

[0098] In the data defragmentation process, when the system decides to migrate a piece of data from its current fragmented physical address to a new continuous physical address, the core processing flow of the method is triggered. At this time, the system does not need to modify the mapping relationship held by any upper logical entity (such as the source volume or snapshot), but performs a key operation: establishing a mapping relationship between the intermediate address representing the unique identity of the data and the new physical address after migration.

[0099] This operation can be completed by updating the second mapping relationship. Specifically, the system updates the physical address entry corresponding to the intermediate address in the second mapping relationship from the original fragmented address to the new continuous physical address. This mechanism ensures that even if the physical storage location of the data changes after defragmentation, all upper logical views that reference the intermediate address through the first mapping relationship can automatically and transparently point to the latest physical location of the data without any modification.

[0100] In the embodiments of the present application, this processing mode brings significant technical advantages: the high complexity operation of updating a large number of logical mappings in the traditional defragmentation process is converted into a low complexity operation of updating only a single intermediate address mapping, greatly improving the execution efficiency of defragmentation, reducing system resource consumption, and ensuring data access continuity and consistency.

[0101] The following specific examples clearly show the significant technical advantages of the method described in the present application in reducing metadata operation overhead during defragmentation.

[0102] In the traditional storage architecture, the defragmentation process faces a serious metadata modification overhead problem. When a piece of data needs to be migrated from the fragmented original physical address to the continuous new physical address, the system must trace back and modify the mapping relationship of all logical entities that reference the data. As shown in Figure 6 If a piece of data is shared by a source volume and n snapshot volumes, the defragmentation needs to update a total of n mapping relationships in the source volume, snapshot 1, snapshot 2, and snapshot n. Assuming that the overhead of a single mapping modification is K, the total metadata overhead is n*K, and the complexity is O(n). This metadata modification mechanism proportional to the number of shares seriously restricts the efficiency of defragmentation and becomes a bottleneck for system performance improvement.

[0103] After adopting the layered mapping management method based on the intermediate address described in the embodiments of the present application, the metadata modification mechanism in the defragmentation process is fundamentally optimized. As shown in Figure 7As shown in the figure, by introducing a compressed address layer (i.e., an intermediate address layer), all upper-layer logical entities (source volumes and snapshots) refer to data through a unified intermediate address. When a defragmentation operation is performed, the system does not need to care about the sharing relationship of the upper-layer snapshots, and only needs to modify the mapping relationship of the intermediate address to the physical address once, and points the intermediate address to a new physical address to complete all updates. This reduces the amount of metadata modification from n*K in the traditional method to a constant K, and reduces the complexity from O(n) to O(1).

[0104] In some possible embodiments, the method further includes: calculating a globally unique identifier based on a content feature of the data, as the intermediate address.

[0105] Specifically, a globally unique identifier is generated by feature extraction and calculation of the content of the data through a specific algorithm. In a preferred embodiment, a cryptographic hash function (such as SHA-256, MD5, etc.) is used to calculate the content of the data to generate a fixed-length hash value as the intermediate address. This generation method ensures that different content data must produce different intermediate addresses, and the same content data must produce the same intermediate address, thereby naturally realizing the unique identification of data at the system level.

[0106] This method of generating an intermediate address based on content features brings important technical effects: first, the intermediate address is given strong semantic information, so that the address itself represents the content features of the data, rather than just a random identifier assigned by the system; second, this generation method naturally supports the data deduplication function, when different logical addresses write data with the same content, the system will calculate the same intermediate address, thereby automatically identifying duplicate data and implementing merging at the physical storage layer, further improving storage space utilization; finally, the reliability of the system is enhanced, any tampering or damage of data will cause the content features to change, thereby changing the intermediate address, facilitating the system to verify the data integrity.

[0107] Based on the same inventive concept, the present application also provides an address mapping management device, as shown in the figure, the device includes: Figure 8

[0108] The operation receiving module 801 is configured to receive a storage space management operation that causes a data physical address to change;

[0109] The address determining module 802 is configured to determine an intermediate address corresponding to a logical address of the data according to a first mapping relationship, wherein the intermediate address uniquely identifies the data;

[0110] The mapping relationship updating module 803 is configured to update a second mapping relationship, and map the intermediate address to a new physical address after the change; ​

[0111] The first mapping relationship is used to maintain mapping of logical addresses to intermediate addresses, and the second mapping relationship is used to maintain mapping of intermediate addresses to physical addresses.

[0112] In some possible embodiments, the storage space management operation is a data compression flush operation.

[0113] The apparatus further includes a data writing module 804 configured to, before receiving a storage space management operation resulting in a change of a physical address of data:

[0114] allocate the intermediate address for the data in response to a write request of the data, and update the first mapping relationship to establish mapping of the logical address corresponding to the request and the allocated intermediate address;

[0115] write the uncompressed data into the cache.

[0116] In some possible embodiments, the data writing module 804 is further configured to:

[0117] return a write completion response to the host after writing the uncompressed data into the cache.

[0118] after receiving the storage space management operation resulting in the change of the physical address of the data and updating the second mapping relationship, flush the uncompressed data in the cache to the backend storage medium according to the new physical address after compression.

[0119] In some possible embodiments, the apparatus further includes a data reading module 805 configured to:

[0120] in response to a data reading request carrying a target logical address;

[0121] query the first mapping relationship to obtain a target intermediate address corresponding to the target logical address;

[0122] find corresponding uncompressed data in the cache by taking the target intermediate address as an index;

[0123] return the found uncompressed data.

[0124] In some possible embodiments, the data reading module 805 is further configured to, after finding corresponding uncompressed data in the cache:

[0125] if the corresponding uncompressed data is not found in the cache, query the second mapping relationship to obtain a target physical address corresponding to the target intermediate address;

[0126] read compressed data from the target physical address in the backend storage medium, and return the decompressed data.

[0127] In some possible embodiments, the storage space management operation is a defragmentation operation.

[0128] The mapping relationship updating module 803 is configured to update a second mapping relationship, mapping the intermediate address to the changed new physical address, including:

[0129] In the defragmentation process, in response to the operation of migrating the data from the original physical address to the new physical address, the intermediate address is mapped to the new physical address.

[0130] In some possible embodiments, the first mapping relationship is shared by the source volume and at least one snapshot, so as to share the data by sharing the intermediate address.

[0131] The apparatus further includes an address generating module 800 configured to generate a globally unique identifier as the intermediate address based on the content feature of the data.

[0132] After introducing the address mapping management method and apparatus of the example embodiments of the present application, next, the address mapping management device according to another example embodiment of the present application is introduced.

[0133] Those skilled in the art can understand that various aspects of the present application can be implemented as a system, a method or a program product. Therefore, various aspects of the present application can be embodied as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" herein.

[0134] In some possible embodiments, the address mapping management device according to the present application can include at least one processor and at least one memory. The memory stores program code which, when executed by the processor, causes the processor to perform the steps of the address mapping management method according to various example embodiments of the present application described above.

[0135] The address mapping management device 960 according to this embodiment of the present application will be described below with reference to Figure 9 Figure 9 The displayed address mapping management device 960 is only an example and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0136] As Figure 9 ​As shown, the address mapping management device 960 is in the form of a general electronic device. The components of the address mapping management device 960 can include, but are not limited to, the at least one processor 961 described above, the at least one memory 962 described above, and a bus 963 that connects the various system components, including the memory 962 and the processor 961.

[0137] The bus 963 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.

[0138] The memory 962 can include a readable medium for storing data, such as volatile memory (RAM) 9621, and / or cache memory 9622, and can further include read-only memory (ROM) 9623.

[0139] The memory 962 can also include a program / utility 9625 having a set of programs / modules 9624, including an operating system, one or more application programs, other program modules, and program data, each or some combination thereof, which can include implementations of the network environment as described in each of the examples or some combination thereof.

[0140] The address mapping management device 960 can also communicate with one or more external devices 964 such as a keyboard or a pointing device, through an input / output (I / O) interface(s) 965. And, the address mapping management device 960 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the Internet, through a network adapter 966. As depicted, the network adapter 966 communicates with the other components of the address mapping management device 960 via the bus 963. It should be understood that although not shown, other hardware and / or software components could be used in conjunction with the address mapping management device 960. These include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0141] In some possible embodiments, various aspects of the address mapping management method provided by the present application can also be implemented as a program product, including a program code, which, when executed on a computer device, causes the computer device to perform the steps of the address mapping management method according to various exemplary embodiments of the present application described above in the specification.

[0142] The program product of the embodiments of the present application for address mapping management can employ a portable compact disc read-only memory (CD-ROM) and include a program code, and can be run on an electronic device. However, the program product of the present application is not limited thereto, and in the present document, a readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0143] The program product of the embodiments of the present application for address mapping management can employ a portable compact disc read-only memory (CD-ROM) and include a program code, and can be run on an electronic device. However, the program product of the present application is not limited thereto, and in the present document, a readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0144] The readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, in which a readable program code is carried. Such a propagated data signal can take on many forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The readable signal medium can also be any readable medium that can send, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device.

[0145] The program code contained in the readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, and the like, or any suitable combination thereof.

[0146] The program code may, through the use of program components, be implemented in any of various ways, including procedure-based execution, object-oriented execution, and / or virtual machine-based execution. A program component or other component may, for example, be implemented in hardware or software in combination with off-the-shelf components, or custom components. Additionally or alternatively, the program code may be implemented in transitory or non-transitory forms.

[0147] It should be noted that, although the above detailed description refers to several units or sub-units of the apparatus, such a division is merely illustrative and not mandatory. Indeed, according to an embodiment of the application, features and functions of two or more units described above can be embodied in one unit. Conversely, features and functions of one unit described above can be split into several units.

[0148] Moreover, while operations of the methods of the present application are described in a particular order in the figures, this is not required or implied in any particular order for performing the operations to achieve the desired results. Additionally or alternatively, certain steps can be omitted, combined into fewer steps, and / or split into multiple steps.

[0149] Those skilled in the art will appreciate that embodiments of the present application can be devised for a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code.

[0150] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and blocks in the flowcharts Figure 1 one or more flowcharts and blocks in the flowcharts

[0151] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and blocks in the flowcharts Figure 1 one or more flowcharts and blocks in the flowcharts

[0152] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and blocks in the flowcharts Figure 1 one or more flowcharts and blocks in the flowcharts

[0153] While the preferred embodiments of the application have been described, additional variations and modifications can be employed, as will be appreciated by those of ordinary skill in the art, once armed with the foregoing disclosure. Therefore, the following claims are intended to include all such modifications and variations as falling within the scope of the present application.

[0154] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. An address mapping management method, characterized in that, The method includes: Received a storage space management operation that causes a change in the physical address of the data; Based on the first mapping relationship, an intermediate address corresponding to the logical address of the data is determined, wherein the intermediate address uniquely identifies the data; Update the second mapping relationship to map the intermediate address to the new physical address after the change; The first mapping relationship is used to maintain the mapping from logical address to intermediate address, and the second mapping relationship is used to maintain the mapping from intermediate address to physical address.

2. The method according to claim 1, characterized in that, The storage space management operation is a data compression and flush operation; Before receiving a storage space management operation that causes a change in the physical address of the data, the method further includes: In response to the write request of the data, the intermediate address is allocated to the data, and the first mapping relationship is updated to establish a mapping between the logical address corresponding to the request and the allocated intermediate address; Write the uncompressed data to the cache.

3. The method according to claim 2, characterized in that, After writing the uncompressed data to the cache, the method further includes: returning a write completion response to the host; After receiving a storage space management operation that causes a change in the physical address of the data and updating the second mapping relationship, the method further includes: After compressing the uncompressed data in the cache, it is flushed back to the backend storage medium according to the new physical address.

4. The method according to claim 3, characterized in that, The method further includes: In response to a data read request carrying the target logical address; Query the first mapping relationship to obtain the target intermediate address corresponding to the target logical address; Using the target intermediate address as an index, search for the corresponding uncompressed data in the cache; Returns the found uncompressed data.

5. The method according to claim 4, characterized in that, After searching for the corresponding uncompressed data in the cache, the method further includes: If no corresponding uncompressed data is found in the cache, the second mapping relationship is queried to obtain the target physical address corresponding to the target intermediate address; In the backend storage medium, compressed data is read from the target physical address, decompressed, and then returned.

6. The method according to claim 1, characterized in that, The storage space management operation is a defragmentation operation; The update of the second mapping relationship, mapping the intermediate address to the changed new physical address, includes: During defragmentation, in response to the operation of migrating the data from the original physical address to the new physical address, a mapping is established between the intermediate address and the new physical address.

7. The method according to claim 1, characterized in that, The first mapping relationship is shared by the source volume and at least one snapshot to share the data by sharing the intermediate address.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: A globally unique identifier is calculated based on the content characteristics of the data and used as the intermediate address.

9. An address mapping management device, characterized in that, The device includes: The operation receiving module is used to receive storage space management operations that cause changes in the physical address of data; The address determination module is used to determine an intermediate address corresponding to the logical address of the data according to a first mapping relationship, wherein the intermediate address uniquely identifies the data; The mapping relationship update module is used to update the second mapping relationship, mapping the intermediate address to the new physical address after the change; The first mapping relationship is used to maintain the mapping from logical address to intermediate address, and the second mapping relationship is used to maintain the mapping from intermediate address to physical address.

10. An address mapping management device, characterized in that, The method includes at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-8.

11. A computer storage medium, characterized in that, The computer storage medium stores a computer program that enables the computer to perform the method as described in any one of claims 1-8.

Images