Data processing method, electronic device, storage medium, and program product

Abstract

The present application discloses a data processing method, electronic device, storage medium, and program product, relating to the field of computer technology. The method includes obtaining modified data after modifying snapshot data of a target source volume and obtaining a first snapshot relationship key-value pair corresponding to the target source volume, wherein the first snapshot relationship key-value pair includes a first physical block address, and based on the first physical block address, determining the first original data corresponding to the modified data, storing the first original data in a target storage location of a target terminal, and storing the modified data in a storage location in the target terminal corresponding to the first physical block address. The method of the present application reduces the problem of excessive storage resource occupation of the terminal itself, thereby saving storage resources.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, electronic device, storage medium, and program product. Background Art

[0002] With the rapid development of computer technology, data security has become a core element to ensure the smooth operation of business, and data snapshots, as an important backup technology, have been widely used.

[0003] In related technologies, after a terminal takes a snapshot of a source volume, the resulting snapshot data (the target volume) is stored in the terminal itself. If the data in the source volume is modified, the terminal will store both the source volume, the pre-modification snapshot data, and the modified snapshot data. This consumes a lot of the terminal's storage resources and affects the user experience. Summary of the Invention

[0004] The present application provides a data processing method, electronic device, storage medium and program product to at least solve the problem in the related art that the terminal body occupies a large amount of storage resources, affecting the user experience.

[0005] This application provides a data processing method, including:

[0006] Obtain modified data after modifying the snapshot data of the target source volume;

[0007] Obtaining a first snapshot relationship key-value pair corresponding to the target source volume, where the first snapshot relationship key-value pair includes a first physical block address;

[0008] Determining first original data corresponding to the modified data according to the first physical block address;

[0009] storing the first original data in a target storage location of a target terminal;

[0010] The modified data is stored in a storage location corresponding to the first physical block address in the target terminal.

[0011] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned data processing methods when executing the computer program.

[0012] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned data processing methods are implemented.

[0013] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned data processing methods when executed by a processor.

[0014] In the present application, by obtaining the modified data after modifying the snapshot data of the target source volume and obtaining the first snapshot relationship key-value pair corresponding to the target source volume, wherein the first snapshot relationship key-value pair includes the first physical block address, and according to the first physical block address, determining the first original data corresponding to the modified data, the first original data is stored in the target storage location of the target terminal, and the modified data is stored in the storage location corresponding to the first physical block address in the target terminal. The method of the present application, by obtaining the modified data after modifying the snapshot data of the target source volume and the first snapshot relationship key-value pair including the first physical block address, determines the original data corresponding to the modified data according to the physical block address, and stores the original data and the modified data in the corresponding locations of the target terminal respectively. Compared with the related art, after the source volume data is modified, the terminal body stores the source volume and the snapshot data before and after the modification at the same time, resulting in excessive storage resource occupation, the present application reduces the problem of excessive storage resource occupation of the terminal body, thereby saving storage resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0016] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0017] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application;

[0018] Figure 3 A flowchart of a method for storing first original data in a target storage location of a target terminal provided by an embodiment of the present application;

[0019] Figure 4 A flowchart of a method for deleting snapshot data provided in an embodiment of the present application;

[0020] Figure 5 A flowchart of a method for establishing a mapping relationship between a target volume LBA and a PBA provided in an embodiment of the present application;

[0021] Figure 6 A schematic diagram of space allocation for a target terminal provided in an embodiment of the present application;

[0022] Figure 7 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0023] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0025] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0026] As digital transformation accelerates, businesses are using more and more data. Data security and integrity have become core elements for ensuring business continuity. Key businesses require data traceability and rapid recovery, making efficient data backup a crucial requirement. Therefore, data snapshots, as a key technology for data backup, have gained widespread adoption.

[0027] Taking a data snapshot quickly and efficiently captures and saves the data state at a specific point in time, providing strong support for data protection, business continuity, and resource management. It generates data copies for backup and recovery without affecting the normal reading and writing of the original data. In the event of data loss, accidental deletion, or system failure, a rapid rollback to the historical state can be achieved, minimizing business interruption. In testing and development scenarios, creating test data based on snapshots ensures data authenticity while avoiding interference with production data. Therefore, snapshot technology, through incremental storage, can save storage resources, improve data management efficiency, and meet the multiple requirements of data security, availability, and flexibility.

[0028] In related technologies, a terminal, such as a first terminal, performs a snapshot of a source volume to obtain snapshot data, which can also be called a target volume. The snapshot data is then stored in the first terminal. If data in the source volume is modified, the terminal stores both the source volume and the snapshot data before and after the modification.

[0029] However, storing the source volume and snapshot data before and after modification in the main body will result in more local storage resources being occupied, affecting the user experience and increasing storage costs.

[0030] Therefore, in response to the problems in the related technology, it was found during the research process that if the first terminal only saves the source volume, the snapshot data corresponding to the source volume is stored in another target terminal different from the first terminal, such as a cluster, so that the cluster can store the data before and after the modification, which can effectively reduce the storage resources in the first terminal. Therefore, the present application proposes a data processing method, specifically, first obtaining the modified data of the target source volume snapshot data after modification, and at the same time obtaining the first snapshot relationship key-value pair corresponding to the source volume, wherein the first snapshot relationship key-value pair includes the first physical block address. With the help of the physical block address, the first original data corresponding to the modified data can be accurately located and stored in the target storage location of the target terminal, while the modified data is stored in the storage location corresponding to the first physical block address in the target terminal.

[0031] To facilitate understanding of the method of this application, an application scenario is provided below as an example. Figure 1 , Figure 1 An application scenario diagram provided for an embodiment of the present application includes a first terminal 01 and a target terminal 02, wherein the first terminal 01 may include a first controller 011, the target terminal 02 may be a cluster, and the cluster may include a storage device 021 and a second controller 022.

[0032] The first controller 011 performs snapshot processing on the target source volume to obtain snapshot data. If data modification occurs in the target source volume at this time, the first controller 011 obtains the modified data after modifying the snapshot data of the target source volume and the first snapshot relationship key-value pair corresponding to the target source volume, wherein the first snapshot relationship key-value pair includes the first physical block address. The first controller 011 determines the first original data corresponding to the modified data based on the first physical block address, and stores the first original data in the target storage location of the storage device 021 in the target terminal 02, and stores the modified data in the storage location corresponding to the first physical block address in the storage device 021. The target storage location can be determined by the second controller 022 in the target terminal 02 based on the pre-allocated storage space.

[0033] It is understandable that the above examples are only for illustration, and the present application does not limit the number, type, and function of the first terminal, the target terminal, the first controller, the second controller, and the storage device.

[0034] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0035] See Figure 2 , Figure 2 This is a flow chart of a data processing method provided in an embodiment of the present application. The execution subject of the method may be a data processing device, which may be implemented by a computer program; it may also be implemented by a medium storing a relevant computer program, such as a USB flash drive and / or an optical disk, or it may also be implemented by a physical device integrated or installed with a relevant computer program, such as an electronic device, which may be a smart terminal, etc. The method may include the following steps:

[0036] S201: Obtain modified data after modifying snapshot data of a target source volume.

[0037] In this embodiment, the execution subject takes the first terminal as an example. The first terminal may also be referred to as a production site. The production site obtains modified data after modifying the snapshot data of the target source volume.

[0038] The snapshot data, namely the target volume, is stored in the target terminal. In this embodiment, the target terminal takes a cluster as an example, and the cluster can also be called a remote site.

[0039] S202: Obtain a first snapshot relationship key-value pair corresponding to the target source volume, where the first snapshot relationship key-value pair includes a first physical block address.

[0040] In this embodiment, multiple snapshot key-value pairs for the target source volume are pre-created. These snapshot key-value pairs, also known as metadata, are composed of the form [K, V]. K represents key data, which includes identification information for the target source volume. This identification information may be, for example, a Universally Unique Identifier (UUID), the timestamp t when the snapshot was taken, and the Logical Block Address (LBA) corresponding to the target source volume. V represents value data, which includes the Physical Block Address (PBA) corresponding to the target source volume.

[0041] The production site determines a first snapshot relationship key-value pair according to the snapshot data to which the modified data belongs and the corresponding timestamp, where the value data in the first snapshot relationship key-value pair represents the first physical block address.

[0042] For example, the first snapshot relationship key-value pair [K1, V1] may be [UUID_1:t1:lba1,pba2], where lba1 represents the logical block address corresponding to the target source volume at time t1, and pba2 represents the first physical block address corresponding to the target source volume at time t1.

[0043] S203: Determine the first original data corresponding to the modified data according to the first physical block address.

[0044] At the remote site, target volumes at different timestamps corresponding to target source volumes are stored in different storage locations in the resource pool, and the storage locations correspond to physical block addresses.

[0045] Alternatively, the resource pool can be a Redundant Array of Independent Disks (RAID) created at a remote site using multiple disks. The RAID includes metadata space, data space, and unallocated space. The metadata space is used to store snapshot key-value pairs, and the data space is used to store snapshot data.

[0046] The production site determines the first original data data1 stored in the snapshot data space of the remote site according to the first physical block address. The first original data can also be understood as the old data before modification corresponding to the modified data.

[0047] One possible way to obtain the first original data is:

[0048] The production site determines the storage location of the target terminal, i.e., the remote site, corresponding to the first physical block address based on the first physical block address, and reads the target data stored in the storage location corresponding to the first physical block address. The target data is the first original data corresponding to the modified data.

[0049] S204: Store the first original data in a target storage location of the target terminal.

[0050] The production site sends the first original data to the target terminal so that the target terminal stores the first original data in a target storage location of the remote site, wherein the physical block address corresponding to the target storage location is another address different from the first physical block address.

[0051] S205: Store the modified data in a storage location corresponding to the first physical block address in the target terminal.

[0052] Since the production site moves the first original data from the storage location corresponding to the first physical block address in the target terminal, the storage location corresponding to the first physical block address is currently idle. Therefore, the production site can store the modified data in the storage location.

[0053] In snapshot scenarios, the first original data is migrated from the storage location corresponding to the first physical block address to a new location, and modified data is written to the original location, preserving snapshot integrity and the ability for multiple versions of data to coexist. This operation allows the original data state at the time of snapshot creation to be stored independently, and modifications do not affect the initial physical block address pointed to by the snapshot, ensuring the traceability of historical data. Only modified data is incrementally stored, reducing storage resource usage. This ensures data reliability while improving snapshot management flexibility and storage efficiency.

[0054] In this application, by storing snapshot data off-site at a target terminal, i.e., a remote site, a physically isolated redundant storage architecture is constructed, enabling data copies and target volumes to form a space-level disaster recovery backup with the source volumes. When a disaster occurs at the first terminal, such as a hardware failure or a natural disaster, the target volume, i.e., the snapshot data, stored at the target terminal maintains its integrity because it is independent of the first terminal's physical environment. This allows for rapid reconstruction and recovery at the first terminal based on the snapshot data stored at the target terminal after a disaster occurs.

[0055] In the above embodiment of the present application, by obtaining the modified data after the snapshot data of the target source volume is modified and obtaining the first snapshot relationship key-value pair corresponding to the target source volume, wherein the first snapshot relationship key-value pair includes the first physical block address, and according to the first physical block address, determining the first original data corresponding to the modified data, the first original data is stored in the target storage location of the target terminal, and the modified data is stored in the storage location corresponding to the first physical block address in the target terminal. The method of the present application, by obtaining the modified data after the snapshot data of the target source volume is modified and the first snapshot relationship key-value pair including the first physical block address, determines the original data corresponding to the modified data according to the physical block address, and stores the original data and the modified data in the corresponding locations of the target terminal respectively. Compared with the related art, after the source volume data is modified, the terminal body stores the source volume and the snapshot data before and after the modification at the same time, resulting in excessive storage resource occupation, this embodiment reduces the problem of excessive storage resource occupation of the terminal body, thereby saving storage resources.

[0056] Furthermore, based on the above embodiment, the process of storing the first original data in the target storage location of the target terminal is described through the following embodiment.

[0057] See Figure 3 , Figure 3 A flowchart of a method for storing first original data in a target storage location of a target terminal provided in an embodiment of the present application is provided. The method may include the following steps:

[0058] S301: Determine a first hash value of the first original data according to the first original data.

[0059] Optionally, based on a preset hash algorithm, the first original data is input into the hash algorithm, which maps input data of any length into a hash value of fixed length through a specific mathematical transformation, thereby determining the "fingerprint value" of the first original data, namely the first hash value hash1.

[0060] S302: Match the first Hash value in pre-established deduplication key-value pairs, where the deduplication key-value pairs include a Hash value and index information.

[0061] Deduplication key-value pairs are key structures used in data deduplication technology to identify and manage duplicate data blocks. The "key" is typically a unique hash value generated by a hash algorithm, serving as the data's "digital fingerprint," while the "value" records the index information of the data block in the storage system. Their core function is to quickly determine whether data is duplicated by comparing hash values. If duplicates are found, existing data is directly referenced to avoid duplicate storage.

[0062] Through data deduplication technology, only one physical copy of the duplicated parts of the target volume data generated by multiple snapshots is retained and stored in the storage pool. Different snapshots share the copy through logical mapping, thereby maximizing space utilization.

[0063] Because the hash value is unique, a match can be performed among pre-established deduplication key-value pairs based on the first hash value.

[0064] S303: If the first target deduplication key-value pair is not matched in the pre-established deduplication key-value pairs, a space application request is sent to the target terminal.

[0065] The space application request is used by the target terminal to generate a second physical block address to store the first original data.

[0066] If the first target deduplication key-value pair is not matched in the pre-established deduplication key-value pairs, it means that this modification is the first modification or the first original data has no deduplication data, and the production site sends a space application request to the remote site.

[0067] After receiving the space application request sent by the production site, the remote site determines the second physical block address pba2 from the pre-allocated data space.

[0068] When the first original data, data1, is not deduplicated, it means that there is no identical copy of the data in the storage pool, making it impossible to store it once and reference it multiple times through the deduplication mechanism. In data modification scenarios, to ensure the integrity and traceability of the first original data, such as for snapshot rollbacks, version management, or remote disaster recovery, the first original data must be migrated and saved before new data is written. Therefore, due to the lack of a reusable deduplicated copy, a new physical block address, pba2, must be applied for in the remote site storage pool A to independently store the first original data, thereby meeting the business requirements for data persistence and version management.

[0069] S304: Receive a second physical block address sent by the target terminal.

[0070] The remote site sends the second physical block address pba2 to the production site, so that the production site can receive the second physical block address pba2 sent by the remote site.

[0071] The production site and the remote site generate a new snapshot relationship key-value pair [K2, V2] = [UUID_1: t1: lba1, pba2] based on the second physical block address pba2. At the same time, the production site and the remote site generate a deduplication key-value pair [K'1, V'1] = [hash1, pba2:1]. The "1" in pba2:1 is the index count value, indicating that the data is mapped to multiple LBAs, and the first insertion count is 1.

[0072] S305: Store the first original data in a target storage location corresponding to the second physical block address in the target terminal.

[0073] The production site sends the first original data to the target terminal, so that the target terminal stores the first original data in a target storage location corresponding to the second physical block address pba2.

[0074] In this embodiment, if the production site matches the first target deduplication key-value pair among the pre-established deduplication key-value pairs, it indicates that the modified first original data data1 is deduplicated data and does not need to be sent to the target terminal. The production site then obtains target index information from the first target deduplication key-value pair, which includes the third physical block address pba3 and the first index count value count1.

[0075] A new first snapshot relationship key-value pair [K3, V3]=[UUID_1:t1:lba1,pba3] of the target source volume is generated according to the third physical block address pba3, the identification information UUID_1 of the target source volume, the snapshot timestamp information t1, and the logical block address lba1.

[0076] The first target deduplication key-value pair is updated based on the first index count value. Specifically, the first index count value count1 is added to a preset value, such as 1, to obtain a second index count value count1+1. The index count value in the first target deduplication key-value pair is updated to the second index count value, to obtain an updated first target deduplication key-value pair [K'2, V'2] = [hash1, pba2: count1+1].

[0077] In the above-mentioned embodiment of the present application, the first hash value of the first original data is determined based on the first original data, and a match is performed in a pre-established deduplication key-value pair based on the first hash value, wherein the deduplication key-value pair includes a hash value and index information. If the first target deduplication key-value pair is not matched in the pre-established deduplication key-value pair, a space application request is sent to the target terminal, and the space application request is used for the target terminal to generate a second physical block address to store the first original data, receive the second physical block address sent by the target terminal, and then store the first original data in the target storage location corresponding to the second physical block address in the target terminal. The method of this embodiment avoids duplicate storage by determining that there is no duplicate data in the first original data based on the first hash value.

[0078] In this application, when the production site wants to delete a snapshot of the target source volume stored in the target terminal, the following steps are required. Figure 4 , Figure 4 A flowchart of a method for deleting snapshot data provided in an embodiment of the present application includes:

[0079] S401: Acquire data to be deleted from a target source volume, where the data to be deleted is snapshot data of the target source volume.

[0080] Get the data to be deleted from the target source volume. Assume that the data to be deleted is the snapshot data of the target source volume (identified by UUID_1) at time t2.

[0081] S402: Determine the second original data corresponding to the data to be deleted according to the second snapshot relationship key-value pair corresponding to the target source volume, where the second snapshot relationship key-value pair includes a fourth physical block address.

[0082] For example, it is assumed that the second snapshot relationship key-value pair corresponding to the target source volume is [K4, V4]=[UUID_1:t2:lba2,pba4].

[0083] The second original data data3 is stored in the target terminal at a storage location corresponding to the fourth physical block address pba4.

[0084] The production site determines the second original data data3 stored in the snapshot data space of the remote site according to the fourth physical block address pba4.

[0085] S403: Determine a second target deduplication key-value pair from pre-established deduplication key-value pairs based on the second original data.

[0086] A second hash value of the second original data is determined based on the second original data, and a match is performed on a pre-established deduplication key-value pair based on the second hash value to obtain a second target deduplication key-value pair.

[0087] Optionally, based on a preset hash algorithm, the second original data is input into the hash algorithm, which will map the input data of any length into a hash value of fixed length through a specific mathematical transformation, thereby determining the "fingerprint value" of the second original data, that is, the second hash value hash2, and according to hash2, matching is performed in the pre-established deduplication key-value pair to obtain the second target deduplication key-value pair [K'3,V'3]=[hash2,pba4:count2].

[0088] S404: Determine whether to delete the second original data stored in the target terminal according to the second target deduplication key-value pair.

[0089] One possible implementation is:

[0090] Obtain the third index count value count2 in the second target deduplication key-value pair, subtract a preset value, such as 1, from the third index count value to obtain the fourth index count value count2-1, and determine whether to delete the second original data stored in the target terminal based on the fourth index count value count2-1.

[0091] Optionally, if the fourth index count value count2-1 is equal to a preset threshold, assuming the preset threshold is 0, it indicates that it is the last deduplication data index, and it is determined to delete the second original data data3 stored in the target terminal.

[0092] After determining to delete the second original data stored in the target terminal, the second original data data3 stored in the target terminal is deleted, and the second snapshot relationship key-value pair [K4, V4] = [UUID_1: t2: lba2, pba4] and the second target deduplication key-value pair [K'3, V'3] = [hash2, pba4: count2] are deleted.

[0093] In the deduplication mechanism, the index count value records how many target volumes reference the same data. When the index count value reaches 0, it means that no target volumes reference data3 in pba4, and the data has become "orphaned data." Deleting data3 at this time frees up physical storage space and reduces storage resource waste.

[0094] The second snapshot relationship key-value pair records the mapping between "lba2 and pba4 in snapshot t2." When data3 is deleted, this mapping becomes meaningless. If not deleted, invalid metadata will remain, causing incorrect references in subsequent queries. Deleting the key-value pair ensures that the metadata is consistent with the actual storage state.

[0095] The second target deduplication key-value pair stores the mapping between hash2 and pba4:count2. When count2 - 1 = 0, data3 corresponding to hash2 is no longer referenced by any data. Deleting this deduplication key-value pair prevents invalid entries from remaining in the deduplication index, ensuring that the deduplication mechanism can correctly determine whether new data is duplicated when it is subsequently written.

[0096] Optionally, if the fourth index count value count2-1 is greater than a preset threshold, assuming the preset threshold is 0, it means that there are other logical block addresses that have a mapping relationship with the data3, and it is determined not to delete the second original data data3 stored in the target terminal.

[0097] After determining not to delete the second original data stored in the target terminal, the second snapshot relationship key-value pair [K4, V4]=[UUID_1:t2:lba2, pba4] can also be deleted. According to the fourth index count value, the index count value in the second target deduplication key-value pair is updated to the fourth index count value, and the updated second target deduplication key-value pair is obtained, namely [K'4, V'4]=[hash2, pba4: count2-1].

[0098] In the above-mentioned embodiment of the present application, by obtaining the data to be deleted of the target source volume, the data to be deleted is the snapshot data of the target source volume, and according to the second snapshot relationship key-value pair corresponding to the target source volume, the second original data corresponding to the data to be deleted is determined, the second snapshot relationship key-value pair includes a fourth physical block address, and the second original data is stored in the target terminal at a storage location corresponding to the fourth physical block address. According to the second original data, a second target deduplication key-value pair is determined from a pre-established deduplication key-value pair, and according to the second target deduplication key-value pair, it is determined whether to delete the second original data stored in the target terminal. The method of this embodiment improves the accuracy of the result of determining whether to delete the second original data stored in the target terminal by using the second target deduplication key-value pair.

[0099] In this application, in order to improve its own space utilization and achieve efficient data management, the target terminal adopts a solution of uniformly storing the target volumes of multiple snapshots in a storage pool. Therefore, when the target volume of the snapshot is needed, the target terminal needs to materialize the target volume based on the snapshot metadata, that is, the snapshot relationship key-value pair. Specifically, based on the source volume identification information and timestamp included in K in the snapshot relationship key-value pair [K, V], it can be clearly determined which source volume is the snapshot created at which point in time. Then, based on the logical block address LBA and the physical block address PBA included in V, a mapping relationship between the target volume LBA and PBA is established to achieve the purpose of materializing the target volume.

[0100] When a source volume, for example, UUID=UUID_2, is needed to snapshot the corresponding target volume, such as the snapshot created at time t3, the target terminal needs to materialize the target volume based on the snapshot metadata information. Figure 5 , Figure 5 A flowchart of a method for establishing a mapping relationship between a target volume LBA and a PBA provided in an embodiment of the present application includes the following steps:

[0101] S501: Obtain identification information of a source volume and a timestamp when performing snapshot processing.

[0102] S502: Traverse all LBAs of the source volume to obtain LBA addresses.

[0103] For example, assume that the LBA address is lba3.

[0104] S503: Generate key data in the snapshot relationship key-value pair according to the identification information, timestamp, and LBA address of the source volume.

[0105] For example, K=UUID_2:t3:lba3.

[0106] S504: Query existing snapshot relationship key-value pairs according to the key data to determine corresponding value data.

[0107] Exemplarily, V=pba5.

[0108] S505: Establish a mapping relationship between key data and value data.

[0109] In the above-mentioned embodiment of the present application, a mapping relationship between the logical address and the physical block address is established through the UUID, timestamp, LBA and PBA information in the snapshot metadata, so as to realize the accurate positioning and calling of data, avoid the management chaos caused by the dispersion of data storage, and significantly improve the space utilization of the target terminal. The rapid materialization process is based on the rapid search of key-value pairs, which also effectively improves the data access efficiency.

[0110] In this application, the target terminal takes a cluster as an example, and can also dynamically adjust the storage space of the storage metadata and target volume. Assuming that the cluster consists of multiple control nodes, each control node will create an allocator Alloc, and the allocator of each node will centrally manage the storage space of all snapshots.

[0111] The target terminal divides the space in the storage pool into metadata space, data space, and unallocated space. For example, 1GB of space is allocated to the metadata space and 100GB of space is allocated to the data space during initialization. The remaining unallocated space is used for dynamic allocation when metadata and data space are insufficient.

[0112] Control nodes subscribe to each other's allocator events. For example, if control nodes Node1 and Node2 subscribe to each other, when a generation site sends a space request to control node Node1, and Node1 detects insufficient metadata / data space, it triggers a space request event for the unallocated area, completing the space request. This event is also subscribed to by control node Node2, triggering synchronization of allocator data on control node Node2 with the space allocation information on Node1, ensuring consistent allocator space information within the cluster.

[0113] When the allocated space exceeds the preset capacity value of the total capacity, for example, 80%, when generating a site deletion snapshot, the control node allocator that receives the request at the target terminal triggers a space call, releases the space back to unallocated, and simultaneously subscribes to trigger other node allocators to synchronize space allocation information.

[0114] In order to facilitate understanding of the above content, the following exemplary embodiments are used for illustration. Figure 6 , Figure 6 A schematic diagram of space allocation for a target terminal provided in an embodiment of the present application includes:

[0115] S601: The target terminal receives a space application request sent by a production site.

[0116] S602: The target terminal determines whether it is the first request based on the space application request.

[0117] S603: If it is the first request, allocate 1 GB of space for the metadata space and allocate 100 GB of space for the data space.

[0118] S604: Trigger subscription and synchronize space allocation information to other control nodes.

[0119] S605: If it is not the first request, determine whether the space application request is a write request.

[0120] S606: If it is a write request, determine whether the current space meets the space required by the write request.

[0121] S607: If satisfied, write metadata or snapshot data.

[0122] S608: If not satisfied, allocate space for metadata or snapshot data from the unallocated space, and then execute step S607 to write metadata or snapshot data, and then execute step S604.

[0123] S609: If it is not a write request, it is a delete request.

[0124] S610: Determine whether the current space exceeds 80% of the total space.

[0125] S611: If exceeded, release the space corresponding to the deletion request to the unallocated space, and then execute step S604.

[0126] S612: If it is not exceeded, release the space corresponding to the deletion request to the corresponding metadata space or data space, and then execute step S604.

[0127] In the above-mentioned embodiment of the present application, by dividing the storage pool into metadata, data and unallocated space, initial allocation is combined with dynamic adjustment to avoid static allocation waste, and unallocated space is replenished with metadata / data space on demand, thereby improving the efficiency of storage resource utilization. The control node synchronizes space allocation information in real time through an event subscription mechanism to ensure that the allocation status of each node is consistent, reducing space management conflicts in a distributed environment. Dynamic expansion when write requests trigger insufficient space and space recovery when delete requests trigger an over-threshold form flexible space management with on-demand allocation and excess release, thereby improving the flexibility and applicability of space allocation.

[0128] This application also provides a data processing device, see Figure 7 , Figure 7 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application includes:

[0129] The acquisition module 701 is used to acquire modified data after modifying the snapshot data of the target source volume.

[0130] The acquisition module 701 is configured to acquire a first snapshot relationship key-value pair corresponding to a target source volume, where the first snapshot relationship key-value pair includes a first physical block address.

[0131] The determination module 702 is configured to determine the first original data corresponding to the modified data according to the first physical block address.

[0132] The processing module 703 is configured to store the first original data in a target storage location of a target terminal.

[0133] The processing module 703 is configured to store the modified data in a storage location corresponding to the first physical block address in the target terminal.

[0134] In one possible implementation, the determining module 702 is specifically configured to:

[0135] According to the first physical block address, a storage location of a target terminal corresponding to the first physical block address is determined.

[0136] Target data stored at a storage location corresponding to the first physical block address is read, where the target data is first original data corresponding to the modified data.

[0137] In a possible implementation, the processing module 703 is specifically configured to:

[0138] A first hash value of the first original data is determined based on the first original data.

[0139] According to the first Hash value, a match is performed in a pre-established deduplication key-value pair, where the deduplication key-value pair includes a Hash value and index information.

[0140] If the first target deduplication key-value pair is not matched in the pre-established deduplication key-value pairs, a space application request is sent to the target terminal, where the space application request is used by the target terminal to generate a second physical block address to store the first original data.

[0141] Receive a second physical block address sent by the target terminal.

[0142] The first original data is stored in a target storage location corresponding to the second physical block address in the target terminal.

[0143] In one possible implementation, the first snapshot relationship key-value pair further includes identification information of the target source volume, snapshot timestamp information, and a logical block address. The processing module 703 is further configured to:

[0144] If a first target deduplication key-value pair is matched in the pre-established deduplication key-value pairs, target index information in the first target deduplication key-value pair is obtained, where the target index information includes a third physical block address and a first index count value.

[0145] A new first snapshot relationship key-value pair of the target source volume is generated according to the third physical block address, identification information of the target source volume, snapshot timestamp information, and the logical block address.

[0146] Update the first target deduplication key-value pair according to the first index count value.

[0147] In a possible implementation, the processing module 703 is further configured to:

[0148] The first index count value is accumulated plus a preset value to obtain a second index count value.

[0149] Update the index count value in the first target deduplication key-value pair to the second index count value.

[0150] In a possible implementation, the processing module 703 is further configured to:

[0151] Get the data to be deleted from the target source volume. The data to be deleted is the snapshot data of the target source volume.

[0152] According to the second snapshot relationship key-value pair corresponding to the target source volume, the second original data corresponding to the data to be deleted is determined, the second snapshot relationship key-value pair includes a fourth physical block address, and the second original data is stored in a storage location corresponding to the fourth physical block address in the target terminal.

[0153] According to the second original data, a second target deduplication key-value pair is determined from pre-established deduplication key-value pairs.

[0154] It is determined whether to delete the second original data stored in the target terminal according to the second target deduplication key-value pair.

[0155] In a possible implementation, the processing module 703 is further configured to:

[0156] A second hash value of the second original data is determined based on the second original data.

[0157] According to the second Hash value, a match is performed among pre-established deduplication key-value pairs to obtain a second target deduplication key-value pair.

[0158] In a possible implementation, the processing module 703 is further configured to:

[0159] Obtain a third index count value in the second target deduplication key-value pair.

[0160] The third index count value is subtracted from a preset value to obtain a fourth index count value.

[0161] It is determined whether to delete the second original data stored in the target terminal according to the fourth index count value.

[0162] In a possible implementation, the processing module 703 is further configured to:

[0163] If the fourth index count value is equal to the preset threshold, it is determined to delete the second original data stored in the target terminal.

[0164] If the fourth index count value is greater than the preset threshold, it is determined not to delete the second original data stored in the target terminal.

[0165] In one possible implementation, after determining to delete the second original data stored in the target terminal, the processing module 703 is further configured to:

[0166] The second original data stored in the target terminal is deleted.

[0167] Delete the second snapshot relationship key-value pair and the second target deduplication key-value pair.

[0168] In one possible implementation, after determining not to delete the second original data stored in the target terminal, the processing module 703 is further configured to:

[0169] Delete the second snapshot relationship key-value pair.

[0170] The second target deduplication key-value pair is updated according to the fourth index count value.

[0171] In a possible implementation, the processing module 703 is further configured to:

[0172] The index count value in the second target deduplication key-value pair is updated to the fourth index count value.

[0173] For the description of the features in the embodiment corresponding to the data processing device, reference can be made to the relevant description of the embodiment corresponding to the data processing method, which will not be repeated here.

[0174] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the electronic device provided by this embodiment includes: at least one processor 801 and a memory 802. Optionally, the electronic device further includes a communication component 803. The processor 801, the memory 802 and the communication component 803 are connected via a bus 804.

[0175] During the specific implementation process, at least one processor 801 executes the computer-executable instructions stored in the memory 802, so that the at least one processor 801 executes the above-mentioned data processing method embodiment.

[0176] The specific implementation process of the processor 801 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0177] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the application may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0178] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage.

[0179] A bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0180] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned data processing method embodiments when run.

[0181] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0182] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned data processing method embodiments are implemented.

[0183] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned data processing method embodiments are implemented.

[0184] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0185] The above is a detailed introduction to a data processing method, electronic device, storage medium, and program product provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A data processing method, characterized in that: include: Obtain modified data after modifying the snapshot data of the target source volume; Obtaining a first snapshot relationship key-value pair corresponding to the target source volume, where the first snapshot relationship key-value pair includes a first physical block address; determining, according to the first physical block address, first original data corresponding to the modified data; Determine a first hash value of the first original data according to the first original data; Matching a pre-established deduplication key-value pair according to the first hash value, where the deduplication key-value pair includes a hash value and index information; If the first target deduplication key-value pair is not matched in the pre-established deduplication key-value pairs, sending a space application request to the target terminal, wherein the space application request is used by the target terminal to generate a second physical block address for storing the first original data; receiving the second physical block address sent by the target terminal; storing the first original data in a target storage location corresponding to the second physical block address in the target terminal; The modified data is stored in a storage location corresponding to the first physical block address in the target terminal.

2. The method according to claim 1, characterized in that The determining, according to the first physical block address, first original data corresponding to the modified data includes: determining, according to the first physical block address, a storage location of a target terminal corresponding to the first physical block address; Target data stored at a storage location corresponding to the first physical block address is read, where the target data is first original data corresponding to the modified data.

3. The method according to claim 2, characterized in that The first snapshot relationship key-value pair also includes identification information of the target source volume, snapshot timestamp information, and a logical block address. The method further includes: If the first target deduplication key-value pair is matched in the pre-established deduplication key-value pair, obtaining target index information in the first target deduplication key-value pair, the target index information including the third physical block address and the first index count value; generating a new first snapshot relationship key-value pair for the target source volume according to the third physical block address, the identification information of the target source volume, the snapshot timestamp information, and the logical block address; The first target deduplication key-value pair is updated according to the first index count value.

4. The method according to claim 3, characterized in that Updating the first target deduplication key-value pair according to the first index count value includes: Accumulate a preset value by adding the first index count value to obtain a second index count value; Update the index count value in the first target deduplication key-value pair to the second index count value.

5. The method according to claim 1, wherein Also includes: Acquire the data to be deleted of the target source volume, where the data to be deleted is snapshot data of the target source volume; determining, based on a second snapshot relationship key-value pair corresponding to the target source volume, second original data corresponding to the data to be deleted, the second snapshot relationship key-value pair including a fourth physical block address, and the second original data being stored in a storage location corresponding to the fourth physical block address in the target terminal; determining, based on the second original data, a second target deduplication key-value pair from pre-established deduplication key-value pairs; Determine whether to delete the second original data stored in the target terminal according to the second target deduplication key-value pair.

6. The method according to claim 5, characterized in that The determining, based on the second original data, a second target deduplication key-value pair from pre-established deduplication key-value pairs includes: determining a second hash value of the second original data according to the second original data; According to the second Hash value, a match is performed in pre-established deduplication key-value pairs to obtain a second target deduplication key-value pair.

7. The method according to claim 5, characterized in that The determining whether to delete the second original data stored in the target terminal according to the second target deduplication key-value pair includes: Obtaining a third index count value in the second target deduplication key-value pair; Subtract a preset value from the third index count value to obtain a fourth index count value; It is determined whether to delete the second original data stored in the target terminal according to the fourth index count value.

8. The method according to claim 7, characterized in that The determining, according to the fourth index count value, whether to delete the second original data stored in the target terminal includes: If the fourth index count value is equal to a preset threshold, determining to delete the second original data stored in the target terminal; If the fourth index count value is greater than a preset threshold, it is determined not to delete the second original data stored in the target terminal.

9. The method according to claim 8, characterized in that After determining to delete the second original data stored in the target terminal, the method further includes: deleting the second original data stored in the target terminal; Delete the second snapshot relationship key-value pair and the second target deduplication key-value pair.

10. The method according to claim 8, characterized in that After determining not to delete the second original data stored in the target terminal, the method further includes: Deleting the second snapshot relationship key-value pair; Update the second target deduplication key-value pair according to the fourth index count value.

11. The method according to claim 10, characterized in that Updating the second target deduplication key-value pair according to the fourth index count value includes: The index count value in the second target deduplication key-value pair is updated to the fourth index count value.

12. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the data processing method according to any one of claims 1 to 11 when executing the computer program.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the data processing method according to any one of claims 1 to 11.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 11 are implemented.

Images