Data operation method, apparatus, device, and storage medium

Abstract

The present disclosure provides a data operation method and device, equipment and storage medium, relates to the field of artificial intelligence, in particular to the technical field of cloud computing, cloud storage, cloud network and cloud database. The specific implementation scheme is: in response to a data operation request, determining a target logical area to which target data to be operated belongs and a target logical block to which the target logical area belongs; determining a target physical group associated with the target logical block based on a mapping relationship between candidate logical blocks cached locally and candidate physical groups; wherein each candidate logical block includes at least two logical areas; each candidate physical group includes at least two physical segments; selecting a target physical segment associated with the target logical area from at least two physical segments included in the target physical group; and performing data operation on a storage space where the target physical segment is located. The present disclosure can improve the execution efficiency of data operation.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and in particular to the fields of cloud computing, cloud storage, cloud networks and cloud databases, specifically to a data operation method, apparatus, device and storage medium. Background Technology

[0002] With the development of database systems, data operations are becoming increasingly frequent. Currently, in database systems, user-created files are logically divided into multiple parts according to a specified length. Logically, data operations have a minimum operation length, which can be understood as follows: regardless of the actual length of the data being operated on, data operations must be performed on at least the minimum operation length. Summary of the Invention

[0003] This disclosure provides a data manipulation method, apparatus, device, and storage medium.

[0004] According to one aspect of this disclosure, a data manipulation method is provided, executed by a client, comprising:

[0005] In response to a data operation request, determine the target logical area to which the target data to be operated belongs and the target logical block to which the target logical area belongs;

[0006] Based on the mapping relationship between candidate logical blocks and candidate physical groups in the local cache, a target physical group associated with the target logical block is determined; wherein, each candidate logical block includes at least two logical regions; and each candidate physical group includes at least two physical segments;

[0007] Select the target physical segment associated with the target logical region from at least two physical segments included in the target physical group;

[0008] Perform data operations on the storage space where the target physical segment is located.

[0009] According to one aspect of this disclosure, a data manipulation apparatus is provided, executed by a client, comprising:

[0010] The target logic block determination module is used to determine the target logic area to which the target data to be operated belongs and the target logic block to which the target logic area belongs in response to a data operation request.

[0011] The target physical group determination module is used to determine the target physical group associated with the target logical block based on the mapping relationship between candidate logical blocks and candidate physical groups in the local cache; wherein each candidate logical block includes at least two logical areas; and each candidate physical group includes at least two physical segments;

[0012] The target physical segment selection module is used to select a target physical segment associated with the target logical area from at least two physical segments included in the target physical group;

[0013] The data operation module is used to perform data operations on the storage space where the target physical segment is located.

[0014] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the data manipulation method described in any embodiment of this disclosure.

[0018] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the data manipulation method described in any embodiment of this disclosure.

[0019] According to another aspect of this disclosure, a computer program product is provided, wherein the computer program, when executed by a processor, implements the data manipulation method described in any embodiment of this disclosure.

[0020] The embodiments disclosed herein can improve the execution efficiency of data operations.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0022] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0023] Figure 1 This is a flowchart of a data manipulation method disclosed in an embodiment of the present disclosure;

[0024] Figure 2 This is a schematic diagram of a data operating system disclosed in an embodiment of the present disclosure.

[0025] Figure 3 This is a flowchart of another data manipulation method disclosed according to an embodiment of this disclosure;

[0026] Figure 4This is a schematic diagram of the structure of a physical segment according to an embodiment of the present disclosure;

[0027] Figure 5 This is a flowchart of another data manipulation method disclosed according to an embodiment of this disclosure;

[0028] Figure 6 This is a scenario diagram of a data manipulation method disclosed in an embodiment of this disclosure;

[0029] Figure 7 This is a schematic diagram of the structure of a data manipulation device disclosed in an embodiment of the present disclosure;

[0030] Figure 8 This is a block diagram of an electronic device used to implement the data operation method of the embodiments of this disclosure. Detailed Implementation

[0031] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0032] Figure 1 This is a flowchart of a data manipulation method disclosed in an embodiment of the present disclosure. This embodiment can be applied to situations involving data manipulation. The method of this embodiment can be executed by a data manipulation device, which can be implemented in software and / or hardware and specifically configured in an electronic device with a certain data processing capability. The electronic device can be a client, which may include mobile phones, tablets, vehicle terminals, and desktop computers, etc.

[0033] See Figure 1 The data manipulation methods, executed by the client, include:

[0034] S101. In response to a data operation request, determine the target logical area to which the target data to be operated belongs and the target logical block to which the target logical area belongs.

[0035] S102. Based on the mapping relationship between candidate logical blocks and candidate physical groups in the local cache, determine the target physical group associated with the target logical block; wherein, each candidate logical block includes at least two logical areas; each candidate physical group includes at least two physical segments.

[0036] S103. Select the target physical segment associated with the target logical area from at least two physical segments included in the target physical group.

[0037] S104. Perform data operations on the storage space where the target physical segment is located.

[0038] A data operation request can be used to trigger the device to perform corresponding data operations on target data. Optionally, the data operation request can be triggered by a user. The content of the data operation request can include the logical storage location of the target data and the data operation to be performed on the target data. The target data can be the data to be operated on. The logical storage location can be a logically defined storage location. Optionally, the logical storage location can include at least one of the following: the database (or cloud storage space) to which the data belongs, the logical block to which the data belongs in the database (or target cloud storage space), and the logical area to which the data belongs in the logical block. The target logical area can be the logical area to which the target data belongs. The target logical block can be the logical block to which the target data belongs. The target logical block can include a target logical area and at least one other logical area. The logical storage location of the target data can be used to identify the target data. For example, the target data is data in logical area A of logical block X, and the storage location of the target data is logical area A in logical block X. The target data can be identified by its logical storage location to distinguish it from other data. Data operations can include data querying, data storage, or data editing. Data editing can include data deletion or data modification.

[0039] Optionally, in response to a data operation request, the target logical area to which the target data to be operated belongs and the target logical block to which the target logical area belongs can be directly extracted from the data operation request.

[0040] Optionally, in response to a data operation request, the target logical region to which the target data to be operated belongs can be directly extracted from the data operation request. The target logical block to which the target logical region belongs can be determined based on the partitioning relationship between logical blocks and logical regions in the metadata management cluster.

[0041] Optionally, in response to a data operation request, the target logical region to which the target data to be operated belongs can be directly extracted from the data operation request. Alternatively, the target logical block to which the target logical region belongs can be determined based on the division relationship between logical regions and logical blocks in the local cache.

[0042] Candidate logical blocks and candidate physical groups represent storage locations for the same data from different dimensions. A candidate logical block can be a logical storage location for the data, which can be understood as a logically partitioned storage location. A candidate logical block is a virtual storage location, not an actual storage location. A candidate physical group can be a physical storage location for the data, which can be understood as the actual storage location of the data. Each candidate logical block can include at least two logical regions. Each candidate physical group can include at least two physical segments. The number of logical regions included in a candidate logical block and the number of physical segments included in a candidate physical group can be the same or different. This is determined and adjusted based on the experience of technical personnel. The mapping relationship between candidate logical blocks and candidate physical groups can be used to characterize the association between the logical storage location of the data and the physical storage location in the storage space. The mapping relationship between candidate logical blocks and candidate physical groups is persistent, meaning that the mapping relationship between candidate logical blocks and candidate physical groups remains unchanged. The mapping relationship between candidate logical blocks and candidate physical groups can be stored in a local cache for easy querying. The target physical group can be a physical group that has a mapping relationship with the target logical block.

[0043] Specifically, based on the mapping relationship between candidate logical blocks and candidate physical groups in the local cache, the target logical block and the target physical group with which it is mapped can be queried in the local cache according to the block identifier of the target logical block. The block identifier of the target logical block can be used to uniquely identify the target logical block. For example, the identifier of the target logical block may include at least one of the following: the name, abbreviation, sequence number, and code of the target logical block.

[0044] A target logical block comprises at least two logical regions. This can be understood as a target logical block comprising a target logical region and at least one other logical region. A target physical group comprises at least two physical segments. This can be understood as a target physical group comprising a target physical segment and at least one other physical segment. There is also a mapping relationship between the target logical region and the target physical segment. A target physical segment can be a physical segment that has a mapping relationship with a target logical region.

[0045] Optionally, the number of physical segments contained in the target physical group can be the same as the number of logical regions contained in the target logical block. When establishing the mapping relationship between physical segments in the target physical group and logical regions in the target logical block, a one-to-one correspondence can be established between physical segments and logical regions with the same identifier. Accordingly, physical segments with the same identifier as the target logical region can be selected from the physical segments contained in the target physical group and determined as the target physical segments associated with the target logical region.

[0046] Optionally, the number of physical segments contained in the target physical group can differ from the number of logical regions contained in the target logical block. Alternatively, a mapping relationship can be established between each physical segment in the target physical group and each logical block in the target logical block according to preset association rules. Correspondingly, target physical segments associated with the target logical region can be selected from the physical segments included in the target physical group according to preset association rules.

[0047] Specifically, based on the data operation request, the corresponding data operation can be determined, and data operations can be performed on the storage space where the target physical segment is located.

[0048] Optionally, if the data operation request is a data deletion request, the data to be deleted can be deleted from the storage space where the target physical segment resides. The target data can be the data to be deleted.

[0049] Optionally, when the data operation request is a data modification request, the original data in the storage space where the target physical segment resides can be deleted, and the data to be modified can be written to the storage space where the target physical segment resides. Correspondingly, the content of the data modification request also includes the data to be modified. The target data can be the original data in the storage space.

[0050] In an optional embodiment of the present invention, performing data operations on the storage space where the target physical segment is located includes: when the data operation request is a data storage request, writing the data to be stored into the storage space where the target physical segment is located.

[0051] The data storage request also includes the data to be stored.

[0052] Specifically, when the data operation request is a data query request, the data to be stored can be determined based on the data query request, and the data to be stored can be written into the storage space where the target physical segment is located.

[0053] By writing the data to be stored into the storage space where the target physical segment is located when the data operation request is a data storage request, the writing and storage of data is realized, and the efficiency of data writing and storage is improved.

[0054] Figure 2 This is a schematic diagram of the data operating system provided according to the technical solution of this disclosure. For example... Figure 2As shown, the data operating system can include three parts: a storage cluster, a metadata management cluster, and a client. The storage cluster consists of multiple storage nodes (i.e., storage servers). Storage nodes manage and store physical segments. Each storage node has multiple disks, and each disk contains multiple physical segment files. When storage capacity is insufficient, storage nodes can be added for expansion; when a storage node fails, it can be removed from the system. The metadata management cluster manages and stores the persistent mapping relationship between logical areas and physical segments. For example, logical area 1 of cloud disk 1 is stored in physical segment 1, and logical area 2 of cloud disk 2 is stored in physical segment 3. The client can be used to proxy data operations. In existing technology, a persistent mapping relationship is established between logical areas and physical segments, and this mapping relationship is stored in the metadata management cluster. When operating on data, the client first queries the metadata management cluster, i.e., queries the mapping relationship between logical areas and physical segments, and then performs data operations on the storage nodes in the storage cluster, i.e., performs data operations on the corresponding physical segments. Because the number of logical areas and their corresponding physical segments is very large, the number of mapping relationships between logical areas and physical segments is also very large. Each time a data operation is performed, a large number of mapping relationships between logical areas and physical segments need to be queried in real time in the metadata management cluster, which increases the access latency of the metadata management cluster and makes the data operation inefficient. At the same time, the metadata management cluster is prone to reaching its bottleneck due to the large number of real-time query operations performed on it.

[0055] According to the technical solution of this disclosure, in response to a data operation request, the target logical region to which the target data to be operated belongs and the target logical block to which the target logical region belongs are determined. Based on the mapping relationship between candidate logical blocks and candidate physical groups in the local cache, the target physical group associated with the target logical block is determined. From at least two physical segments included in the target physical group, the target physical segment associated with the target logical region is selected. Data operations are performed on the storage space where the target physical segment is located. By performing coarse-grained persistent mapping between candidate logical blocks and candidate physical groups, compared with the existing technology that directly performs fine-grained mapping between logical regions and physical segments, the amount of mapping relationship storage is reduced, and local caching storage of mapping relationships is realized. During the data operation process, there is no need to query the metadata management cluster, avoiding access latency of the metadata management cluster. At the same time, it also reduces the number of mapping relationship queries during the data operation process, improving the efficiency of data operation. In addition, it also reduces the additional query pressure on the metadata management cluster and avoids the bottleneck of the metadata management cluster.

[0056] Figure 3This is a flowchart of another data operation method disclosed in the embodiments of this disclosure, which is further optimized and extended based on the above technical solution, and can be combined with the above optional implementation methods. Specifically, determining the target logical region to which the target data to be operated belongs and the target logical block to which the target logical region belongs is as follows: when the data operation request is a data query request, determine the target logical page to which the target data to be operated belongs; determine the target logical region to which the target logical page belongs based on the division relationship between the logical regions and logical pages in the local cache; and determine the target logical block to which the target logical region belongs based on the division relationship between the logical blocks and logical regions in the local cache.

[0057] See Figure 3 The data manipulation methods, executed by the client, include:

[0058] S301. In response to a data operation request, if the data operation request is a data query request, determine the target logical page to which the target data to be operated belongs.

[0059] S302. Based on the partitioning relationship between logical areas and logical pages in the local cache, determine the target logical area to which the target logical page belongs.

[0060] S303. Based on the division relationship between logical blocks and logical areas in the local cache, determine the target logical block to which the target logical area belongs.

[0061] S304. Based on the mapping relationship between candidate logical blocks and candidate physical groups in the local cache, determine the target physical group associated with the target logical block; wherein, each candidate logical block includes at least two logical areas; each candidate physical group includes at least two physical segments.

[0062] S305. Select the target physical segment associated with the target logical area from at least two physical segments included in the target physical group.

[0063] S306. Perform data operations on the storage space where the target physical segment is located.

[0064] In a database system, a database (or cloud storage) file can be created. For a single database (or cloud storage) file, it can be logically divided according to a preset logical area length, resulting in at least two logical areas. For each logical area, it can be logically divided according to a preset logical page length, resulting in at least two logical pages. Logical areas can be grouped according to a preset number of logical areas, resulting in logical blocks. The preset logical area length is greater than the preset logical page length. Optionally, the preset logical area length can be at the MB level; the preset logical page length can be at the KB level, for example, a few KB to tens of KB. For example, the preset logical page length can be 4KB; the preset logical area length can be 1MB; and the preset number of logical areas can be 128. The preset number of logical areas, the preset logical page length, and the preset logical area length can be set and adjusted based on the experience of technical personnel. Logical blocks, logical areas, and logical pages are all logical storage locations with different granularities. The preset logical page length corresponding to a logical page can be the minimum operation length for data operations. This can be understood as meaning that regardless of the actual length of the data being operated on, at least one logical page must be used for data operations. The target logical page can be the logical page to which the target data belongs. The content of a data query request can include an identifier for the target logical page. For example, a data query request could be to read data from page 4 of database 1.

[0065] Specifically, in response to a data operation request, if the data operation request is a data query request, the target logical page to which the target data to be operated belongs is determined based on the identifier of the target logical page contained in the data query request.

[0066] The division relationship between logical regions and logical pages can be pre-stored in the local cache, making it easy to quickly determine the logical region to which a logical page belongs.

[0067] Specifically, the target logical area to which the target logical page belongs can be determined based on the division relationship between the logical areas and logical pages in the local cache and the identifier of the target logical page.

[0068] The division relationship between logical blocks and logical regions can be pre-stored in the local cache, making it easy to quickly determine the logical block to which a logical region belongs.

[0069] Specifically, the target logical block to which the target logical area belongs can be determined based on the division relationship between the logical blocks and logical areas in the local cache and the identifier of the target logical area.

[0070] In an optional embodiment of this disclosure, data operations are performed on the storage space where the target physical segment is located, including: when the data operation request is a data query request, obtaining the backup storage node of the target physical segment; sending the target physical segment and the data operation request to any backup storage node, so that the backup storage node determines the target data to be queried; and obtaining the target data fed back by the backup storage node.

[0071] The actual physical content of database (or cloud storage) files can be stored in a storage system. Managing or storing data in a storage system essentially involves managing or storing at least two physical segments within that system. To ensure data reliability, the storage system can be a distributed storage system, with each physical segment's data distributed across at least two backup storage nodes. Backup storage nodes are used to back up the data in a physical segment. For example, a target physical segment can have three replicas, each located on a different backup storage node. The data in each backup storage node is completely identical. The mapping between physical segments and backup storage nodes can be stored in a metadata management cluster or a local cache. For instance, three replicas of physical segment X can be stored on backup storage nodes U, V, and W, respectively. The mapping between physical segment X and backup storage nodes U, V, and W can be stored in a metadata management cluster or a local cache.

[0072] Based on the preset physical area length, a physical segment can be divided into at least two physical areas. Optionally, the length of the physical area can be the same as the length of the logical area. The physical area stores the actual physical content of a specific logical area. Figure 4 This is a schematic diagram of the physical segment structure provided according to the technical solution of this disclosure. For example... Figure 4 As shown, a single physical segment can include a data file and a corresponding index file. The data file can be used to store the actual physical content of logical areas in different databases (or cloud storage spaces); the index file can be used to store the mapping relationship between logical areas and physical areas. It can be understood that the content recorded in the index file indicates which logical area in which database (or cloud storage space) is stored in a specific physical area of ​​the physical segment. For example, for index 1, it records the actual physical content of the first logical area in database 1 stored in the first physical area of ​​this physical segment.

[0073] Optionally, the physical area can be divided into at least two physical pages according to a preset physical page length. Optionally, the length of a physical page can be the same as the length of a logical page. A physical page stores the actual physical content of a specific logical page. Correspondingly, data files can also be used to store the actual physical content of logical pages in different database (or cloud storage) files; index files can also be used to store the mapping relationship between logical pages and physical pages. It can be understood that the records in the index file represent the actual physical content of which logical page in which database (or cloud storage) is stored in a specific physical page of a physical segment. For example, for index 1, it records the actual physical content of the first logical page in database 1 stored within the first physical page of that physical segment.

[0074] Optionally, the target data to be queried may include data stored in a physical segment, data stored in a physical area, or data stored in a physical page. Specifically, it can be determined according to the data query request. For example, if the data query request is to query the data of the t-th logical block in cloud disk H, the corresponding target data to be queried can be the data of the target physical segment; if the data query request is to query the data of the t-th logical area in cloud disk H, the corresponding target data to be queried can be the data of the target physical area; if the data query request is to query the data of the t-th logical page in cloud disk H, the corresponding target data to be queried can be the data of the target physical page.

[0075] Specifically, when the data operation request is a data query request, the mapping relationship between physical segments and backup storage nodes can be obtained from the metadata management cluster or local cache to determine the backup storage node for the target physical segment. The target physical segment and data operation request can be sent to any backup storage node, enabling that backup storage node to determine the target data to be queried based on the data query request and return it to the client. The client can then retrieve the target data returned by the backup storage node.

[0076] By distributing the data in the target physical segment, the reliability of the data in the target physical segment is further improved. When the data operation request is a data query request, the backup storage node of the target physical segment is obtained, and the target physical segment and data operation request are sent to any backup storage node. This balances the accuracy of data query and the flexibility of backup storage node selection during the data query process. At the same time, the adaptability between the target data and the data query request is ensured through the data query request, which further improves the accuracy of the data query results.

[0077] According to the technical solution of this disclosure, when the data operation request is a data query request, the target logical page to which the target data to be operated belongs is determined. Based on the partitioning relationship between the logical area and the logical page in the local cache, the target logical area to which the target logical page belongs is determined. Based on the partitioning relationship between the logical block and the logical area in the local cache, the target logical block to which the target logical area belongs is determined. By pre-dividing the logical storage location into three granularities and storing the partitioning relationship between the logical area and the logical page, as well as the partitioning relationship between the logical area and the logical block in the local cache, when performing a data query, the logical area to which the logical page belongs, and the logical block to which the logical area belongs, can be quickly identified based on the partitioning relationship in the local cache. This reduces the query time for partitioning relationships and further improves the efficiency of data query.

[0078] Figure 5 This is a flowchart of another data operation method disclosed in the embodiments of this disclosure, which is further optimized and extended based on the above technical solution, and can be combined with the above optional implementation methods. The mapping relationship between candidate logical blocks and candidate physical groups can be determined in the following way: using a first hash algorithm, based on the block identifier of the candidate logical block and the total number of candidate physical groups, the candidate physical groups associated with the candidate logical block are determined, and the mapping relationship between the candidate logical block and the associated candidate physical groups is obtained.

[0079] See Figure 5 The data manipulation methods, executed by the client, include:

[0080] S501. In response to a data operation request, determine the target logical area to which the target data to be operated belongs and the target logical block to which the target logical area belongs.

[0081] S502. Based on the mapping relationship between candidate logical blocks and candidate physical groups in the local cache, determine the target physical group associated with the target logical block; wherein, each candidate logical block includes at least two logical areas; each candidate physical group includes at least two physical segments; wherein, the mapping relationship between candidate logical blocks and candidate physical groups is determined in the following way: using the first hash algorithm, based on the block identifier of the candidate logical block and the total number of candidate physical groups, determine the candidate physical group associated with the candidate logical block, and obtain the mapping relationship between the candidate logical block and the associated candidate physical group.

[0082] S503. Select the target physical segment associated with the target logical area from at least two physical segments included in the target physical group.

[0083] S504. Perform data operations on the storage space where the target physical segment is located.

[0084] A block identifier can be used to uniquely identify a candidate logical block. Optionally, the block identifier may include at least one of the following: block name, block abbreviation, block sequence number, and block code.

[0085] Specifically, a first hash algorithm can be used to calculate the block identifier of the candidate logical block and the total number of candidate physical groups to determine the group identifier of the candidate physical group associated with the candidate logical block, thus obtaining the mapping relationship between the candidate logical block and the associated candidate physical group. The first hash algorithm can include hash algorithms, consistent hashing algorithms, or Crush (Controlled Replication Under Scalable Hashing) algorithms, etc.

[0086] For example, the group identifier of a candidate physical group can be calculated using the following formula:

[0087] Hash(Chunk O) = O%P;

[0088] Where Hash is the first hash algorithm; Chunk O is the candidate logical block; O is the block identifier of the candidate logical block; and P is the total number of candidate physical groups.

[0089] In an optional embodiment of this disclosure, the method further includes: in the case of physical segment expansion, obtaining the newly added physical groups and updating the total number of physical groups based on the newly added physical groups; wherein the newly added physical groups include at least two physical segments; in the case of obtaining a new logical block, using a first hash algorithm, determining the candidate physical groups associated with the new logical block based on the block identifier of the new logical block and the updated total number of physical groups, and obtaining the mapping relationship between the new logical block and the associated candidate physical groups; and persistently storing the mapping relationship between the new logical block and the associated candidate physical groups in the metadata management cluster.

[0090] When the storage capacity of a storage system is insufficient, expansion is necessary. Current expansion technologies involve directly adding new physical segments. However, when persistent mappings are used between logical regions and physical segments, adding new physical segments also requires establishing new mapping relationships between logical regions and physical segments. This further increases the already large number of mapping relationships in the metadata management cluster, prolonging access latency during mapping lookup and reducing data operation efficiency. Besides persistent mappings between logical regions and physical segments, hash algorithms can be used to determine the mapping relationship between newly added physical segments and logical regions through computation. This method avoids storing mapping relationships in the metadata management cluster, but the change in the total number of physical segments affects the already established mapping relationships between physical segments and logical regions. Essentially, expanding physical segments requires adjusting existing mapping relationships and migrating data from existing physical segments to ensure the accuracy of all mapping relationships between physical segments and logical regions. In this case, the amount of data to be migrated is enormous.

[0091] Scaling up at the physical group level, rather than directly at the physical segment level, results in a large number of mapping relationships compared to fine-grained persistent mappings between physical segments and logical areas. These mapping relationships cannot be stored locally and must be stored in the metadata management cluster, putting query pressure on the metadata management cluster and reducing the efficiency of querying mapping relationships within the cluster, thus affecting data operation efficiency. Scaling up at the physical group level, on the other hand, utilizes coarse-grained persistent mappings between physical groups and logical blocks, reducing the number of mapping relationships to be stored. This allows for local storage of mapping relationships, avoiding additional query pressure on the metadata management cluster and improving the efficiency of querying mapping relationships.

[0092] The mapping relationship between logical blocks and their associated candidate physical groups is persistently stored in the metadata management cluster. Once a mapping relationship between a logical block and its associated candidate physical group is determined, the mapping relationship remains unchanged. When a new mapping relationship is added between a logical block and its associated candidate physical group, the new mapping relationship is also persistently stored in the metadata management cluster; in this case, the new mapping relationship also remains unchanged. Furthermore, the new mapping relationship will not affect the existing mapping relationships.

[0093] Specifically, in the case of physical segment expansion, i.e., when the storage capacity of the storage system is insufficient, the newly added physical groups can be obtained, and the total number of physical groups can be updated. When obtaining a new logical block, the first hash algorithm can be used to calculate the block identifier of the new logical block and the updated total number of physical groups to determine the group identifier of the candidate physical groups associated with the new logical block, thus obtaining the mapping relationship between the new logical block and the associated candidate physical groups.

[0094] By acquiring new physical groups during physical segment expansion and expanding according to the level of the physical groups, and utilizing coarse-grained persistent mapping between physical groups and logical blocks, the amount of mapping relationship storage is reduced, enabling local storage of mapping relationships and avoiding additional query pressure on the metadata management cluster, while also improving the query efficiency of mapping relationships. By updating the total number of physical groups, when acquiring new logical blocks, the first hash algorithm is used to determine the candidate physical groups associated with the new logical blocks based on the block identifier of the new logical blocks and the updated number of physical groups, obtaining the mapping relationship between the new logical blocks and the associated candidate physical groups. The first hash algorithm is used to establish the mapping relationship between the new logical blocks and the associated candidate physical groups, increasing the complexity of determining the mapping relationship between the new logical blocks and the associated candidate physical groups, and improving the security of the persistent mapping between the new logical blocks and the associated candidate physical groups. By persistently storing the mapping relationship between the new logical blocks and the associated candidate physical groups, while expanding the physical segments, the newly added mapping relationships are prevented from affecting the already established mapping relationships, further improving the accuracy, real-time performance, and comprehensiveness of the mapping relationships in the metadata management cluster.

[0095] In an optional embodiment of this disclosure, the method further includes: determining whether a mapping relationship between the target logical block and the target physical group is stored locally; if not, obtaining the mapping relationship between the target logical block and the target physical group from the metadata management cluster and caching the mapping relationship locally.

[0096] Local can be understood as the client's local machine. The mapping relationship between logical blocks and physical groups in the metadata management cluster is updated in real time. That is, the mapping relationship between logical blocks and physical groups in the metadata management cluster is the most comprehensive.

[0097] Specifically, based on the block identifier of the target logical block and the group identifier of the target physical group, it can query whether the mapping relationship between the target logical block and the target physical group is stored locally. If not, the mapping relationship between the target logical block and the target physical group is obtained from the metadata management cluster and cached locally.

[0098] By determining whether a mapping relationship between the target logical block and the target physical group is stored locally, and if not, retrieving the mapping relationship from the metadata management cluster and caching it locally, the mapping relationship in the local cache is updated using the mapping relationship in the metadata management cluster. This improves the comprehensiveness of the mapping relationship in the local cache. In subsequent data operations, the mapping relationship between the target logical block and the target physical group can be queried directly in the client's local cache without having to access the metadata management cluster again, further improving the query efficiency of the mapping relationship and reducing the query pressure on the metadata management cluster.

[0099] In an optional embodiment of this disclosure, selecting a target physical segment associated with a target logical region from at least two physical segments included in a target physical group includes: using a second hash algorithm to determine the target physical segment associated with the target logical region based on the sequence number of the target logical region and the number of candidate physical segments included in the candidate physical group.

[0100] The sequence number of the target logical region can be used to uniquely identify it. The second hash algorithm can include hash algorithms, consistent hashing algorithms, or Crush (Controlled Replication Under Scalable Hashing) algorithms, etc. The first hash algorithm and the second hash algorithm can be the same or different. Both the first and second hash algorithms are used to determine the mapping relationship between logical storage locations and physical storage locations. However, the granularity of the mapping relationship corresponding to the first and second hash algorithms is different. Specifically, the first hash algorithm is used to determine the mapping relationship between candidate logical blocks and candidate physical groups; the second hash algorithm is used to determine the mapping relationship between target logical regions and target physical segments. It can be understood that the first hash algorithm is used for coarse-grained mapping, and the second hash algorithm is used for fine-grained mapping.

[0101] Specifically, a second hash algorithm can be used to calculate the sequence number of the target logical region and the number of candidate physical segments included in the candidate physical group to obtain the sequence number of the target physical segment associated with the target logical region, and thus determine the target physical segment associated with the target logical region.

[0102] For example, the following formula can be used to calculate the sequence number of the target physical segment:

[0103] Hash'(Extent R) = R%S;

[0104] Where Hash' is the second hash algorithm; Extent R is the target logical region; R is the sequence number of the target logical region; and S is the number of candidate physical segments included in the candidate physical group.

[0105] By employing a second hash algorithm, the target physical segment associated with the target logical region is determined directly based on the sequence number of the target logical region and the number of candidate physical segments included in the candidate physical group. This ensures the security of the mapping relationship between the target logical region and the target physical segment while simplifying the determination process and improving the efficiency and accuracy of target physical segment determination. Furthermore, by using the second hash algorithm to determine the mapping relationship between the target logical region and the target physical segment, the mapping relationship between the logical region and the physical segment avoids occupying storage space in the local cache or metadata management cluster, further improving the execution efficiency of data operations.

[0106] According to the technical solution of this disclosure, by adopting a first hash algorithm, the candidate physical groups associated with the candidate logical blocks are determined based on the block identifier of the candidate logical blocks and the total number of candidate physical groups, thereby obtaining the mapping relationship between the candidate logical blocks and the associated candidate physical groups. This increases the complexity of determining the mapping relationship between the candidate logical blocks and the candidate physical groups and improves the security of the persistent mapping between the candidate logical blocks and the candidate physical groups.

[0107] Figure 6 This is a scenario diagram of a data manipulation method disclosed in an embodiment of this disclosure. Figure 6 This is a preferred embodiment. For example... Figure 6 As shown, the specific solution is as follows:

[0108] 1. Physical segments can be grouped, with each group consisting of N physical segments. Typically, N can be set to 128.

[0109] 2. When expanding a physical segment, the number of physical segments is increased. Expansion can be done in units of physical groups, meaning that each expansion involves expanding a physical group containing N physical segments.

[0110] 3. Form a logical block (Chunk) from every N consecutive logical extents of the cloud disk or database file; wherein each logical block (Chunk) includes at least two logical extents (Extents); and each logical extent (Extent) includes at least two logical pages (Pages).

[0111] 4. For each logical chunk, map it to a physical group using the first hash algorithm, and then persist the Chunk->Group mapping relationship in the metadata management cluster. Since a logical chunk consists of N logical extents, the number of logical chunks is much smaller than the number of logical extents. Therefore, the number of Chunk->Group mapping relationships is much smaller than the number of Extent->Segment mapping relationships. This allows all Chunk->Group mapping relationships to be cached in the client's local memory.

[0112] 5. For a target logical region (Extent K), first calculate the target logical block (Chunk) it belongs to, then query the Chunk->Group mapping relationship from the client's local cache to obtain the target physical group (Group) corresponding to the target logical block (Chunk); then query the N physical segments (Segments) contained in the target physical group (Group) from the local cache, i.e. segment_0, segment_1, ..., segment_N-1; finally, use the second hash algorithm, i.e., Hash'(Extent K) = K%N, to calculate the target physical segment (Segment) mapped to the target logical region (Extent K).

[0113] For example, the specific process of data query is as follows:

[0114] 1) The client receives a data query request. It first calculates which target logical area (Extent) and which target logical block (Chunk) the target logical page (Page) is located in. The data query request could be "to read the data on page 4 of cloud disk 1 or database file 1". Page 4 is located in the first target logical block, which is Chunk 0. Since the number of logical blocks (Chunks) is much smaller than the number of logical areas (Extents), the Chunk->Group mapping can be entirely cached in the client's local memory.

[0115] 2) The client queries the local cache to find that the target logical block (Chunk 0) is mapped to the target physical group (Group K). Then, the client queries the local cache to find that the target physical group (Group K) consists of N physical segments (Segments), namely segment_0, segment_1, ..., segment_N-1. The client calculates that the target logical page (Page) is located on the target logical area (Extent 0). Then, using the second hash algorithm, Hash'(Extent 0) = 0%N, the client calculates that the target logical area (Extent 0) is mapped to the target physical segment (Segment_0). The client queries the local cache (or pulls it from the metadata management cluster if it is not found) to find that the three replicas of the target physical segment (Segment_0) are located on backup storage nodes 1, 3, and 7.

[0116] 3) The client sends a data query request to one of the backup storage nodes 1: read the 4th page of data from cloud disk 1 or database file 1 from the target physical segment (segment_0).

[0117] 4) When the backup storage node receives a data query request, it queries the index file of the target physical segment (segment_0) to obtain the location of the physical area in the data file corresponding to the content of the target logical area (Extent0) where the 4th page of data is located. Then, it reads the data of this page from the data file and returns it to the client.

[0118] Compared to directly using a hash algorithm to determine the mapping relationship between physical segments and logical areas, which requires a large amount of data migration when expanding physical segments, this solution eliminates the need for data migration during physical segment expansion. Compared to fine-grained persistent mapping between logical areas and physical segments, this solution offers lower query latency for mapping relationships. Furthermore, by using coarse-grained persistent mapping between logical blocks and physical groups, the amount of storage required for mapping relationships is reduced, resulting in lower resource demands on the metadata management cluster, lower data operation costs, and solving the bottleneck problem of the metadata management cluster.

[0119] According to embodiments of this disclosure, Figure 7 This is a structural diagram of the data manipulation device in an embodiment of this disclosure, which is applicable to situations involving the operation of a data manipulation method. The device is implemented in software and / or hardware and is specifically configured in an electronic device with a certain data processing capability.

[0120] like Figure 7A data manipulation device 700 is shown, comprising: a target logical block determination module 701, a target physical group determination module 702, a target physical segment selection module 703, and a data manipulation module 704. The target logical block determination module 701, in response to a data manipulation request, determines the target logical region to which the target data to be manipulated belongs and the target logical block to which the target logical region belongs. The target physical group determination module 702, based on the mapping relationship between candidate logical blocks and candidate physical groups in the local cache, determines the target physical group associated with the target logical block. Each candidate logical block includes at least two logical regions; each candidate physical group includes at least two physical segments. The target physical segment selection module 703, from the at least two physical segments included in the target physical group, selects the target physical segment associated with the target logical region. The data manipulation module 704, for performing data manipulation on the storage space where the target physical segment is located.

[0121] According to the technical solution of this disclosure, in response to a data operation request, the target logical region to which the target data to be operated belongs and the target logical block to which the target logical region belongs are determined. Based on the mapping relationship between candidate logical blocks and candidate physical groups in the local cache, the target physical group associated with the target logical block is determined. From at least two physical segments included in the target physical group, the target physical segment associated with the target logical region is selected. Data operations are performed on the storage space where the target physical segment is located. By performing a coarse-grained persistent mapping between candidate logical blocks and candidate physical groups, compared with the existing technology that directly performs a fine-grained mapping between logical regions and physical segments, the number of mapping relationships is reduced, and local caching of mapping relationships is realized. During the data operation process, there is no need to query the metadata management cluster, avoiding access latency of the metadata management cluster. At the same time, it also reduces the number of mapping relationship queries during the data operation process, improving the efficiency of data operation. In addition, it reduces the additional query pressure on the metadata management cluster and avoids the bottleneck of the metadata management cluster.

[0122] In an optional embodiment of this disclosure, the target logical block determination module 701 includes: a target logical page determination unit, configured to determine the target logical page to which the target data to be operated belongs when the data operation request is a data query request; a target logical area determination unit, configured to determine the target logical area to which the target logical page belongs based on the partitioning relationship between the logical area and the logical page in the local cache; and a target logical block determination unit, configured to determine the target logical block to which the target logical area belongs based on the partitioning relationship between the logical block and the logical area in the local cache.

[0123] In an optional embodiment of this disclosure, the apparatus further includes: a mapping relationship determination module, configured to use a first hash algorithm to determine the candidate physical groups associated with the candidate logical blocks based on the block identifier of the candidate logical blocks and the total number of candidate physical groups, thereby obtaining a mapping relationship between the candidate logical blocks and the associated candidate physical groups.

[0124] In an optional embodiment of this disclosure, the apparatus further includes: a new physical group acquisition module, configured to acquire newly added physical groups during physical segment expansion, and update the total number of physical groups based on the newly added physical groups; wherein the newly added physical groups include at least two physical segments; a new mapping relationship determination module, configured to, when acquiring a new logical block, use a first hash algorithm to determine candidate physical groups associated with the new logical block based on the block identifier of the new logical block and the updated total number of physical groups, thereby obtaining a mapping relationship between the new logical block and the associated candidate physical groups; and a new mapping relationship storage module, configured to persistently store the mapping relationship between the new logical block and the associated candidate physical groups in the metadata management cluster.

[0125] In an optional embodiment of this disclosure, the apparatus further includes a mapping relationship caching module, configured to determine whether a mapping relationship between a target logical block and a target physical group is stored locally; if not, to obtain the mapping relationship between the target logical block and the target physical group from the metadata management cluster and cache the mapping relationship locally.

[0126] In an optional embodiment of this disclosure, the target physical segment selection module 703 includes: a target physical segment selection unit, used to determine the target physical segment associated with the target logical region by employing a second hash algorithm based on the sequence number of the target logical region and the number of candidate physical segments included in the candidate physical group.

[0127] In an optional embodiment of this disclosure, the data operation module 704 includes: a backup storage node acquisition unit, configured to acquire the backup storage node of the target physical segment when the data operation request is a data query request; a backup storage node request unit, configured to send the target physical segment and the data operation request to any backup storage node, so that the backup storage node determines the target data to be queried; and a target data acquisition unit, configured to acquire the target data fed back by the backup storage node.

[0128] In an optional embodiment of this disclosure, the data operation module 704 includes: a storage space writing unit, used to write the data to be stored into the storage space where the target physical segment is located when the data operation request is a data storage request.

[0129] The above-described data manipulation device can execute the data manipulation method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the data manipulation method.

[0130] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0131] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0132] Figure 8 A schematic area diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0133] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0134] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0135] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as data manipulation methods. For example, in some embodiments, the data manipulation methods may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the data manipulation methods described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform data manipulation methods by any other suitable means (e.g., by means of firmware).

[0136] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard objects (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or area diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0138] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0139] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0140] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0141] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0142] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0143] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A data manipulation method, executed by a client, the method comprising: In response to a data operation request, determine the target logical area to which the target data to be operated belongs and the target logical block to which the target logical area belongs; Based on the mapping relationship between candidate logical blocks and candidate physical groups in the local cache, a target physical group associated with the target logical block is determined; wherein, each candidate logical block includes at least two logical areas; each candidate physical group includes at least two physical segments; the candidate logical block and the candidate physical group have a persistent mapping relationship; the candidate logical block and the candidate physical group are storage locations of the same data in different dimensions; the candidate logical block is the logical storage location of the data; the candidate physical group is the physical storage location of the data; Select the target physical segment associated with the target logical region from at least two physical segments included in the target physical group; Perform data operations on the storage space where the target physical segment is located; The mapping relationship between the candidate logical blocks and the candidate physical groups is determined in the following way: The first hash algorithm is used to calculate the block identifier of the candidate logical block and the total number of candidate physical groups to determine the group identifier of the candidate physical group associated with the candidate logical block, and to obtain the mapping relationship between the candidate logical block and the associated candidate physical group. The step of selecting the target physical segment associated with the target logical region from at least two physical segments included in the target physical group includes: The second hash algorithm is used to determine the target physical segment associated with the target logical region based on the sequence number of the target logical region and the number of candidate physical segments included in the candidate physical group.

2. The method according to claim 1, wherein, The process of determining the target logical region to which the target data to be operated belongs and the target logical block to which the target logical region belongs includes: If the data operation request is a data query request, determine the target logical page to which the target data to be operated belongs; Based on the division relationship between logical regions and logical pages in the local cache, determine the target logical region to which the target logical page belongs; Based on the division relationship between logical blocks and logical areas in the local cache, the target logical block to which the target logical area belongs is determined.

3. The method according to claim 1, further comprising: When a physical segment is expanded, the newly added physical groups are obtained, and the total number of physical groups is updated based on the newly added physical groups; wherein, the newly added physical groups include at least two physical segments; When a new logical block is acquired, the first hash algorithm is used to determine the candidate physical groups associated with the new logical block based on the block identifier of the new logical block and the total number of updated physical groups, thereby obtaining the mapping relationship between the new logical block and the associated candidate physical groups. The mapping between new logical blocks and associated candidate physical groups is persisted in the metadata management cluster.

4. The method according to claim 1, further comprising: Determine whether a mapping relationship between the target logical block and the target physical group is stored locally. If not, obtain the mapping relationship between the target logical block and the target physical group from the metadata management cluster and cache the mapping relationship locally.

5. The method according to claim 1, wherein, The data operations performed on the storage space where the target physical segment is located include: If the data operation request is a data query request, obtain the backup storage node of the target physical segment; Send the target physical segment and data operation request to any backup storage node, so that the backup storage node determines the target data to be queried; Obtain the target data returned by the backup storage node.

6. The method according to claim 1, wherein, The data operations performed on the storage space where the target physical segment is located include: If the data operation request is a data storage request, the data to be stored is written into the storage space where the target physical segment is located.

7. A data manipulation device, executed by a client, the device comprising: The target logic block determination module is used to determine the target logic area to which the target data to be operated belongs and the target logic block to which the target logic area belongs in response to a data operation request. The target physical group determination module is used to determine the target physical group associated with the target logical block based on the mapping relationship between candidate logical blocks and candidate physical groups in the local cache; wherein each candidate logical block includes at least two logical areas; each candidate physical group includes at least two physical segments; the mapping relationship between the candidate logical block and the candidate physical group is persistent; the candidate logical block and the candidate physical group are storage locations of the same data in different dimensions; the candidate logical block is the logical storage location of the data; the candidate physical group is the physical storage location of the data; The target physical segment selection module is used to select a target physical segment associated with the target logical area from at least two physical segments included in the target physical group; The data operation module is used to perform data operations on the storage space where the target physical segment is located; The mapping relationship between the candidate logical blocks and the candidate physical groups is determined by the following modules: The mapping relationship determination module is used to use the first hash algorithm to calculate the block identifier of the candidate logical block and the total number of candidate physical groups, determine the group identifier of the candidate physical group associated with the candidate logical block, and obtain the mapping relationship between the candidate logical block and the associated candidate physical group. The target physical segment selection module includes: The target physical segment selection unit is used to determine the target physical segment associated with the target logical area by using a second hash algorithm, based on the sequence number of the target logical area and the number of candidate physical segments included in the candidate physical group.

8. The apparatus according to claim 7, wherein, The target logic block determination module includes: The target logical page determination unit is used to determine the target logical page to which the target data to be operated belongs when the data operation request is a data query request. The target logical area determination unit is used to determine the target logical area to which the target logical page belongs based on the partitioning relationship between the logical areas and logical pages in the local cache. The target logic block determination unit is used to determine the target logic block to which the target logic area belongs based on the partitioning relationship between logic blocks and logic areas in the local cache.

9. The apparatus according to claim 7, further comprising: A new physical group acquisition module has been added to acquire the newly added physical groups when the physical segment is expanded, and to update the total number of physical groups based on the newly added physical groups; wherein the newly added physical groups include at least two physical segments; A new mapping relationship determination module is added, which is used to determine the candidate physical groups associated with the new logical block by using the first hash algorithm when a new logical block is obtained, based on the block identifier of the new logical block and the total number of updated physical groups, and obtain the mapping relationship between the new logical block and the associated candidate physical groups. A new mapping relationship storage module has been added to persistently store the mapping relationship between new logical blocks and associated candidate physical groups in the metadata management cluster.

10. The apparatus according to claim 7, further comprising: The mapping relationship caching module is used to determine whether the mapping relationship between the target logical block and the target physical group is stored locally. If not, the mapping relationship between the target logical block and the target physical group is obtained from the metadata management cluster and cached locally.

11. The apparatus according to claim 7, wherein, The data operation module includes: The backup storage node acquisition unit is used to acquire the backup storage node of the target physical segment when the data operation request is a data query request. A backup storage node request unit is used to send the target physical segment and data operation request to any backup storage node, so that the backup storage node determines the target data to be queried. The target data acquisition unit is used to acquire the target data returned by the backup storage node.

12. The apparatus according to claim 7, wherein, The data operation module includes: The storage space writing unit is used to write the data to be stored into the storage space where the target physical segment is located when the data operation request is a data storage request.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the data manipulation method according to any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the data manipulation method according to any one of claims 1-6.

15. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images