Method, apparatus, and computer program product for data storage

Abstract

Embodiments of the present disclosure relate to a method, device and computer program product for data storage. The method comprises determining a hash value of each node in a set of nodes based on attribute information of each node in the set of nodes. The method further comprises determining a node identifier of each node in the set of nodes based on geographical location information and the hash value of each node in the set of nodes. The node identifier is used to indicate a logical position of the node in the distributed storage system. The method further comprises determining a target node in the set of nodes based on the node identifier of each node in the set of nodes in response to a processing request for first data by a first node in the set of nodes. The target node is used to process the processing request for the first data. Embodiments of the present disclosure achieve geographical location-based distribution of each node in the distributed storage system. In this way, low latency and high efficiency of data processing within a region can be achieved.

Description

Technical Field

[0001] Embodiments of this disclosure generally relate to the field of computers, and more particularly to methods, apparatus and computer program products for data storage. Background Technology

[0002] Currently, an increasing number of cloud and edge nodes, along with a large number of clients, are participating in 5G networks and edge storage and computing as nodes. Various types of nodes are connected together to form a distributed storage system (also known as a distributed storage network) to provide users with high-throughput and high-availability data.

[0003] Distributed storage systems based on edge storage, organized in a decentralized manner, have been proposed. However, as the number of participating nodes increases, the performance of distributed storage systems degrades due to the need for longer search paths or more hops to find the metadata associated with the stored data (e.g., distributed hash tables). Therefore, there is a desire to design more optimized storage system architectures and data retrieval mechanisms to improve the performance of distributed storage systems. Summary of the Invention

[0004] Embodiments of this disclosure provide methods, apparatus, and computer program products for data storage.

[0005] In a first aspect of this disclosure, a method for data storage is provided. The method includes: determining a hash value for each of a set of nodes based on attribute information of each node in a distributed storage system. The method further includes: determining a node identifier for each of the nodes based on their respective geographical location information and hash values. The node identifier indicates the logical location of the node within the distributed storage system. The method also includes: in response to a processing request for first data from a first node in the set of nodes, determining a target node in the set of nodes based on the node identifiers of each node. The target node is used to process the processing request for the first data.

[0006] In a second aspect of this disclosure, an electronic device is provided. The electronic device includes at least one processing unit and at least one memory. The at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the electronic device to perform an action including: determining a hash value for each of a set of nodes based on attribute information of each of the nodes in a distributed storage system; determining a node identifier for each of the nodes based on geographical location information and the hash value of each of the nodes, the node identifier indicating the logical location of the node in the distributed storage system; and, in response to a processing request for first data from a first node in the set of nodes, determining a target node in the set of nodes based on the node identifier of each of the nodes, the target node being used to process the processing request for the first data.

[0007] In a third aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored in a non-transitory computer storage medium and includes machine-executable instructions. When executed by a device, the machine-executable instructions cause the device to perform any step of the method described in the first aspect of this disclosure.

[0008] The summary section is provided to present the chosen concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or essential features of this disclosure, nor is it intended to limit the scope of this disclosure. Attached Figure Description

[0009] The above and other objects, features and advantages of this disclosure will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0010] Figure 1 A schematic block diagram of an example distributed storage system that can be implemented therein according to some embodiments of the present disclosure is shown;

[0011] Figure 2 A flowchart is shown illustrating an example method for data storage according to some embodiments of this disclosure;

[0012] Figure 3 A schematic diagram of a hash ring-based distributed storage architecture according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A schematic diagram of a routing table-based data lookup architecture according to some embodiments of the present disclosure is shown;

[0014] Figure 5A schematic diagram illustrating the storage of metadata based on a hash ring according to some embodiments of the present disclosure is shown; and

[0015] Figure 6 A schematic block diagram of an example device that can be used to implement embodiments of the present disclosure is shown.

[0016] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0017] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0018] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0019] As mentioned above, nodes in a distributed storage system based on edge storage are typically organized in a decentralized manner. When a client requests access to application data, it first needs to locate the application data's metadata in the distributed hash table of the distributed storage system. However, the request to query metadata will be forwarded multiple times among the decentralized global nodes before the metadata indicating the storage location of the application data can be obtained.

[0020] The inventors noted that, considering device and data mobility, application data is typically requested or shared between nodes within the same region to ensure service continuity. In particular, in scenarios such as smart manufacturing and autonomous driving, data needs to be transmitted more efficiently within specific regions, rather than being requested or shared across regions in a global network.

[0021] However, conventional decentralized solutions do not consider the geographical relationships between nodes. This conventional mechanism treats all decentralized nodes in the global view equally. For example, nodes that are geographically (also known as physically) adjacent may be logically far apart. Conversely, nodes that are geographically far apart may be logically adjacent. Therefore, because nodes are unaware of the existence of their physically adjacent nodes, requests for data consumption within each region are affected by the inefficiency of target node addressing. This, in turn, leads to high latency and inefficient data sharing between edge nodes (e.g., edge servers).

[0022] Furthermore, data queries in conventional decentralized solutions suffer from inefficiency. In a conventional approach, metadata for data generated by a source node is broadcast to the decentralized machines and stored on logically adjacent target nodes. However, these target nodes may be physically far from the source node that generated the data, or even from the region where the current node is located. In this case, because the metadata is stored on a target node far from the source node, queries for data or data blocks typically require several hops. At each hop, queries may be made to physically distant nodes with high response latency to determine if they hold the data. Therefore, data queries can be time-consuming.

[0023] In view of this, embodiments of the present disclosure propose a method for data storage to address one or more of the aforementioned problems and / or other potential problems. In this scheme, a node identifier is determined based on both the geographical location information of each node and the hash value of each node. The node identifier is used to determine the logical location of the node in the distributed storage system, i.e., its location in the hash space. Furthermore, in this scheme, in response to a data processing request, a target node is determined based on the node identifiers of the nodes.

[0024] This solution considers the geographical location information of each node in both the logical spatial positioning of nodes and the determination of target nodes for data processing requests. This geographically aware edge node distribution scheme enables distributed storage systems to achieve more efficient data sharing and data querying. In particular, it ensures lower latency and higher efficiency for data requests or queries within the same region.

[0025] The basic principles and several exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings. Figure 1 A schematic diagram of a distributed storage network 100 for data storage according to some embodiments of the present disclosure is shown. It should be understood that... Figure 1The number and arrangement of entities, elements and modules shown are merely examples; the distributed storage network 100 may include different numbers and arrangements of entities, elements and modules.

[0026] like Figure 1 As shown, the distributed storage system 100 in the edge computing environment can be implemented as a hierarchical structure, including a terminal layer 110, an edge node layer 130, and a cloud layer 150.

[0027] Terminal layer 110 typically includes various types of terminal nodes 112, 114, 116, etc. (also known as clients). Examples of terminal nodes include, but are not limited to: desktop computers, laptop computers, smartphones, wearable devices, security cameras, smart manufacturing equipment, smart home devices, IoT devices, smart cars, etc. It should be understood that, although... Figure 1 Only three terminal nodes are shown in the illustration, but this is merely exemplary and does not limit the invention in any way. The distributed storage network 100 may include any number and type of terminal nodes.

[0028] Terminal nodes 112, 114, and 116 can join the distributed storage system 100 via wired or wireless connections based on their geographical location, network configuration, etc., to communicate and couple with each other. For example, terminal node 112 can be a smartphone, accessing the corresponding cell based on its geographical location, connecting to the nearest node (also called an edge node), and switching to another node as its location changes. As another example, terminal node 114 can be a smart manufacturing device, accessing the corresponding cell based on its geographical location, and connecting to the nearest node. Yet another example is a smart car, connecting to a nearby vehicle-to-everything (V2X) edge node based on its location, and switching to the appropriate V2X edge node while driving.

[0029] The edge node layer 130 typically includes multiple nodes (also called edge nodes or peer nodes) distributed across different geographical locations, such as nodes 132, 143, 136, 138, 142, 144, and 146. Each node can join the distributed storage system 100 via, for example, wired or wireless connections to communicate and couple with each other. In this document, the nodes in the edge node layer 130 can be any computing device with computing resources or a storage device with storage resources. It should be understood that... Figure 1 The number and arrangement of nodes shown are merely exemplary and do not limit the invention in any way. The distributed storage network 100 may include any number, type, and arrangement of (edge) nodes.

[0030] In this document, for illustrative purposes, the edge node layer 130 is divided into different regions, such as the first region 131 and the second region 141 shown in the figure. Nodes 132, 134, 136, and 138 located in the first region 131 can be considered to be geographically proximate. Similarly, nodes 142, 144, and 146 located in the second region 141 can also be considered to be geographically proximate. It should be understood that the above region division is merely exemplary and not restrictive. In some embodiments, larger or smaller regions may be divided, or regions of other shapes may be divided, or regions containing more or fewer nodes may be divided.

[0031] Cloud layer 150 can include multiple clouds, such as clouds 152, 154, etc. Each cloud can be a cluster of multiple server devices integrated through hardware and software computing, provided by various cloud service providers, and serving as a whole. It should be understood that... Figure 1 The number and arrangement of clouds shown are merely exemplary and do not limit the invention in any way. The distributed storage network 100 may include any number, any type, and any arrangement of clouds.

[0032] In the distributed storage system 100, each node possesses both storage and communication capabilities. It can respond to data access requests from other nodes and establish a communication path with the requesting node to transmit the requested data. According to embodiments of this disclosure, the distributed storage system 100 stores data (also called application data) and metadata of the data. The metadata may include at least the storage location of the application data, the size of the application data, and the source of the application data. According to embodiments of this disclosure, a node requesting access to data can query the data's metadata to obtain the data's storage location, thereby establishing a communication path with the corresponding node to access the data. As mentioned above, the existing data lookup mechanism of distributed storage systems is not conducive to efficiently obtaining application data's metadata, leading to a decrease in system performance. The following refers to... Figures 2 to 6 And combined Figure 1 The methods for storing and querying data according to embodiments of the present disclosure are described in further detail.

[0033] although Figure 1 The distributed storage system 100 is shown to include a specific number of terminal nodes, edge nodes, and a cloud; however, those skilled in the art will understand that the number of nodes in the distributed storage system 100 is not limited thereto. Furthermore, although... Figure 1 The distributed storage system 100 is shown in the topology shown in the figure, but its topology is not limited to this. For example, it may include more or fewer levels and any other connected tree topology.

[0034] The above combination Figure 1An example distributed storage system 100 according to some embodiments of the present disclosure is described below. Reference will be made to... Figure 2 This describes a flowchart of a method 200 for data storage according to some embodiments of the present disclosure. Method 200 can be described by... Figure 1 The distributed storage system 120 is implemented. It should be understood that method 200 can also be performed by other suitable devices or apparatuses. Method 200 may include additional actions not shown and / or the actions shown may be omitted; the scope of this disclosure is not limited in this respect. For ease of explanation, reference will be made to... Figure 1 Let's describe method 200.

[0035] like Figure 2 As shown in box 210, the distributed storage system 100 determines the hash value of a set of nodes based on their respective attribute information. The node attribute information may include, for example, the node's name (e.g., device name), the node's IP address, etc. In some embodiments, the distributed storage system 100 may employ any suitable hash algorithm, such as SHA256, to determine the hash value of a node based on its attribute information. The hash value of a node may be represented in hexadecimal or binary. It should be understood that the SHA256 example listed above is merely exemplary and not limiting. Any suitable hash algorithm may be used here and elsewhere in the text to calculate hash values ​​for nodes or for data.

[0036] At box 220, the distributed storage system 100 determines a set of node identifiers for each node based on their respective geographic location information and a determined hash value. In this document, the node identifier is also referred to as a geographic-based node identifier. The node identifier is used to indicate the logical location of a node within the distributed storage system 100, such as its location in the hash space. The node identifier can be represented in hexadecimal or binary form. The geographic location information of the node can be determined using any positioning method, such as GPS positioning. This scheme is not limited in this respect.

[0037] In some embodiments, the distributed storage system 100 can determine the location prefixes of a set of nodes based on their respective geographical location information. The location prefixes can be in hexadecimal or binary form and can have a predetermined number of bits. This predetermined number of bits can be pre-determined. Alternative or additional locations can be determined based on the size of the geographical regions divided within the distributed storage system 100 or the number of nodes within each region.

[0038] In some embodiments, the distributed storage system 100 may determine the prefix of a first node located in a first geographical region as a first prefix. For example, the distributed storage system 100 may set the prefixes of nodes 132, 134, 136, and 138 in the first region 131 as the first prefix. Taking a predetermined three-digit location prefix as an example, the prefixes of nodes 132, 134, 136, and 138 may be set to 000. The distributed storage system 100 may also determine the prefix of a second node located in a second geographical region different from the first geographical region as a second prefix different from the first prefix. For example, the distributed storage system 100 may set the prefixes of nodes 142, 144, and 146 in the second region 141 as the second prefix. Taking a predetermined three-digit location prefix as an example, the prefixes of nodes 142, 144, and 146 may be set to 001.

[0039] In some embodiments, the position prefixes of each set of nodes can be added as prefixes to the hash values ​​of each set of nodes to obtain the node identifiers for each set of nodes. For example, the node identifiers can be determined using the following:

[0040] GPID=Geo prefix+SHA256(Peer Identity)hash value (1)

[0041] In equation (1), GPID is used to represent the node identifier, Geo prefix is ​​used to represent the position prefix, SHA256() is used to represent the hash algorithm used, Peer Identity is used to represent the node's attribute information (e.g., node ID, node name, node IP address, etc.), and SHA256(Peer Identity)hash value is used to represent the node's hash value.

[0042] It should be understood that the methods for determining node identifiers described above are merely exemplary and are not intended to limit the scope of this disclosure. Any suitable method can be used to determine node identifiers, such as using other hash algorithms to determine the hash value of a node, or using other methods to determine the position prefix.

[0043] The above describes several examples of determining the node identifiers of each node. Based on the determined node identifiers, the logical location of each node in the distributed storage system 100, such as its location in the hash space, can be determined. For example, the logical distance between nodes can be determined based on the node identifiers using an XOR operation.

[0044] In some embodiments, the nodes can be distributed in the form of a hash ring in the distributed storage system 100. The distance between the nodes on the hash ring can identify the logical distance between the nodes. Figure 3 A schematic diagram of a distributed storage architecture based on a hash ring 300 according to some embodiments of the present disclosure is shown.

[0045] In some embodiments, hash ring 300 may refer to a virtual ring mapped to the hash value space calculated from the hash function. For example, when the hash space takes values ​​from 0 to 2... 32 When the value is -1, the hash values ​​can be evenly mapped onto the virtual ring. For example, hash value 0 can be mapped to the 12 o'clock position on the ring, and hash value 2... 31 Mapped to the 6 o'clock position of the ring, and so on. It should be understood that the range of values ​​for the hash space and the example mappings described above are merely exemplary and not restrictive. Any hash space with an appropriate range of values ​​can be used.

[0046] According to embodiments of this disclosure, the same hash function can be used to compute the hash value of each node in a set of nodes. Furthermore, the position prefix can be determined for each node in the set according to the same rules (e.g., setting the same number of positions in the position prefix). Based on the node identifier determined for each node, for example according to equation (1), each node can be mapped to the hash ring 300. Figure 3 As shown, for example only, nodes in a set of nodes are mapped to positions 310, 320, 330, 340 and 350 of hash ring 300 based on their hash values.

[0047] As described above, nodes in a group are distributed at corresponding positions on the hash ring according to their node identifiers, which are determined at least in part based on the node's geographical location. Therefore, nodes that are close together on the hash ring are generally also geographically close. For example, nodes 310, 320, and 330 on hash ring 300 are geographically close. Nodes 340 and 350 on hash ring 300 are farther away from nodes 310, 320, and 330. In the example of hash ring 300, nodes 310, 320, and 330 can be considered to be located in the same region (or geographical area). For illustrative purposes, although... Figure 3 Only five positions corresponding to nodes are shown, but the number of positions representing nodes on the hash ring can be arbitrary, depending on the number of nodes in a set.

[0048] It should be understood that although each node is mapped to a specific location in the hash ring, the range of storage space that each node has can include the range of hash space between that node and the next node.

[0049] By distributing nodes on a hash ring based on node identifiers, the nodes can be distributed according to their geographical location. In this way, the position of a node on the hash ring reflects its geographical location. This location-based distributed storage system 100 facilitates data storage and retrieval operations. In particular, for regional data consumption, this location-based distributed storage system 100 can achieve lower latency and higher efficiency. This location-based distributed storage system 100 is especially suitable for fields such as smart manufacturing and smart cars.

[0050] Continue to refer to Figure 2 At box 230, in response to a processing request for first data from a first node in a set of nodes, the distributed storage system 100 determines a target node in the set of nodes based on the node identifiers of the respective nodes. The target node is used to process the processing request for the first data. The first data may also be referred to as the first data block.

[0051] In some embodiments, a processing request from the first node for the first data may be, for example, a read request from the first node for the first data. In some embodiments, a processing request from the first node for the first data may also be other access requests for the first data, such as write requests. In this case, the distributed storage system 100 may determine at least one node whose logical location is no more than a threshold distance from the first node as a candidate node set based on the node identifiers of each of the nodes in the set. For example, at least one node on the hash ring 300 that is no more than a threshold distance from node 310, such as node 320, may be determined as a candidate node set. The threshold distance may be predetermined or may be set according to the size of the region or the number of nodes in the region.

[0052] Additionally, the distributed storage system 100 can search for a target node storing the first data from the candidate node set. The found target node can provide the first data to the first node. That is, the first node can read or access the first data from the target node.

[0053] In some embodiments, the distributed storage system 100 may store the node identifiers of a set of candidate nodes that are logically close to the first node in a routing table (also called a bucket). The distributed storage system 100 may use this routing table to find the target node for storing the first data. The distributed storage system 100 may also associate the node identifiers and addresses of each candidate node as entries in the routing table.

[0054] Figure 4 A schematic diagram of a routing table-based data lookup architecture according to some embodiments of the present disclosure is shown. Figure 4In the example, node A is the first node. The node identifier for node A is defined as "xxx…000", where "xxx" represents the position prefix of node A, "000" represents the last three digits of the identifier, and "…" indicates that the middle digits are omitted for clarity. At least one node with the same position prefix as node A can be added to the routing table for node A. For example, the logical distance between nodes can be determined by using an XOR algorithm based on their node identifiers, thus adding nodes whose logical distance does not exceed a threshold distance to node A's routing table. An appropriate threshold distance can be set to ensure (i.e., identical position prefixes).

[0055] Routing table 410 shows an example routing table for node A. In routing table 410, all nodes have the same location prefix "xxx". The first column of routing table 410 shows the range of logical distances to node A, and the second column shows the nodes within the corresponding distance range and their node identifiers. For example, node B, which is logically 1 distance from node A, has the node identifier "xxx…001". Similarly, the node identifiers of other candidate nodes C, D, E, F, G, and H are also shown. In the example of routing table 410, the distance threshold is set to 7, so the routing table includes only 7 candidate nodes. It should be understood that the distance threshold can be set to a smaller or larger value, and the routing table can include fewer or more candidate nodes.

[0056] Figure 4 The diagram also illustrates a lookup tree 420 based on the routing table. The first data can be found by traversing nodes B, C, D, E, F, G, and H in, for example, from left to right, according to the lookup tree 420. As mentioned earlier, data typically moves within the same area, so the first data is highly likely to be found among the candidate nodes in the routing table 410. Compared to conventional methods that usually require several hops (each hop queries several nodes in the routing table) to find the first data, this method, using a routing table determined by node identifiers based on geographical location, allows for rapid location of the target node storing the first data within a single routing table. This avoids hopping to other routing tables (i.e., to nodes far from the area) to find the data. In this way, data lookup time is saved, leading to faster data location.

[0057] Additionally or alternatively, in some embodiments, the processing request of the first node for the first data may include, for example, the first node requesting the storage of metadata of the first data generated at the first node. The metadata may include at least one or more of the following: the storage location of the first data, the size of the first data, and the source of the first data. When a node (e.g., a client device) requests access to the first data, it typically first needs to locate the metadata of the first data in the distributed hash table of the distributed storage system. Therefore, it is desirable that the metadata of the first data be stored on nodes located in the same region as the first data, thereby accelerating data location and retrieval.

[0058] In some embodiments, in order to enable metadata to be stored on nodes located in the same region as the first data, the distributed storage system 100 may determine a data identifier for the metadata based on the first node identifier of the first node and the hash value of the first data. The hash value of the first data may be determined using the same hash algorithm as the hash value of the compute node. The following describes an example method for determining the data identifier:

[0059] GCID=Geo prefix+SHA256(Content Identity)hash value (2)

[0060] In equation (2), GCID is used to represent the data identifier, Geo prefix is ​​used to represent the position prefix, SHA256() is used to represent the hash algorithm used, Content Identity is used to represent the content identifier of the first data (e.g., data attributes, data ID, etc.), and SHA256(Content Identity)hash value is used to represent the hash value of the data.

[0061] It should be understood that the methods for determining data identifiers described above are merely exemplary and are not intended to limit the scope of this disclosure. Any suitable method can be used to determine data identifiers, such as using other hash algorithms to determine the hash value of the data, or using other methods to determine the position prefix.

[0062] In some embodiments, the distributed storage system 100 may determine a target node in a set of nodes based on a comparison between the node identifiers of each node in the set and the data identifiers determined by the methods described above. For example, the distributed storage system 100 may, based on the determined data identifiers, identify the node from the set of nodes whose logical distance to the data identifier is closest (e.g., calculated by an XOR algorithm) as the target node. This target node can be used to store the metadata.

[0063] Alternatively, the distributed storage system 100 may, based on the determined data identifier, identify at least one node from the respective node identifiers of a set of nodes as a candidate target node set, provided that the logical distance to the data identifier does not exceed a predetermined threshold distance. The distributed storage system 100 may arbitrarily select from the candidate target node set or select a target node based on, for example, the remaining storage space of the node. The scope of this disclosure is not limited in this respect.

[0064] In this way, it can be ensured that the metadata of the first data generated by the first node is stored on the target node located in the same region as the first node. It should be understood that the target node can be any node located in that region. The target node can also be the first node itself. By storing the metadata of the first data on the target node in the same region, subsequent retrieval and processing of the first data are facilitated. This further improves the performance of the distributed storage system 100.

[0065] Additional or alternative land, in Figure 4 In the example using routing table 410, distributed storage system 100 can find the node with the closest logical distance to the data identifier from routing table 410 as the target node. By using the routing table, the target node can be found more conveniently, thereby further reducing the time spent on data storage.

[0066] Figure 5 A schematic diagram illustrating the storage of metadata based on a hash ring 300 according to some embodiments of the present disclosure is shown. Figure 5 In the example, first data 510 is generated at node 310. Distributed storage system 100 can determine the data hash value of metadata 520 for the first data 510 based on the node identifier of node 310 and the hash value of the first data 510. Distributed storage system 100 can then determine the target node with the closest logical path on the hash ring 300 based on the data hash value. For example, in Figure 5 In the example, node 330 can be identified as the target node. The distributed storage system 100 can broadcast the metadata 520 and store it in node 330.

[0067] It should be understood that Figure 5 The examples provided are merely illustrative and not restrictive. In other examples, other nodes located in the same region as node 310, such as node 320, or node 310 itself, can be identified as target nodes based on the node identifiers of each node and the data identifier.

[0068] By using the geographical location-based node identifier and data identifier method disclosed herein, the first data generated by the first device and its metadata can be stored in the same region (e.g., on the same node in the same region or on different nodes in the same region). Data requests (e.g., read or write requests) for this data are typically issued by nodes located in the same region. Therefore, by utilizing the scheme of this disclosure, metadata can be quickly located within this same region, thereby quickly locating the data. This scheme thus avoids the high probability of inefficient cross-region data queries and data retrieval in conventional schemes. In summary, this disclosure proposes an optimized distributed storage system scheme that can reduce system latency and improve system efficiency.

[0069] Figure 6 A schematic block diagram of an example device 600 that can be used to implement embodiments of the present disclosure is shown. For example, such as Figure 1 The distributed storage system 100 shown can be implemented by device 600. For example... Figure 6 As shown, device 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 602 or loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. CPU 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0070] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as a keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as a disk, optical disk, etc.; and communication unit 609, such as a network card, modem, wireless transceiver, etc. In some embodiments, input samples can be input to device 600 via input unit 606. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0071] The various processes and handling described above, such as method 200, can be executed by processing unit 601. For example, in some embodiments, method 200 can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by CPU 601, one or more actions of method 200 described above can be performed.

[0072] This disclosure can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0073] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0074] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0075] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0076] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0077] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0078] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0079] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0080] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for data storage, comprising: determining respective hash values of a set of nodes in a distributed storage system based on respective attribute information of the set of nodes; determining respective node identifiers of the set of nodes based on respective geographical location information of the set of nodes and the hash values, the node identifiers being used to indicate logical positions of the nodes in the distributed storage system; and in response to a processing request for first data by a first node of the set of nodes, determining a target node of the set of nodes for processing the processing request for the first data based on the respective node identifiers of the set of nodes; wherein the node identifier of each of the nodes is determined at least in part according to the hash value of the node and the geographical location information of the node; wherein the node identifiers collectively establish a hash topology for the nodes, wherein particular nodes that are geographically close to each other within a given region have respective node identifiers that are logically close to each other in the hash topology; and wherein determining the target node comprises performing at least one XOR operation to determine logical distances between the node identifiers of the nodes.

2. The method of claim 1, wherein determining the respective node identifiers of the set of nodes comprises: determining respective location prefixes of the set of nodes based on the respective geographical location information of the set of nodes, the location prefixes having a predetermined number of bits; and adding the respective location prefixes of the set of nodes as prefixes to the respective hash values of the set of nodes to obtain the respective node identifiers of the set of nodes.

3. The method of claim 2, wherein determining the respective location prefixes of the set of nodes comprises: determining a location prefix of a first node of the set of nodes located in a first geographical region as a first prefix; and determining a location prefix of a second node of the set of nodes located in a second geographical region different from the first geographical region as a second prefix different from the first prefix.

4. The method of claim 1, wherein the processing request for the first data comprises: the first node requests a read of the first data.

5. The method of claim 4, wherein determining the target node of the set of nodes comprises: based on the respective node identifiers of the set of nodes, determining at least one node that is within a threshold distance from a logical position of the first node as a candidate node set; from the candidate node set, finding the target node that stores the first data; and providing, by the target node, the first data to the first node.

6. The method of claim 5, wherein determining a candidate node set comprises: performing an XOR operation on a second node identifier of a second node of the set of nodes and a first node identifier of the first node to determine a logical distance between the first node and the second node; and if the logical distance does not exceed the threshold distance, adding the second node to the candidate node set.

7. The method of claim 1, wherein the processing request for the first data comprises: the first node requests storage of metadata of the first data generated at the first node.

8. The method of claim 7, wherein determining the target node in the set of nodes comprises: The data identifier of the metadata is determined based on the first node identifier of the first node and the hash value of the first data; as well as Based on the comparison between the node identifier of each of the group of nodes and the data identifier, the target node in the group of nodes is determined, and the target node is used to store the metadata.

9. An electronic device, comprising: At least one processor; as well as At least one memory storing computer program instructions, the at least one memory and the computer program instructions being configured, together with the at least one processor, to cause the electronic device to perform actions, the actions including: Based on the attribute information of a group of nodes in a distributed storage system, determine the hash value of each of the group of nodes. Based on the geographical location information of each node in the group and the hash value, a node identifier is determined for each node in the group, the node identifier indicating the logical location of the node in the distributed storage system; and In response to a processing request for first data from a first node in the group of nodes, a target node in the group of nodes is determined based on the node identifier of each node in the group, and the target node is used to process the processing request for the first data. The node identifier of each node is determined at least in part based on the hash value of the node and the geographical location information of the node; The node identifiers collectively establish a hash topology for each node, wherein specific nodes that are geographically close to each other within a given region have node identifiers that are logically close to each other in the hash topology; and Determining the target node includes performing at least one XOR operation to determine the logical distance between the node identifiers of the nodes in the node set.

10. The electronic device of claim 9, wherein determining the node identifier for each of the set of nodes comprises: The location prefix of each node in the group is determined based on its respective geographical location information, and the location prefix has a predetermined number of bits. as well as The position prefix of each of the group of nodes is added as a prefix to the hash value of each of the group of nodes to obtain the node identifier of each of the group of nodes.

11. The electronic device of claim 10, wherein determining the position prefix of each of the set of nodes comprises: The position prefix of the first node in the first geographical region of the group of nodes is determined as the first prefix; as well as The position prefix of the second node in the set of nodes located in a second geographical region different from the first geographical region is determined to be a second prefix different from the first prefix.

12. The electronic device of claim 9, wherein the processing request for the first data comprises: The first node responds to the read request for the first data.

13. The electronic device of claim 12, wherein determining the target node in the set of nodes comprises: Based on the node identifiers of each of the aforementioned nodes, at least one node whose logical position is no more than a threshold distance from the first node is identified as a candidate node set. Search the target node that stores the first data from the candidate node set; as well as The target node provides the first data to the first node.

14. The electronic device of claim 13, wherein determining the candidate node set comprises: Perform an XOR operation between the second node identifier of the second node in the group of nodes and the first node identifier of the first node to determine the logical distance between the first node and the second node; as well as If the logical distance does not exceed the threshold distance, the second node is added to the candidate node set.

15. The electronic device of claim 9, wherein the processing request for the first data comprises: The first node requests to store the metadata of the first data generated at the first node.

16. The electronic device of claim 15, wherein determining the target node in the set of nodes comprises: The data identifier of the metadata is determined based on the first node identifier of the first node and the hash value of the first data; as well as Based on the comparison between the node identifier of each of the group of nodes and the data identifier, the target node in the group of nodes is determined, and the target node is used to store the metadata.

17. A computer program product tangibly stored on a non-volatile computer-readable medium and comprising machine-executable instructions that, when executed, cause a device to perform the method according to any one of claims 1-8.

Images