Path selection method, data pulling method and wide-area storage service cluster system

Abstract

Embodiments of the present application provide a path selection method, a data pulling method and a wide-area storage service cluster system. The method can be used in the field of computer network technology. The method comprises the following steps: determining a target node corresponding to a data pulling request in a wide-area storage service cluster; determining a plurality of target paths between the target node according to a node path graph in the wide-area storage service cluster; determining path weights of the plurality of target paths according to attribute information of nodes in the wide-area storage service cluster; determining a shortest path in the plurality of target paths according to the path weights; and determining a second shortest path according to the path weights, the shortest path and a preset weight difference threshold value, wherein the shortest path and the second shortest path are both used to transmit the data pulling request to the target node. The method can improve the data pulling efficiency.

Description

Technical Field

[0001] The present application relates to the field of computer network technology, and in particular to a path selection method, a data pulling method, and a wide-area storage service cluster system. Background Art

[0002] In computer cloud storage applications, data storage and access infrastructure is a key factor in the user experience. When users upload or download files to or from the cloud over the internet, the speed, stability, and security of data transmission directly determine their overall perception of the service. Therefore, ensuring an efficient and secure data interaction experience has become a key concern.

[0003] Currently, the method of pulling the data required by users through a wide area storage network is often to calculate the shortest network access path based on the weights between node networks. Then, based on the data pull request issued by the user, the data is pulled from the cache of the target node through the shortest network access path, thereby quickly obtaining the data required by the user.

[0004] However, when pulling data through the above path selection method, there is a problem of low data pulling efficiency. Summary of the Invention

[0005] The embodiments of the present application provide a path selection method, a data pulling method, and a wide-area storage service cluster system to solve the problem of low data pulling efficiency.

[0006] In a first aspect, an embodiment of the present application provides a path selection method, including:

[0007] Determine the target node corresponding to the data pull request in the wide area storage service cluster;

[0008] Determine multiple target paths to the target node based on the node path graph in the wide area storage service cluster;

[0009] Determine the path weights of multiple target paths based on the attribute information of nodes in the wide area storage service cluster;

[0010] Determine the shortest path among multiple target paths based on the path weight;

[0011] According to the path weight, the shortest path and the preset weight difference threshold, the second shortest path is determined. Both the shortest path and the second shortest path are used to transmit the pull data request to the target node.

[0012] In one possible implementation, determining the next shortest path based on the path weight, the shortest path, and a preset weight difference threshold includes:

[0013] Determine the weight differences between multiple target paths and the shortest path based on the path weights and the path weight of the shortest path;

[0014] The second shortest path is determined according to the weight difference and a preset weight difference threshold, where the second shortest path is a path whose weight difference with the shortest path is not greater than the preset weight difference threshold.

[0015] In one possible implementation, determining path weights of multiple target paths based on attribute information of nodes in the wide area storage service cluster includes:

[0016] Determine node weights based on the attribute information of nodes in the wide area storage service cluster;

[0017] According to the node weights, the path weights of multiple target paths are determined.

[0018] In one possible implementation, determining the node weight based on attribute information of the nodes in the wide area storage service cluster includes:

[0019] Determining a first weight of the node based on network delay information in the attribute information, determining a second weight of the node based on bandwidth capacity information in the attribute information, and determining a third weight of the node based on cost information in the attribute information;

[0020] The node weight is determined according to the first node weight, the second node weight and the third node weight, wherein the first node weight and the third node weight are inversely proportional to the node weight, and the second node weight is in direct proportion to the node weight.

[0021] In a second aspect, an embodiment of the present application provides a data pulling method, comprising:

[0022] In response to receiving a pull data request, determining a shortest path and a second shortest path for transmitting the pull data request, where the shortest path and the second shortest path are obtained according to the path selection method of the first aspect;

[0023] Transmit the pull data request to the target node based on the shortest path and the second shortest path;

[0024] Receive the returned target data, which is the fastest returned required data among the shortest path and the second shortest path.

[0025] In a possible implementation, after receiving the returned target data, the method further includes:

[0026] The adjacent intermediate nodes in the shortest path and the second shortest path cache the demand data to obtain the cached demand data. The adjacent intermediate nodes are intermediate nodes connected to the storage service node, and the intermediate nodes are nodes between the storage service node and the target node.

[0027] If the shortest path and the second shortest path receive a new data pull request, and the target of the new data pull request is the demand data, the storage service node receives the demand data of the returned target cache, and the demand data of the target cache is the demand data of the cache returned fastest among the adjacent intermediate nodes of the shortest path and the adjacent intermediate nodes of the second shortest path.

[0028] In a possible implementation, after caching the demand data at adjacent intermediate nodes in the shortest path and the second shortest path to obtain the cached demand data, the method further includes:

[0029] If the shortest path and the second shortest path do not receive a new data pull request, or the target of the new data pull request is not the required data, then the cache retention time of the adjacent intermediate node is obtained;

[0030] If the cache retention time is longer than the preset cache retention time threshold, the cached demand data is deleted.

[0031] In the third aspect, an embodiment of the present application provides a wide-area storage service cluster system, including a central service node and storage service nodes corresponding to multiple regions. The central service node is used to periodically detect the attribute information and reachability information of multiple storage service nodes, and update the node path map according to the reachability information. The storage service node is used to execute the path selection method of the first aspect above.

[0032] In a fourth aspect, an embodiment of the present application provides a path selection device, including:

[0033] A first determination module is used to determine a target node corresponding to a data pull request in the wide area storage service cluster;

[0034] A second determination module is configured to determine multiple target paths to a target node based on a node path graph in the wide area storage service cluster;

[0035] A third determination module is configured to determine path weights of multiple target paths based on attribute information of nodes in the wide area storage service cluster;

[0036] a fourth determining module, configured to determine the shortest path among multiple target paths based on the path weights;

[0037] The fifth determination module is used to determine the second shortest path according to the path weight, the shortest path and a preset weight difference threshold, and both the shortest path and the second shortest path are used to transmit the pull data request to the target node.

[0038] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0039] Memory stores computer-executable instructions;

[0040] The processor executes the computer-executed instructions stored in the memory to implement the first aspect and / or the various possible implementation manners of the first aspect.

[0041] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, and the computer-readable storage medium stores computer-executed instructions. When the computer-executed instructions are executed by a processor, the computer-executed instructions are used to implement the first aspect and / or the various possible implementation manners of the first aspect.

[0042] In a seventh aspect, an embodiment of the present application provides a computer program product, and the computer program product includes a computer program. When the computer program is executed by a processor, the computer program implements the first aspect and / or the various possible implementation manners of the first aspect.

[0043] The path selection method, the data pulling method, and the wide-area storage service cluster system provided by the embodiments of the present application determine a target node corresponding to a data pulling request in a wide-area storage service cluster; determine a plurality of target paths between the target node according to a node path graph in the wide-area storage service cluster; determine path weights of the plurality of target paths according to attribute information of nodes in the wide-area storage service cluster; determine a shortest path in the plurality of target paths according to the path weights; and determine a second shortest path according to the path weights, the shortest path, and a preset weight difference threshold value. The shortest path and the second shortest path are both used to transmit the data pulling request to the target node. After the shortest path is determined, the weight difference between other paths and the shortest path is calculated. According to the weight difference and the preset weight difference threshold value, the second shortest path with a weight close to that of the shortest path is determined. The data pulling request is transmitted through the shortest path and the second shortest path at the same time, so that the data pulling efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles behind the present application.

[0045] Figure 1 The scene schematic diagram of path selection provided by the present application is shown in the following figure:

[0046] Figure 2 The flowchart of the path selection method provided by the embodiments of the present application is shown in the following figure:

[0047] Figure 3 The schematic diagram of determining the shortest path and the second shortest path provided by the embodiments of the present application is shown in the following figure:

[0048] Figure 4 The flowchart of another path selection method provided by the embodiments of the present application is shown in the following figure:

[0049] Figure 5A flowchart of a data pulling method provided by an embodiment of the present application is shown in the following figure.

[0050] Figure 6 A schematic diagram of updating a node path map provided by an embodiment of the present application is shown in the following figure.

[0051] Figure 7 A structural schematic diagram of a path selection device provided by an embodiment of the present application is shown in the following figure.

[0052] Figure 8 A structural schematic diagram of a data pulling device provided by an embodiment of the present application is shown in the following figure.

[0053] Figure 9 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in the following figure.

[0054] The above figures have shown the specific embodiments of the present application, which will be described in more detail hereinafter. These figures and the written description are not intended to limit the scope of the present application in any way, but to illustrate the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0055] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is made with reference to the accompanying drawings in which like reference numerals refer to like elements, unless the context of use indicates otherwise. The following exemplary embodiments described are not meant to be all inclusive in terms of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the appended claims.

[0056] First, the terms involved in the present application are explained:

[0057] Wide-area storage service cluster: refers to a storage system composed of storage service nodes distributed in multiple geographic locations, which are connected by high-speed network to provide unified and highly available data storage services;

[0058] Storage service node: refers to a physical or virtual server in a wide-area storage service cluster, which is responsible for actual data storage, management and processing. Each node is an independent work unit in the corresponding region, which collaborates to provide unified data storage services.

[0059] Figure 1 A schematic diagram of a path selection scenario provided by the present application is shown in the following figure. Figure 1As shown, the specific application scenario of this application is that user A-1 in region A creates a bucket bucket-movie (not shown in the figure) and specifies the index location as region A; user A-2 in region A writes object movie 1 to bucket bucket-movie, and the object will be written to the BOSS data storage space of region A nearby; user B in region B writes object movie 2 to bucket bucket-movie, and the object will be written to the BOSS data storage space of region B nearby; when the user in region A requests object movie 2, the BOSS service in region A will pull data from region B through the cloud-to-cloud highway, through the path region A-region B, or the path region A-region C-region B, and return it to the user.

[0060] Based on the above scenario, it can be seen that in the existing technology, the method of pulling data by selecting the shortest network access path to transmit the data pull request has the technical problem of low data pulling efficiency due to the performance differences between storage clusters in different regions and different response times to requests.

[0061] The path selection method, data pulling method and wide area storage service cluster system provided in this application determine the shortest path through the node path graph and the node attribute information, and then calculate the difference in weight between other paths and the shortest path. Based on the weight difference and the preset weight difference threshold, the second shortest path with a weight close to the shortest path is determined, and the technical means of simultaneously transmitting the data pulling request through the shortest path and the second shortest path solves the technical problem of low data pulling efficiency.

[0062] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0063] Figure 2 The flow chart of the path selection method provided in the embodiment of the present application is as follows. The execution subject of this method can be a server or other server, and this embodiment is not particularly limited here. Figure 2 As shown, the method may include:

[0064] S201: Determine a target node corresponding to a data pull request in a wide area storage service cluster.

[0065] The pull data request can refer to a request initiated by a client or an application to a wide-area storage service cluster to obtain specific data resources. The pull data request can include demand data information and target node information. The demand data information can be description information of specific data to be obtained. The target node information can be information indicating a node from which the data is to be pulled.

[0066] The method for determining the target node corresponding to the pull data request in the wide-area storage service cluster can include determining the target node from the storage service nodes in the wide-area storage service cluster according to the target node information in the pull data request.

[0067] S202, determining a plurality of target paths between the target node according to a node path graph in the wide-area storage service cluster.

[0068] The node path graph can refer to a graph used to visually represent the nodes in the wide-area storage service cluster and the connection relationship between the nodes.

[0069] The target path can refer to all connection paths between the storage service node that initiates the pull data request and the target node.

[0070] The method for determining the plurality of target paths between the target node can include taking the storage service node that initiates the pull data request as a starting node, and determining all possible connection modes between the starting node and the target node in the node path graph, including direct connection and indirect connection through other storage service nodes as intermediate nodes.

[0071] S203, determining path weights of the plurality of target paths according to attribute information of the nodes in the wide-area storage service cluster.

[0072] The attribute information of the nodes can refer to various performance-related parameters of the nodes, such as bandwidth, delay, packet loss rate, etc.

[0073] The path weight can refer to a comprehensive index for measuring the advantages and disadvantages of a path relative to other paths, which can be calculated based on the attribute information of the nodes on the path.

[0074] In the embodiments of the present application, the method for determining the path weights of the plurality of target paths according to the attribute information of the nodes in the wide-area storage service cluster can include:

[0075] determining node weights according to the attribute information of the nodes in the wide-area storage service cluster;

[0076] determining the path weights of the plurality of target paths according to the node weights.

[0077] The node weight can refer to a numerical index for quantifying and evaluating the relative importance, performance, or fitness of each node in the wide-area storage service cluster, and comprehensively reflects various attribute information of the node.

[0078] According to the node weight, the method for determining the path weight of the plurality of target paths can include summing the weights of all nodes on the path, and can also include defining a weight calculation model in advance, assigning an initial weight to each node on the path, and then combining the node weight to comprehensively obtain the path weight of the plurality of target paths.

[0079] In the embodiments of the present application, the method for determining the node weight according to the attribute information of the nodes in the wide-area storage service cluster can include:

[0080] According to the network latency information in the attribute information, determining a first weight of the node, according to the bandwidth capacity information in the attribute information, determining a second weight of the node, and according to the cost information in the attribute information, determining a third weight of the node.

[0081] According to the first weight of the node, the second weight of the node, and the third weight of the node, determining the node weight, wherein the first weight of the node and the third weight of the node are in an inverse proportional relationship with the node weight, and the second weight of the node is in a positive proportional relationship with the node weight.

[0082] The first weight of the node, the second weight of the node, and the third weight of the node can refer to weights calculated based on the network latency information, the bandwidth capacity information, and the cost information, respectively. The calculation method can include first standardizing the information, and then determining the first, second, and third weights of the node according to a preset weight mapping table.

[0083] The method for determining the node weight can include weighted summing the first, second, and third weights of the node according to a certain proportion. The lower the network latency and cost and the higher the bandwidth capacity, the higher the relative importance of the node should be. Therefore, the first weight of the node and the third weight of the node can be set in an inverse proportional relationship with the node weight, and the second weight of the node can be set in a positive proportional relationship with the node weight.

[0084] In other embodiments, the first weight of the node and the third weight of the node can be set in a positive proportional relationship with the node weight, and the second weight of the node can be set in an inverse proportional relationship with the node weight. At this time, the lower the weight of the node, the higher the relative importance of the node.

[0085] S204, determining a shortest path in the plurality of target paths according to the path weight.

[0086] The shortest path can refer to a path with the best comprehensive performance in the plurality of target paths.

[0087] The method for determining the shortest path may include first obtaining path weights of multiple target paths based on path weights; if the weight of a node in the path is higher, the relative importance of the node is higher, then the target path with the highest path weight is selected as the shortest path; if the weight of a node in the path is lower, the relative importance of the node is higher, then the target path with the lowest path weight is selected as the shortest path.

[0088] S205 : Determine the next shortest path based on the path weight, the shortest path, and a preset weight difference threshold. Both the shortest path and the next shortest path are used to transmit a data pull request to the target node.

[0089] The second shortest path may refer to a path whose weight difference from the shortest path is within a preset range, and there may be multiple second shortest paths.

[0090] In the embodiment of the present application, the method for determining the second shortest path based on the path weight, the shortest path, and a preset weight difference threshold may include:

[0091] Determine the weight differences between multiple target paths and the shortest path based on the path weights and the path weight of the shortest path;

[0092] The second shortest path is determined according to the weight difference and a preset weight difference threshold, where the second shortest path is a path whose weight difference with the shortest path is not greater than the preset weight difference threshold.

[0093] The weight difference may refer to the absolute value of the weight difference between the target path excluding the shortest path and the shortest path. The smaller the weight difference, the smaller the gap between the target path and the shortest path. The larger the weight difference, the larger the gap between the target path and the shortest path.

[0094] The method for determining the weight difference may include subtracting the weights of the multiple target paths from the weight of the shortest path, and then calculating the absolute value to obtain the weight difference.

[0095] The method for determining the second shortest path may include comparing the weight differences between the multiple target paths and the shortest path with a preset weight difference threshold, and if the weight difference is not greater than the weight difference threshold, then determining that the target path is the second shortest path, for example Figure 3 Node 1 needs to pull data from node 3 and calculate the shortest path ranking according to the Dijkstra algorithm:

[0096] First place: Node 1 -> Node 2 -> Node 3, weight 3 + 5 = 8;

[0097] Second place: Node 1 -> Node 4 -> Node 3, weight 6 + 3 = 9;

[0098] Third place: Node 1->Node 3, weight 20;

[0099] The path of the first is determined as the shortest path, and the weight difference of the paths of the second and the third is calculated: |8-9|=1, |8-20|=12.

[0100] The weight difference is 1 and 12 respectively, the weight difference of the second is less than the preset weight difference threshold 5, and the weight difference of the third is greater than the preset weight difference threshold 5, so the path of the second is determined as the second shortest path.

[0101] Therefore, node 1 simultaneously issues a pull data request to the next node of the shortest path, node 2, and the next node of the second shortest path, node 4.

[0102] The path selection method provided by the embodiment of the application can determine the shortest path through the node path graph and the attribute information of the node, then calculate the weight difference between other paths and the shortest path, determine the second shortest path with a weight close to that of the shortest path according to the weight difference and the preset weight difference threshold, and simultaneously transmit the pull data request through the shortest path and the second shortest path, so as to improve the data pull efficiency and avoid the problem of low efficiency caused by re-requesting when a certain path suddenly fails.

[0103] Figure 4 The flowchart of another path selection method provided by the embodiment of the application is shown in FIG. 2, and the method comprises the following steps. Figure 4

[0104] S401, determining the target node corresponding to the pull data request in the wide-area storage service cluster.

[0105] The pull data request can refer to that the client or application initiates a request to the wide-area storage service cluster to obtain specific data resources, and the pull data request can include demand data information and target node information.

[0106] S402, determining the shortest path between the target nodes according to the undirected weighted graph in the wide-area storage service cluster.

[0107] The undirected weighted graph can refer to a structure graph composed of nodes and edges connecting the nodes, each edge has a weight associated therewith, which represents the cost, distance or other measurement standard of the edge. In particular, the edge in the undirected weighted graph is directionless.

[0108] The method for determining the shortest path between the target nodes can comprise determining all paths that can reach the target nodes in the wide-area storage service cluster according to the undirected weighted graph, calculating the weight of each path according to the edges with weights between the nodes in the path, and determining the shortest path in the paths according to the weight of each path. ​

[0109] S403 : Determine the next shortest path according to the undirected weighted graph, the shortest path, and a preset weight difference threshold. Both the shortest path and the next shortest path are used to transmit a request to pull data to the target node.

[0110] Among them, the method for determining the second shortest path may include comparing the weight differences between multiple target paths and the shortest path with a preset weight difference threshold. If the weight difference is not greater than the weight difference threshold, the target path is determined to be the second shortest path.

[0111] Another path selection method provided in an embodiment of the present application can determine the shortest path based on an undirected weighted graph, and then calculate the difference in weights between other paths and the shortest path. Based on the weight difference and a preset weight difference threshold, a second-shortest path with a weight close to the shortest path is determined, and the technical means of simultaneously transmitting data pull requests through the shortest path and the second-shortest path can be used to achieve the effect of improving data pulling efficiency.

[0112] Figure 5 A flow chart of a data extraction method provided in an embodiment of the present application is shown as follows: Figure 5 As shown, the method includes:

[0113] S501 : In response to receiving a data pull request, determine a shortest path and a second shortest path for transmitting the data pull request, where the shortest path and the second shortest path are obtained according to the above-mentioned path selection method.

[0114] The shortest path may refer to a path with the best overall performance among multiple target paths.

[0115] The second shortest path may refer to a path whose weight difference from the shortest path is within a preset range, and there may be multiple second shortest paths.

[0116] S502: Transmit a data pull request to a target node according to the shortest path and the second shortest path.

[0117] The target node may refer to a storage service node whose information needs to be pulled.

[0118] S503: Receive returned target data, where the target data is the fastest returned demand data among the shortest path and the second shortest path.

[0119] Among them, the shortest path and the second shortest path both pull the required data from the target node to the storage service node that issues the data pull request based on the data pull request. The storage service node that issues the data pull request receives the required data returned first and uses it as the target data. The required data returned later is discarded.

[0120] In this embodiment of the present application, after receiving the returned target data, the method may further include:

[0121] The adjacent intermediate nodes in the shortest path and the second shortest path cache the demand data to obtain the cached demand data. The adjacent intermediate nodes are intermediate nodes connected to the storage service node, and the intermediate nodes are nodes between the storage service node and the target node.

[0122] If the shortest path and the second shortest path receive a new data pull request, and the target of the new data pull request is the demand data, the storage service node receives the demand data of the returned target cache, and the demand data of the target cache is the demand data of the cache returned fastest among the adjacent intermediate nodes of the shortest path and the adjacent intermediate nodes of the second shortest path.

[0123] Among them, when the shortest path and the second shortest path pull the required data according to the data pull request, the required data can be cached at the adjacent intermediate node. The adjacent intermediate node is the node in the intermediate node connected to the storage service node that issued the data pull request. If the storage service node issues the same data pull request again, after selecting the shortest path and the second shortest path, when transmitting the data pull request according to the shortest path and the second shortest path, it passes through the adjacent intermediate nodes of the shortest path and the second shortest path. At this time, the adjacent intermediate node can directly return the previously cached required data, compared with transmitting the required data to the target node and then returning it, which saves time and improves efficiency.

[0124] In other embodiments, the nodes for caching the required data can be set according to the specific situation. The caching function can be turned on for the intermediate nodes with larger memory and turned off for the intermediate nodes with smaller memory. On the basis of ensuring the working capacity of the nodes, the efficiency of data pulling is improved.

[0125] When the adjacent intermediate nodes of the shortest path and the adjacent intermediate nodes of the second shortest path return the cached demand data, the storage service node that issues the data pull request receives the cached demand data returned first and uses it as the target cached demand data. The cached demand data returned later is discarded.

[0126] In the embodiment of the present application, after caching the demand data at adjacent intermediate nodes in the shortest path and the second shortest path and obtaining the cached demand data, the method may further include:

[0127] If the shortest path and the second shortest path do not receive a new data pull request, or the target of the new data pull request is not the required data, then the cache retention time of the adjacent intermediate node is obtained;

[0128] If the cache keeping duration is greater than the preset cache keeping duration threshold, the cached demand data is deleted.

[0129] The cache keeping duration can refer to the cache duration of the demand data at the adjacent intermediate node.

[0130] When the cache keeping duration is greater than the preset cache keeping duration threshold, it indicates that the demand data has cooled down, and in order to save the node space, the cached demand data that has cooled down can be deleted, wherein the cache keeping duration threshold can be set according to specific conditions, which is not limited here.

[0131] The data pulling method provided by the embodiment of the application can cache the demand data when pulling data, return the cached demand data when a new pulling data request arrives, and delete the cached data that has not been pulled for a long time, thereby improving the efficiency of data pulling on the basis of ensuring the working capacity of the node.

[0132] A wide-area storage service cluster system includes a center service node and a plurality of storage service nodes corresponding to different regions, the center service node is configured to periodically detect attribute information and reachability information of the plurality of storage service nodes, update a node path graph according to the reachability information, and the storage service node is configured to execute the path selection method.

[0133] The storage service node can include a BOSS service unit, and the center service node can be connected to the BOSS service unit in the plurality of storage service nodes, and push the periodically detected attribute information and reachability information and the node path graph to the BOSS service unit in the storage service node.

[0134] The reachability information can refer to data describing the state of each storage service node itself and the connection state between each storage service node.

[0135] For example Figure 6 The way of updating the node path graph according to the reachability information can include updating the state of each node itself and the connection state between nodes in the node path graph, when it is detected that a network failure occurs between node 2 and node 3, the connection state between node 2 and node 3 in the node path graph is updated to unreachable, and the connection between node 2 and node 3 is disconnected, to avoid the influence of the failure connection on the pulling data process.

[0136] In other embodiments, the center service node can also update an undirected weighted graph according to the reachability information and the attribute information, specifically, update the nodes and edges in the undirected weighted graph according to the reachability information, and update the weights of the edges in the undirected weighted graph according to the attribute information.

[0137] The storage service node can determine the target path and the weight of each target path based on the attribute information and the node path graph, and then execute the above-mentioned path selection method.

[0138] Figure 7 This is a schematic diagram of the structure of the path selection device provided in the embodiment of the present application. Figure 7 As shown, the path selection device 70 includes: a first determination module 701, a second determination module 702, a third determination module 703, a fourth determination module 704 and a fifth determination module 705.

[0139] The first determining module 701 is configured to determine a target node corresponding to a data pull request in the wide area storage service cluster;

[0140] A second determining module 702 is configured to determine multiple target paths to a target node based on a node path graph in the wide area storage service cluster;

[0141] The third determining module 703 is configured to determine the path weights of multiple target paths based on the attribute information of the nodes in the wide area storage service cluster;

[0142] A fourth determining module 704 is configured to determine the shortest path among multiple target paths based on the path weights;

[0143] The fifth determining module 705 is configured to determine the next shortest path according to the path weight, the shortest path, and a preset weight difference threshold, wherein both the shortest path and the next shortest path are used to transmit a data pull request to the target node.

[0144] In this embodiment of the present application, the third determining module 703 may also be used to:

[0145] Determine node weights based on the attribute information of nodes in the wide area storage service cluster;

[0146] According to the node weights, the path weights of multiple target paths are determined.

[0147] In this embodiment of the present application, the third determining module 703 may also be used to:

[0148] Determining a first weight of the node based on network delay information in the attribute information, determining a second weight of the node based on bandwidth capacity information in the attribute information, and determining a third weight of the node based on cost information in the attribute information;

[0149] The node weight is determined according to the first node weight, the second node weight and the third node weight, wherein the first node weight and the third node weight are inversely proportional to the node weight, and the second node weight is in direct proportion to the node weight.

[0150] In the embodiment of the present application, the fifth determining module 705 may also be used to:

[0151] Determine the weight differences between multiple target paths and the shortest path based on the path weights and the path weight of the shortest path;

[0152] The second shortest path is determined according to the weight difference and a preset weight difference threshold, where the second shortest path is a path whose weight difference with the shortest path is not greater than the preset weight difference threshold.

[0153] As can be seen from the above, the path selection device of the embodiment of the present application consists of a first determination module 701, which is used to determine the target node corresponding to the data pull request in the wide area storage service cluster; a second determination module 702, which is used to determine multiple target paths between the target nodes based on the node path map in the wide area storage service cluster; a third determination module 703, which is used to determine the path weights of multiple target paths based on the attribute information of the nodes in the wide area storage service cluster; a fourth determination module 704, which is used to determine the shortest path among multiple target paths based on the path weight; and a fifth determination module 705, which is used to determine the second shortest path based on the path weight, the shortest path and a preset weight difference threshold. Both the shortest path and the second shortest path are used to transmit the data pull request to the target node. Therefore, the embodiment of the present application can determine the shortest path through the node path graph and the node attribute information, and then calculate the difference in weights between other paths and the shortest path. According to the difference in weights and the preset weight difference threshold, the second shortest path with a weight close to the shortest path is determined, and the technical means of simultaneously transmitting the data pulling request through the shortest path and the second shortest path can be used to achieve the effect of improving data pulling efficiency. It can also avoid the problem of inefficiency caused by re-requesting when a sudden failure of a path causes the request to fail.

[0154] Figure 8 This is a structural diagram of the data pulling device provided in the embodiment of the present application. Figure 8 As shown, the data pulling device 80 includes: a response module 801, a transmission module 802 and a receiving module 803.

[0155] The response module 801 determines, in response to receiving the pull data request, the shortest path and the second shortest path for transmitting the pull data request, where the shortest path and the second shortest path are obtained according to the above-mentioned path selection method;

[0156] The transmission module 802 transmits the pull data request to the target node according to the shortest path and the second shortest path;

[0157] The receiving module 803 is configured to receive the returned target data, where the target data is the fastest returned demand data among the shortest path and the second shortest path.

[0158] In the embodiment of the present application, the receiving module 803 may also be used to:

[0159] The adjacent intermediate nodes in the shortest path and the second shortest path cache the demand data to obtain the cached demand data. The adjacent intermediate nodes are intermediate nodes connected to the storage service node, and the intermediate nodes are nodes between the storage service node and the target node.

[0160] If the shortest path and the second shortest path receive a new data pull request, and the target of the new data pull request is the demand data, the storage service node receives the demand data of the returned target cache, and the demand data of the target cache is the demand data of the cache returned fastest among the adjacent intermediate nodes of the shortest path and the adjacent intermediate nodes of the second shortest path.

[0161] In the embodiment of the present application, the receiving module 803 may also be used to:

[0162] If the shortest path and the second shortest path do not receive a new data pull request, or the target of the new data pull request is not the required data, then the cache retention time of the adjacent intermediate node is obtained;

[0163] If the cache retention time is longer than the preset cache retention time threshold, the cached demand data is deleted.

[0164] As can be seen from the above, the path selection device of the embodiment of the present application is composed of a response module 801, which responds to the receipt of a data pull request and determines the shortest path and the second shortest path for transmitting the data pull request, and the shortest path and the second shortest path are obtained according to the above path selection method; a transmission module 802, which transmits the data pull request to the target node according to the shortest path and the second shortest path; and a receiving module 803, which is used to receive the returned target data, and the target data is the fastest returned demand data among the shortest path and the second shortest path. Therefore, the embodiment of the present application can cache the demand data when pulling data, and when a new data pull request arrives, return the cached demand data, and delete the cached data that has not been pulled for a long time, thereby improving the efficiency of data pulling while ensuring the working capacity of the node.

[0165] Figure 9 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 9 As shown, the electronic device 90 provided in this embodiment includes: at least one processor 901 and a memory 902. Optionally, the device 90 further includes a communication component 903. The processor 901, the memory 902, and the communication component 903 are connected via a bus 904.

[0166] During the specific implementation process, at least one processor 901 executes the computer-executable instructions stored in the memory 902, so that the at least one processor 901 performs the above method.

[0167] The specific implementation process of the processor 901 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0168] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly implemented by a hardware processor or implemented by a combination of hardware and software modules in the processor.

[0169] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk memory.

[0170] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be classified into address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0171] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0172] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0173] The above-mentioned readable storage medium can be realized by any type of volatile or nonvolatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0174] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0175] The division of units is only a logical functional division, and in actual implementation, there can be another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0176] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment.

[0177] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0178] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0179] It can be understood by those skilled in the art that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The aforementioned program can be stored in a computer readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, and various media that can store program codes.

[0180] Finally, it should be noted that: those skilled in the art will easily think of other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application that follow the general principles of the present application and include common knowledge or conventional technical means in the art that are not disclosed in the present application, and is not limited to the precise structure described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is only limited by the appended claims.

Claims

1. A path selection method, characterized in that: A storage service node in a wide-area storage service cluster, wherein the wide-area storage service cluster includes storage service nodes corresponding to multiple regions, and the storage service nodes store data corresponding to the regions. The path selection method includes: Determine a target node corresponding to the data pull request in the wide area storage service cluster; Determining multiple target paths to the target node based on a node path graph in the wide area storage service cluster; Determining path weights of the plurality of target paths according to attribute information of nodes in the wide area storage service cluster; determining a shortest path among the plurality of target paths according to the path weights; A second shortest path is determined according to the path weight, the shortest path, and a preset weight difference threshold, and both the shortest path and the second shortest path are used to transmit the data pull request to the target node.

2. The method according to claim 1, characterized in that The determining the second shortest path according to the path weight, the shortest path, and a preset weight difference threshold includes: determining weight differences between the plurality of target paths and the shortest path based on the path weights and the path weight of the shortest path; The second shortest path is determined according to the weight difference and the preset weight difference threshold, where the second shortest path is a path whose weight difference with the shortest path is not greater than the preset weight difference threshold.

3. The method according to claim 1 or 2, characterized in that The determining, based on the attribute information of the nodes in the wide area storage service cluster, the path weights of the plurality of target paths includes: Determining node weights based on attribute information of nodes in the wide area storage service cluster; The path weights of the multiple target paths are determined according to the node weights.

4. The method according to claim 3, characterized in that Determining the node weight according to the attribute information of the nodes in the wide area storage service cluster includes: Determining a first weight of the node based on the network delay information in the attribute information, determining a second weight of the node based on the bandwidth capacity information in the attribute information, and determining a third weight of the node based on the cost information in the attribute information; The node weight is determined according to the first node weight, the second node weight and the third node weight, wherein the first node weight and the third node weight are inversely proportional to the node weight, and the second node weight is in direct proportion to the node weight.

5. A data pulling method, characterized in that: Storage service nodes used in the wide-area storage service cluster include: In response to receiving a pull data request, determining a shortest path and a second shortest path for transmitting the pull data request, wherein the shortest path and the second shortest path are obtained according to the path selection method according to any one of claims 1 to 4; Transmitting the data pull request to a target node according to the shortest path and the second shortest path; Receive returned target data, where the target data is the fastest returned demand data among the shortest path and the second shortest path.

6. The method according to claim 5, characterized in that After receiving the returned target data, the method further includes: The adjacent intermediate nodes in the shortest path and the second shortest path cache the demand data to obtain the cached demand data, wherein the adjacent intermediate nodes are intermediate nodes connected to the storage service node, and the intermediate nodes are nodes between the storage service node and the target node; If the shortest path and the second shortest path receive a new data pull request, and the target of the new data pull request is the demand data, the storage service node receives the demand data of the returned target cache, and the demand data of the target cache is the demand data of the cache returned fastest among the adjacent intermediate nodes of the shortest path and the adjacent intermediate nodes of the second shortest path.

7. The method according to claim 6, characterized in that After caching the demand data at adjacent intermediate nodes in the shortest path and the second shortest path to obtain the cached demand data, the method further includes: If the shortest path and the second shortest path do not receive a new data pull request, or the target of the new data pull request is not the required data, then obtaining the cache retention time of the adjacent intermediate node; If the cache retention time is longer than a preset cache retention time threshold, the cached demand data is deleted.

8. A wide area storage service cluster system, characterized in that: It includes a central service node and storage service nodes corresponding to multiple regions. The central service node is used to periodically detect the attribute information and reachability information of the multiple storage service nodes, and update the node path map according to the reachability information. The storage service node is used to execute the path selection method described in any one of claims 1 to 4.

9. A path selection device, characterized in that: include: A first determination module is used to determine a target node corresponding to a data pull request in the wide area storage service cluster; A second determining module is configured to determine a plurality of target paths to the target node based on a node path graph in the wide area storage service cluster; A third determining module, configured to determine path weights of the plurality of target paths based on attribute information of nodes in the wide area storage service cluster; a fourth determining module, configured to determine a shortest path among the plurality of target paths according to the path weights; A fifth determination module is used to determine a second shortest path according to the path weight, the shortest path and a preset weight difference threshold, wherein both the shortest path and the second shortest path are used to transmit the data pull request to the target node.

10. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 4.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 4 when executed by a processor.

12. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 4 when the computer program is executed by a processor.

Images