Storage flow distribution method, electronic equipment and storage medium

Abstract

The invention discloses a storage traffic distribution method, electronic equipment and a storage medium. The method comprises the following steps: receiving a storage access request for a target data stream; determining a target distribution strategy based on the configuration information; and according to the target allocation strategy, allocating the target data stream to one storage node in the distributed storage system for storage access. Through the method, the flexibility of storage flow distribution can be improved.

Description

Technical Field

[0001] This application relates to the field of data storage technologies, and particularly to a storage traffic allocation method, an electronic device, and a storage medium. Background Art

[0002] In the field of data storage technologies, for a network disk storage system, the storage service, as the core storage module, is responsible for actually storing users' files and related data. The file management service on the business side is mainly responsible for processing operations such as file upload, download, management, and sharing by users. In a distributed cluster deployment mode, to effectively manage the read and write traffic between the file management service and the storage service, a series of storage traffic allocation methods need to be adopted. However, most of the existing storage traffic allocation methods are limited to a single allocation method, resulting in difficulty in making adjustments when facing different business requirements and reducing the flexibility of storage traffic allocation. Summary of the Invention

[0003] The main technical problem to be solved by this application is to provide a storage traffic allocation method, an electronic device, and a storage medium that can improve the flexibility of storage traffic allocation.

[0004] To solve the above technical problem, one technical solution adopted by this application is to provide a storage traffic allocation method, which includes: a current node receives a storage access request for a target data stream, where the current node is a storage node in a distributed storage system; determining a target allocation strategy based on the configuration information of the current node; and allocating the target data stream to a storage node in the distributed storage system for storage access according to the target allocation strategy.

[0005] To solve the above technical problem, another technical solution adopted by this application is: to provide an electronic device, which includes a memory and a processor, the memory stores program instructions, and the processor is configured to execute the program instructions to implement the above storage traffic allocation method.

[0006] To solve the above technical problem, another technical solution adopted by this application is: to provide a computer-readable storage medium, which is used to store program instructions, and the program instructions can be executed to implement the above storage traffic allocation method.

[0007] In the above solution, the current node receives a storage access request for a target data stream, determines a target allocation strategy based on the configuration information of the current node, and allocates the target data stream to a storage node in the distributed storage system for storage access according to the target allocation strategy. This application flexibly determines the corresponding target allocation strategy based on the configuration information of the current node to meet different business requirements, thereby improving flexibility. Brief Description of the Drawings

[0008] Figure 1 It is a schematic flowchart of an embodiment of the storage traffic allocation method provided by the present application;

[0009] Figure 2 It is a schematic diagram of the traffic allocation architecture of the distributed storage system provided by the present application;

[0010] Figures 3a to 3c It is a partial schematic flowchart of the configuration information update provided by the present application;

[0011] Figure 4 It is a partial schematic flowchart of a specific embodiment of the storage traffic allocation method provided by the present application;

[0012] Figure 5 It is a schematic flowchart of an embodiment of the storage traffic allocation device of the present application;

[0013] Figure 6 It is a schematic framework diagram of an embodiment of the electronic device of the present application;

[0014] Figure 7 It is a schematic framework diagram of an embodiment of the computer-readable storage medium of the present application. Detailed Embodiments

[0015] To make the objectives, technical solutions and effects of the present application clearer and more definite, the following further elaborates on the present application with reference to the accompanying drawings and by way of examples.

[0016] It should be noted that the term "several" herein means at least one, and the terms "first", "second", etc. are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. The term "and / or" merely describes the associated relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the associated objects before and after. In addition, the term "at least one" herein means any one of multiple or at least two of any combination of multiple, for example, including at least one of A, B, and C, which may represent including any one or multiple elements selected from the set composed of A, B, and C.

[0017] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of an embodiment of the storage traffic allocation method provided by the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 1 the process sequence shown. As Figure 1 shown, this embodiment includes:

[0018] Step S11: Receive a storage access request for the target data stream.

[0019] Herein, the current node is a storage node in a distributed storage system.

[0020] In one embodiment, the storage node is a core component in the distributed storage system, and the distributed storage system includes at least one storage node. The storage node includes a front-end service, a back-end service, and a storage space. The storage space is used for storing and accessing data streams. The storage traffic allocation method in this embodiment can be executed by the back-end service of the current node. For example, the target data stream is allocated to the front-end service of the current node (a storage node), and the storage access request is successively proxied to the back-end service of the current node. Moreover, the current node undertakes the target data stream transmitted from the outside. When the back-end service selects a storage gateway of another node other than the current node, the target data stream is allocated to the storage space in the other node (another storage node). When the back-end service selects the storage gateway of the current node, the target data stream is allocated to the storage space in the current node.

[0021] Combined with Figure 2 For illustration, Figure 2 is a schematic diagram of the traffic allocation architecture of the distributed storage system provided by this application. The distributed storage system includes at least one storage node, such as Figure 2 Node 1, Node 2,..., Node N in. Herein, each storage node includes a network disk front-end, a network disk back-end, and a network disk storage, which respectively correspond to the front-end service, the back-end service, and the storage space described above (i.e., the network disk front-end corresponds to the front-end service, which will not be elaborated here). Users can access the network disk service cluster through a Chrome web page, an API (Application Programming Interface), or a CLI (Command Line Interface) for web access or API access, so as to evenly allocate the data to be uploaded to the network disk front-ends of each storage node based on the load balancing module. Then, the network disk back-ends of each storage node receive a storage access request for the target data stream (i.e., the data evenly allocated to the corresponding storage node through the load balancing module). Herein, the network disk service cluster includes a load balancing module and at least one storage node, and the network disk storages corresponding to each storage node together form a storage cluster.

[0022] Step S12: Determine the target allocation policy based on the configuration information.

[0023] The target allocation policy is to determine the storage node for storing and accessing the target data stream based on the storage traffic overlap.

[0024] In one embodiment, the configuration information herein can be read from the Redis cache before determining the target allocation policy based on the configuration information of the current node. The Redis cache can store the configuration information in memory to provide fast read and write access, that is, the current node can quickly read the corresponding configuration information from the Redis cache.

[0025] In another embodiment, the current node can include a front-end service, a back-end service, and a storage space. The back-end service can be used to obtain configuration information, and the storage space is used for storage access of data streams. The storage traffic allocation method is executed by the back-end service of the current node. And the configuration information can be read from the Redis cache before determining the target allocation policy based on the configuration information of the current node. For specific descriptions, reference can be made to the relevant descriptions in step S11 and step S12, which will not be elaborated here.

[0026] In another embodiment, the configuration information can include a traffic allocation type, that is, the target allocation policy is determined using the traffic allocation type. In response to the traffic allocation type being the first allocation type, the target allocation policy is determined to be the priority traffic non-overlapping allocation policy. In response to the traffic allocation type being the second allocation type, the target allocation policy is determined to be the normal allocation policy.

[0027] In a specific embodiment, the traffic allocation type can be stored in the configuration information in ways such as numerical encoding, string identification, enumeration type, environment variables, etc. For example, integers or specific encodings are used to represent different traffic allocation types. Specifically, 1 can represent the priority traffic non-overlapping allocation policy, and 0 represents the normal allocation policy.

[0028] Step S13: Allocate the target data stream to a storage node in the distributed storage system for storage access according to the target allocation policy.

[0029] The storage access in this embodiment can be understood as read and write operations. In other words, according to the target allocation policy, the target data stream is written into a corresponding storage node in the distributed storage system for storage, or according to the target allocation policy, the target data stream in a corresponding storage node in the distributed storage system is read.

[0030] It should be noted that, to ensure data availability and fault tolerance, the distributed storage system can also adopt a replication strategy. That is, the target data stream can store replicas on multiple other storage nodes. In the case of reading the corresponding target data stream from a storage node, the distributed storage system can read data from the corresponding storage node according to the target allocation strategy. And due to the existence of replicas, even if the reading is not targeted at the corresponding storage node where the data was initially written, the distributed storage system can still read the corresponding data from other storage nodes. By using the replication strategy, the distributed storage system can reduce the situation where some storage nodes are overloaded while other storage nodes are idle.

[0031] Specifically, when the backend service of a storage node writes the target data stream into the storage space of this storage node, the backend service of the storage node can also send a replica of the target data stream to the storage spaces corresponding to at least one other storage node. Among them, the replica can be sent synchronously or asynchronously, etc., and can be selected according to the actual situation, which will not be elaborated here. And it can be determined according to the actual situation how many storage nodes the replica needs to be sent to. The distributed storage system can read data from the corresponding storage node according to the target allocation strategy. And due to the existence of replicas, even if the reading is not targeted at the corresponding storage node where the data was initially written, the distributed storage system can still read the corresponding data from other storage nodes.

[0032] In one embodiment, when the target allocation strategy is the priority traffic non-overlapping allocation strategy, the target data stream is allocated to the current node for storage access.

[0033] Continue to combine Figure 2 For example, if the current node is Node 1, when the target allocation strategy is the priority traffic non-overlapping allocation strategy, the target data stream is allocated to the network disk storage corresponding to Node 1 for storage access. If the current node is Node 2, when the target allocation strategy is the priority traffic non-overlapping allocation strategy, the target data stream is allocated to the network disk storage corresponding to Node 2 for storage access. Specifically, see the direction indicated by the blue arrow from the backend of the network disk to the network disk storage in the figure.

[0034] In a specific embodiment, the node address of the current node can be called to allocate the target data stream to the current node for storage access. Of course, the storage port corresponding to the current node can also be called to allocate the target data stream to the current node for storage access, which is not limited here.

[0035] For example, when the target allocation strategy is the priority traffic non-overlapping allocation strategy, the node address (such as the IP address) and the corresponding storage port of the current node are called for storage access.

[0036] In yet another specific embodiment, considering that there may be a situation where the current node fails or the storage space of the current node does not deploy storage services, the configuration information may further include a gateway list, and the gateway list contains several node addresses. In response to the result of allocating storage access to the current node being a failure, polling starts from the node address after the node address of the current node in the gateway list to select the storage node corresponding to the currently polled node address, and the target data stream is allocated to the selected storage node for storage access.

[0037] For example, the configuration information may further include a gateway list, and the gateway list contains the A node address, the B node address, and the C node address, and the current node is the A node. In response to the result of allocating storage access to the A node being a failure, polling starts from the node address after the A node address of the A node in the gateway list to select the storage node corresponding to the currently polled node address, that is, the B node address, and the target data stream is allocated to the selected storage node (the B node) for storage access.

[0038] Continue to combine Figure 2 For example, if the current node is node 1 and the storage node corresponding to the currently polled node address selected is node 2, the target data stream is allocated to the network disk storage corresponding to node 2 for storage access. Specifically, see the direction indicated by the orange arrow from the network disk backend in node 1 to the network disk storage in the figure.

[0039] In yet another specific embodiment, the node address of the current node is called to allocate the target data stream to the current node for storage access. The configuration information further includes a gateway list, and the gateway list contains several node addresses. In response to the result of allocating storage access to the current node being a failure, polling starts from the node address after the node address of the current node in the gateway list to select the storage node corresponding to the currently polled node address, and the target data stream is allocated to the selected storage node for storage access. For the specific description, reference can be made to the relevant description in step S13, and details are not described here.

[0040] In yet another embodiment, when the target allocation strategy is a normal allocation strategy, the target data stream is allocated to the first storage node of the target domain name for storage access, or the target data stream is allocated to the second storage node for storage access by using a polling mechanism.

[0041] In a specific embodiment, the target data stream can be preferentially allocated to the first storage node of the target domain name for storage access. The configuration information may further include a domain name field. In response to the existence of a configured domain name in the domain name field, the configured domain name is used as the target domain name, and the target data stream is allocated to the first storage node of the target domain name for storage access. In response to the non-existence of a configured domain name in the domain name field, a polling mechanism is used to allocate the target data stream to the second storage node for storage access.

[0042] For example, the configuration information further includes a domain name field. In the case where ecd.com is included in the domain name field, ecd.com represents the configured domain name. The ecd.com is used as the target domain name, and the target data stream is allocated to the first storage node of ecd.com for storage access. The first storage node of ecd.com may include at least one storage node, which is not limited here. In the case where there is no configured domain name in the domain name field, a polling mechanism is used to allocate the target data stream to the second storage node for storage access.

[0043] In another specific embodiment, the configuration information further includes a domain name field. In response to the existence of a configured domain name in the domain name field, the configured domain name is used as the target domain name, and the target data stream is allocated to the first storage node of the target domain name for storage access. In response to the existence of a configured domain name in the configuration information, but the result of allocating it to the first storage node for storage access fails, a polling mechanism is used to allocate the target data stream to the second storage node for storage access.

[0044] For example, the configuration information further includes a domain name field. In the case where ecd.com is included in the domain name field, ecd.com represents the configured domain name. The ecd.com is used as the target domain name, and the target data stream is allocated to the first storage node of ecd.com for storage access. In response to the failure of allocating the target data stream to the first storage node of ecd.com for storage access, a polling mechanism is used to allocate the target data stream to the second storage node for storage access.

[0045] In another specific embodiment, the configuration information further includes a domain name field. In response to the existence of a configured domain name in the domain name field, the configured domain name is used as the target domain name, and the target data stream is allocated to the first storage node of the target domain name for storage access. In response to the non-existence of a configured domain name in the domain name field, a polling mechanism is used to allocate the target data stream to the second storage node for storage access. In response to the existence of a configured domain name in the configuration information, but the result of allocating it to the first storage node for storage access fails, a polling mechanism is used to allocate the target data stream to the second storage node for storage access. The specific description can refer to the relevant description in step S13, which will not be elaborated here.

[0046] In yet another specific embodiment, the configuration information further includes a gateway list, and the gateway list includes a number of node addresses. Polling can be performed according to each node address in the gateway list to select the storage node corresponding to the currently polled node address as the second storage node, and allocate the target data stream to the second storage node for storage access.

[0047] For example, the storage node corresponding to the currently polled node address selected can be a storage node that has not been allocated in this round, so as to balance the allocation of the target data stream and avoid overloading of some storage nodes. Of course, the selection range is not limited to unallocated nodes, and can also be selected from the storage nodes that have been allocated and unallocated in this round, and can be specifically adjusted according to actual requirements, which is not limited here.

[0048] In yet another specific embodiment, considering that there may be abnormal storage access in the storage node, the node addresses corresponding to the abnormal storage nodes can be filtered to ensure the effectiveness and reliability of the storage traffic allocation. In response to the result of allocating the storage access to the second storage node being a failure, filter the node addresses corresponding to the storage nodes that have been allocated in this round and the node address corresponding to the second storage node from the gateway list, to obtain at least one remaining node address. Polling is performed according to the at least one remaining node address to select the storage node corresponding to the currently polled node address, and allocate the target data stream to the selected storage node for storage access.

[0049] For example, the configuration information may further include a gateway list, the gateway list contains node address A, node address B, node address C, node address D, node address E, the current node is node A, and node address A and node address C belong to the node addresses corresponding to the storage nodes that have been allocated in this round. The second storage node is node B. In response to the result of allocating the storage access to node B being a failure, filter the node address A corresponding to node A and the node address C corresponding to node C that have been allocated in this round, and the node address B corresponding to node B from the gateway list, that is, the remaining node addresses are node address D and node address E. Polling is performed according to node address D and node address E to select the storage node corresponding to the currently polled node address, that is, node D, and allocate the target data stream to node D for storage access.

[0050] Among them, filtering the node addresses corresponding to the storage nodes allocated in this round from the gateway list can be deleting the node addresses corresponding to the storage nodes allocated in this round from the gateway list, that is, in the case of the next-round target data stream allocation, the node addresses corresponding to the filtered storage nodes will not be called. Of course, filtering the node addresses corresponding to the storage nodes allocated in this round from the gateway list can also be temporarily deleting the node addresses corresponding to the storage nodes allocated in this round from the gateway list, that is, in the case of the next-round target data stream allocation, the node addresses corresponding to the filtered storage nodes will still be called. Here, it can be adjusted according to actual requirements without limitation.

[0051] In yet another specific embodiment, the configuration information further includes a gateway list, and the gateway list includes several node addresses. Poll according to each node address in the gateway list to select the storage node corresponding to the currently polled node address as the second storage node, and allocate the target data stream to the second storage node for storage access. In response to the result of allocating the storage access to the second storage node being a failure, filter the node addresses corresponding to the storage nodes allocated in this round and the node addresses corresponding to the second storage node from the gateway list to obtain at least one remaining node address. Poll according to the at least one remaining node address to select the storage node corresponding to the currently polled node address, and allocate the target data stream to the selected storage node for storage access. For specific descriptions, reference can be made to the relevant descriptions in step S13, which will not be elaborated here.

[0052] In yet another embodiment, when the target allocation policy is the priority traffic non-overlapping allocation policy, the target data stream is allocated to the current node for storage access. When the target allocation policy is the ordinary allocation policy, the target data stream is allocated to the first storage node of the target domain name for storage access, or the target data stream is allocated to the second storage node for storage access by using a polling mechanism. For specific descriptions, reference can be made to the relevant descriptions in step S13, which will not be elaborated here.

[0053] In yet another embodiment, considering that the business requirements may change, the configuration information can be changed accordingly to adapt to different business requirements, that is, the storage node can immediately adjust the storage access mode according to the changed configuration information. The configuration information is cached in the current node. In response to receiving the changed configuration information, the configuration information cached in the current node is updated with the changed configuration information.

[0054] In a specific embodiment, the configuration information includes the traffic allocation type. In response to receiving the changed traffic allocation type, the changed allocation type information is updated to the configuration information cached in the current node. Additionally, the changed allocation type information can be further updated to the Redis cache, without limitation here. Among them, the target allocation policy can be adjusted through the Web or the CLI (Command Line Interface).

[0055] Combined with Figure 3a For illustration, Figure 3a FIG. is a partial flowchart of the configuration information update provided by the present application. The configuration information can be obtained by reading from the Redis cache. Step S31: The target user adjusts the traffic allocation type according to the actual business requirements to obtain the changed traffic allocation type. Step S32: The backend service in the storage node receives the changed traffic allocation type. Step S33: Update the changed traffic allocation type to the business database. Step S34: In response to the successful update of the business database, the backend service of the storage node updates the changed traffic allocation type to the Redis cache.

[0056] In another specific embodiment, the configuration information includes the domain name field. In response to receiving the changed domain name field, the changed domain name field is updated to the configuration information cached in the current node. Additionally, the changed allocation type information can be further updated to the Redis cache, without limitation here.

[0057] Combined with Figure 3b For illustration, Figure 3b FIG. is a partial flowchart of the configuration information update provided by the present application. Step S35: In response to the existence of a configured domain name or a changed domain name in the storage cluster, initiate a change request, where the storage cluster can include at least one storage node. Step S36: The backend service receives the changed domain name field. Step S37: Update the changed domain name field to the business database. Step S38: In response to the successful update of the business database, the backend service updates the changed domain name field to the Redis cache.

[0058] In another specific embodiment, the configuration information includes the gateway list. In response to receiving the changed gateway list, the changed gateway list is updated to the configuration information cached in the current node. Additionally, the changed allocation type information can be further updated to the Redis cache, without limitation here.

[0059] Combined with Figure 3c For illustration, Figure 3cIt is a partial process schematic diagram of configuration information update provided by this application. Step S39: In response to the existence of a configuration gateway list or a changed gateway list in the storage cluster, initiate a change request, where the storage cluster may include at least one storage node. Step S310: The backend service receives the changed gateway list. Step S311: Update the changed gateway list to the business database. Step S312: In response to the successful update of the business database, the backend service updates the changed gateway list to the Redis cache.

[0060] Of course, the above three can be combined arbitrarily, and there is no limit here. For example, the configuration information includes traffic allocation type and domain name field. In response to receiving the changed traffic allocation type, update the changed allocation type information to the configuration information cached by the current node. In response to receiving the changed domain name field, update the changed domain name field to the configuration information cached by the current node. Another example, the configuration information includes traffic allocation type, domain name field, and gateway list. In response to receiving the changed traffic allocation type, update the changed allocation type information to the configuration information cached by the current node. In response to receiving the changed domain name field, update the changed domain name field to the configuration information cached by the current node. In response to receiving the changed gateway list, update the changed gateway list to the configuration information cached by the current node. It can be set according to actual needs, and there is no limit here.

[0061] Please refer to Figure 4 , Figure 4 It is a partial process schematic diagram of a specific embodiment of the storage traffic allocation method provided by this application. As Figure 4 shown, the storage traffic allocation method provided in this embodiment may include the following steps:

[0062] Step S41: Read the configuration information from the Redis cache. Step S42: Based on the traffic allocation type in the configuration information, determine whether the target allocation policy is the priority traffic non-overlapping allocation policy. If the judgment result in step S42 is yes, execute step S43: Call the node address of the current node to allocate the target data stream to the current node for storage access. Step S44: Judge whether the result of allocating the storage access to the current node is a failure.

[0063] When the judgment result in step S42 is no, step S45: Determine whether there is a configured domain name field in the configuration information. When the judgment result in step S45 is yes, execute step S46: Use the configured domain name as the target domain name, and allocate the target data stream to the first storage node corresponding to the target domain name for storage access. Step S47: Determine whether the result of allocating the storage access to the first storage node is a failure. When the judgment result in step S45 is no, or when the judgment result in step S47 is yes, execute step S48: Use a polling mechanism to allocate the target data stream to the second storage node for storage access. Step S49: Determine whether the result of allocating the storage access to the second storage node is a failure.

[0064] When the judgment result in step S44 is yes, or when the judgment result in step S49 is yes, execute step S410: Filter the node addresses corresponding to the storage nodes already allocated in this round and the node addresses corresponding to the second storage node from the gateway list to obtain at least one remaining node address. Step S411: Poll according to the at least one remaining node address to select the storage node corresponding to the currently polled node address, and allocate the target data stream to the selected storage node for storage access.

[0065] When the judgment result in step S44 is no, or when the judgment result in step S47 is no, or after step S411 is executed, execute step S412: End.

[0066] Please refer to Figure 5 , Figure 5 which is a schematic flowchart of an embodiment of the storage traffic allocation device of the present application. The storage traffic allocation device 500 includes a storage access request receiving module 510, a target allocation policy determination module 520, and a target data stream allocation module 530. The storage access request receiving module 510 is used for the current node to receive a storage access request for the target data stream, where the current node is a storage node in a distributed storage system. The target allocation policy determination module 520 is used to determine the target allocation policy based on the configuration information of the current node. The target data stream allocation module 530 is used to allocate the target data stream to a storage node in the distributed storage system for storage access according to the target allocation policy.

[0067] In some embodiments, the configuration information includes a traffic allocation type. When the target allocation policy determination module 520 executes to determine the target allocation policy based on the configuration information of the current node, it includes: in response to the traffic allocation type being the first allocation type, determining the target allocation policy as a priority traffic non-overlapping allocation policy. In response to the traffic allocation type being the second allocation type, determining the target allocation policy as a normal allocation policy.

[0068] In some embodiments, when the target data stream allocation module 530 specifically executes the allocation of the target data stream to a storage node in the distributed storage system according to the target allocation policy, it includes: when the target allocation policy is the priority traffic non-overlapping allocation policy, allocating the target data stream to the current node for storage access.

[0069] In some embodiments, when the target data stream allocation module 530 specifically executes the allocation of the target data stream to a storage node in the distributed storage system according to the target allocation policy, it includes: when the target allocation policy is the normal allocation policy, allocating the target data stream to the first storage node corresponding to the target domain name for storage access, or allocating the target data stream to the second storage node by using a polling mechanism for storage access.

[0070] In some embodiments, when the target data stream allocation module 530 specifically executes the allocation of the target data stream to a storage node in the distributed storage system according to the target allocation policy, it includes: when the target allocation policy is the priority traffic non-overlapping allocation policy, allocating the target data stream to the current node for storage access. When the target allocation policy is the normal allocation policy, allocating the target data stream to the first storage node corresponding to the target domain name for storage access, or allocating the target data stream to the second storage node by using a polling mechanism for storage access.

[0071] In some embodiments, when the target data stream allocation module 530 specifically executes the allocation of the target data stream to the current node for storage access, it includes: calling the node address of the current node to allocate the target data stream to the current node for storage access.

[0072] In some embodiments, the configuration information further includes a gateway list, and the gateway list contains several node addresses. After the target data stream allocation module 530 specifically executes the allocation of the target data stream to the current node for storage access, it further includes: in response to the result of the allocation to the current node for storage access being a failure, polling starts from the node address after the node address of the current node in the gateway list to select the storage node corresponding to the currently polled node address, and allocating the target data stream to the selected storage node for storage access.

[0073] In some embodiments, when the target data stream allocation module 530 specifically executes allocating the target data stream to the current node for storage access, it includes: calling the node address of the current node to allocate the target data stream to the current node for storage access. The configuration information further includes a gateway list, and the gateway list contains several node addresses. After the target data stream allocation module 530 specifically executes allocating the target data stream to the current node for storage access, it further includes: in response to the result of allocating to the current node for storage access being a failure, polling starts from the node address after the node address of the current node in the gateway list to select the storage node corresponding to the currently polled node address, and allocating the target data stream to the selected storage node for storage access.

[0074] In some embodiments, the configuration information further includes a domain name field. When the target data stream allocation module 530 specifically executes allocating the target data stream to the first storage node corresponding to the target domain name for storage access, or allocating the target data stream to the second storage node using a polling mechanism for storage access, it includes: in response to the domain name field having a configured domain name, using the configured domain name as the target domain name and allocating the target data stream to the first storage node corresponding to the target domain name for storage access. In response to the domain name field not having a configured domain name, allocating the target data stream to the second storage node using a polling mechanism for storage access.

[0075] In some embodiments, the configuration information further includes a domain name field. When the target data stream allocation module 530 specifically executes allocating the target data stream to the first storage node corresponding to the target domain name for storage access, or allocating the target data stream to the second storage node using a polling mechanism for storage access, it includes: in response to the domain name field having a configured domain name, using the configured domain name as the target domain name and allocating the target data stream to the first storage node corresponding to the target domain name for storage access. In response to the configuration information having a configured domain name, but the result of allocating to the first storage node for storage access being a failure, allocating the target data stream to the second storage node using a polling mechanism for storage access.

[0076] In some embodiments, the configuration information further includes a domain name field. When specifically executing the assignment of the target data stream to the first storage node associated with the target domain name for storage access, or when using a polling mechanism to assign the target data stream to the second storage node for storage access, the target data stream assignment module 530 includes: in response to the presence of a configured domain name in the domain name field, using the configured domain name as the target domain name and assigning the target data stream to the first storage node associated with the target domain name for storage access; in response to the absence of a configured domain name in the domain name field, using a polling mechanism to assign the target data stream to the second storage node for storage access; and in response to the presence of a configured domain name in the configuration information but the result of the assignment to the first storage node for storage access being a failure, using a polling mechanism to assign the target data stream to the second storage node for storage access.

[0077] In some embodiments, the configuration information further includes a gateway list, and the gateway list includes a number of node addresses. When specifically executing the assignment of the target data stream to the second storage node for storage access using a polling mechanism, the target data stream assignment module 530 includes: polling according to each node address in the gateway list to select the storage node corresponding to the currently polled node address as the second storage node, and assigning the target data stream to the second storage node for storage access.

[0078] In some embodiments, after specifically executing the assignment of the target data stream to the second storage node for storage access using a polling mechanism, the target data stream assignment module 530 further includes: in response to the result of the assignment to the second storage node for storage access being a failure, filtering the node address corresponding to the storage node already assigned in this round and the node address corresponding to the second storage node from the gateway list to obtain at least one remaining node address; polling according to the at least one remaining node address to select the storage node corresponding to the currently polled node address, and assigning the target data stream to the selected storage node for storage access.

[0079] In some embodiments, the configuration information further includes a gateway list, and the gateway list includes a number of node addresses. When specifically executing the polling mechanism to allocate the target data stream to the second storage node for storage access, the target data stream allocation module 530 includes: polling according to each node address in the gateway list to select the storage node corresponding to the currently polled node address as the second storage node, and allocating the target data stream to the second storage node for storage access. After the target data stream allocation module 530 specifically executes the polling mechanism to allocate the target data stream to the second storage node for storage access, it further includes: in response to the result of allocating the storage access to the second storage node being a failure, filtering the node address corresponding to the storage node already allocated in this round and the node address corresponding to the second storage node from the gateway list to obtain at least one remaining node address. Polling according to the at least one remaining node address to select the storage node corresponding to the currently polled node address, and allocating the target data stream to the selected storage node for storage access.

[0080] In some embodiments, the configuration information is cached in the current node, and the storage traffic allocation device 500 further includes a configuration information update module, which is used to update the configuration information cached in the current node with the changed configuration information in response to receiving the changed configuration information.

[0081] In some embodiments, the configuration information includes a traffic allocation type. When specifically executing the update of the configuration information cached in the current node with the changed configuration information in response to receiving the changed configuration information, the configuration information update module includes: in response to receiving the changed traffic allocation type, updating the changed allocation type information to the configuration information cached in the current node.

[0082] In some embodiments, the configuration information includes a domain name field. When specifically executing the update of the configuration information cached in the current node with the changed configuration information in response to receiving the changed configuration information, the configuration information update module includes: in response to receiving the changed domain name field, updating the changed domain name field to the configuration information cached in the current node.

[0083] In some embodiments, the configuration information includes a gateway list. When specifically executing the update of the configuration information cached in the current node with the changed configuration information in response to receiving the changed configuration information, the configuration information update module includes: in response to receiving the changed gateway list, updating the changed gateway list to the configuration information cached in the current node.

[0084] In some embodiments, the configuration information includes a traffic allocation type and a domain name field. When specifically executing in response to receiving the changed configuration information, the configuration information update module updates the configuration information cached by the current node by using the changed configuration information, including: in response to receiving the changed traffic allocation type, updating the changed allocation type information to the configuration information cached by the current node; in response to receiving the changed domain name field, updating the changed domain name field to the configuration information cached by the current node.

[0085] In some embodiments, the configuration information includes a traffic allocation type and a gateway list. When specifically executing in response to receiving the changed configuration information, the configuration information update module updates the configuration information cached by the current node by using the changed configuration information, including: in response to receiving the changed traffic allocation type, updating the changed allocation type information to the configuration information cached by the current node; in response to receiving the changed gateway list, updating the changed gateway list to the configuration information cached by the current node.

[0086] In some embodiments, the configuration information includes a domain name field and a gateway list. When specifically executing in response to receiving the changed configuration information, the configuration information update module updates the configuration information cached by the current node by using the changed configuration information, including: in response to receiving the changed domain name field, updating the changed domain name field to the configuration information cached by the current node; in response to receiving the changed gateway list, updating the changed gateway list to the configuration information cached by the current node.

[0087] In some embodiments, the configuration information includes a traffic allocation type, a domain name field, and a gateway list. When specifically executing in response to receiving the changed configuration information, the configuration information update module updates the configuration information cached by the current node by using the changed configuration information, including: in response to receiving the changed traffic allocation type, updating the changed allocation type information to the configuration information cached by the current node; in response to receiving the changed domain name field, updating the changed domain name field to the configuration information cached by the current node; in response to receiving the changed gateway list, updating the changed gateway list to the configuration information cached by the current node.

[0088] In some embodiments, the storage node includes a backend service and a storage space, and the storage space is used for storing and accessing data streams, and the method is executed by the backend service of the current node.

[0089] In some embodiments, before specifically executing to determine the target allocation policy based on the configuration information of the current node, the target allocation policy determination module 520 further includes reading the configuration information from the Redis cache.

[0090] In some embodiments, the storage node includes a backend service and a storage space. The storage space is used for storing and accessing data streams, and the method is executed by the backend service of the current node. Before specifically executing the target allocation policy determination module 520 to determine the target allocation policy based on the configuration information of the current node, it also includes reading the configuration information from the Redis cache.

[0091] Please refer to Figure 6 , Figure 6 which is a schematic framework diagram of an embodiment of the electronic device of the present application. The electronic device 60 includes a mutually coupled memory 61 and a processor 62. The processor 62 is used to execute program instructions stored in the memory 61 to implement the steps in any of the above-described storage traffic allocation method embodiments. In a specific implementation scenario, the electronic device 60 may include, but is not limited to: a microcomputer, a server. In addition, the electronic device 60 may also include mobile devices such as a laptop computer, a tablet computer, etc., which are not limited herein.

[0092] Specifically, the processor 62 is used to control itself and the memory 61 to implement the steps in any of the above-described storage traffic allocation method embodiments. The processor 62 may also be referred to as a CPU (Central Processing Unit). The processor 62 may be an integrated circuit chip with signal processing capabilities. The processor 62 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Additionally, the processor 62 may be implemented jointly by integrated circuit chips.

[0093] Please refer to Figure 7 , Figure 7 which is a schematic framework diagram of an embodiment of the computer-readable storage medium of the present application. The computer-readable storage medium 70 stores program instructions 71 that can be run by a processor. The program instructions 71 are used to implement the steps in any of the above-described storage traffic allocation method embodiments.

[0094] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0095] The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments. Their similarities or resemblances can be referred to each other. For the sake of brevity, they will not be elaborated herein again.

[0096] In several embodiments provided by the present application, it should be understood that the disclosed methods and apparatuses can be implemented in other ways. For example, the apparatus embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the apparatuses or units can be in electrical, mechanical or other forms.

[0097] In addition, each functional unit in the various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0098] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in the various embodiments of the present application. And the aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs and other various media that can store program codes.

Claims

1. A method for storing traffic allocation, characterized in that, The method is applied to a current node, which is a storage node in a distributed storage system. The method includes: Receiving a storage access request for a target data stream; Determining a target allocation policy based on configuration information, where the target allocation policy is to determine a storage node for storing and accessing the target data stream based on storage traffic overlap; Allocating the target data stream to a storage node in the distributed storage system for storage access according to the target allocation policy.

2. The method according to claim 1, wherein The configuration information includes a traffic allocation type; The determining the target allocation policy based on the configuration information includes: In response to the traffic allocation type being a first allocation type, determining the target allocation policy as a priority traffic non-overlap allocation policy; In response to the traffic allocation type being a second allocation type, determining the target allocation policy as a normal allocation policy.

3. The method according to claim 1 or 2, characterized in that, The allocating the target data stream to a storage node in the distributed storage system for storage access according to the target allocation policy includes at least one of the following steps: In the case where the target allocation policy is a priority traffic non-overlap allocation policy, allocating the target data stream to the current node for storage access, where the current node is a storage node in the distributed storage system; In the case where the target allocation policy is a normal allocation policy, allocating the target data stream to a first storage node corresponding to a target domain name for storage access, or allocating the target data stream to a second storage node using a polling mechanism for storage access.

4. The method according to claim 3, characterized in that The allocating the target data stream to the current node for storage access includes: Invoking the node address of the current node to allocate the target data stream to the current node for storage access; And / or, the configuration information further includes a gateway list, and the gateway list contains several node addresses; after allocating the target data stream to the current node for storage access, it further includes: In response to the result of allocating the target data stream to the current node for storage access being a failure, polling starts from the node address after the node address of the current node in the gateway list to select the storage node corresponding to the currently polled node address, and allocating the target data stream to the selected storage node for storage access.

5. The method according to claim 3, characterized in that, The configuration information further includes a domain name field; the allocating the target data stream to a first storage node corresponding to a target domain name for storage access, or allocating the target data stream to a second storage node using a polling mechanism for storage access includes: In response to the domain name field having a configured domain name, using the configured domain name as the target domain name and allocating the target data stream to a first storage node corresponding to the target domain name for storage access; In response to the domain name field not having a configured domain name, allocating the target data stream to a second storage node using a polling mechanism for storage access; and / or, in response to the configuration information having a configured domain name, but the result of allocating the target data stream to the first storage node for storage access being a failure, allocating the target data stream to a second storage node using a polling mechanism for storage access.

6. The method according to claim 5, wherein The configuration information further includes a gateway list, and the gateway list includes a number of node addresses; The step of using a polling mechanism to allocate the target data stream to a second storage node for storage access includes: Polling according to each node address in the gateway list to select the storage node corresponding to the currently polled node address as the second storage node, and allocating the target data stream to the second storage node for storage access; And / or, after using the polling mechanism to allocate the target data stream to a second storage node for storage access, it further includes: In response to the result of allocating the storage access to the second storage node being a failure, filtering the node addresses corresponding to the storage nodes already allocated in this round and the node address corresponding to the second storage node from the gateway list to obtain at least one remaining node address; Polling according to the at least one remaining node address to select the storage node corresponding to the currently polled node address, and allocating the target data stream to the selected storage node for storage access.

7. The method according to claim 1, wherein The configuration information is cached in the current node, and the current node is a storage node in a distributed storage system. The method further includes: In response to receiving the changed configuration information, updating the cached configuration information with the changed configuration information; And / or, the storage node includes a backend service and a storage space, and the storage space is used for storage access of the data stream. The method is executed by the backend service of the current node, and the current node is a storage node in a distributed storage system; And / or, before determining the target allocation policy based on the configuration information, it further includes: reading the configuration information from a Redis cache.

8. The method according to claim 7, wherein The configuration information includes at least one of a traffic allocation type, a domain name field, and a gateway list; The step of, in response to receiving the changed configuration information, updating the cached configuration information with the changed configuration information includes: In response to receiving the changed traffic allocation type, updating the changed allocation type information to the cached configuration information; And / or, in response to receiving the changed domain name field, updating the changed domain name field to the cached configuration information; And / or, in response to receiving the changed gateway list, updating the changed gateway list to the cached configuration information.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores program instructions, and the processor is configured to execute the program instructions to implement the storage traffic allocation method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program instructions, and the program instructions can be executed to implement the storage traffic allocation method according to any one of claims 1-8.

Images