Distribution method and system, storage medium and program product

By responding to the data request of the requesting node on the allocated node, and dynamically allocating data requests based on the load information and preset policies of the storage node, the problem that storage resources cannot be dynamically adjusted in the prior art is solved, and load balancing and efficient utilization of system resources are realized.

CN120066804AActive Publication Date: 2025-05-30INSPUR SUZHOU INTELLIGENT TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202510545293.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-04-28
Publication Date
2025-05-30
Estimated Expiration
2045-04-28

AI Technical Summary

Technical Problem

In the prior art, when multiple host systems are connected to the same storage device through the PCIe interface, each interface is independently managed and the bandwidth is fixed, and cannot be dynamically adjusted, resulting in the inability to achieve dynamic load balancing of system resources.

Method used

By responding to the data request of the requesting node, the load allocation result of the storage node is determined based on the data request, the load information of the multiple storage nodes and the preset load policy, and the data request is allocated to the target storage node to achieve dynamic load balancing.

Benefits of technology

By dynamically adjusting storage resources, optimizing resource utilization, avoiding resource waste, ensuring load balancing of each node, and improving the overall reliability and availability of the system, especially in high concurrent request scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120066804A_ABST
    Figure CN120066804A_ABST
Patent Text Reader

Abstract

The invention provides a distribution method which can be applied to the technical field of computers. The distribution method comprises the following steps: in response to a data request received from a request node, a distribution node determines load distribution results of a plurality of storage nodes based on the data request, load information of the storage nodes and a preset load strategy associated with the load information; and the distribution node distributes the data request to at least one target storage node in the plurality of storage nodes based on the load distribution result, so that the target storage node sends target information to the request node according to the data request. The invention further provides a storage system, a storage medium and a program product.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to an allocation method, system, storage medium and program product. Background Art

[0002] In scenarios such as high-performance computing, big data analysis, and storage area networks, it is necessary to quickly share and access storage resources among multiple nodes. In the related art, multiple host systems are connected to the same storage device through a specific interface (such as a Peripheral Component Interconnect Express, PCIe interface), and to a certain extent, data interaction among multiple systems is realized. However, in the related art, each interface is managed independently and fixed bandwidth is allocated, making it difficult to dynamically adjust resources according to actual needs, resulting in the inability to achieve dynamic load balancing of system resources. Summary of the Invention

[0003] In view of the above problems, the present invention provides an allocation method, system, device, medium and program product.

[0004] According to a first aspect of the present invention, an allocation method is provided, including: in response to receiving a data request from a request node, an allocation node determines a load allocation result of multiple storage nodes based on the data request, the load information of the multiple storage nodes, and a preset load policy associated with the load information; the allocation node allocates the data request to at least one target storage node among the multiple storage nodes based on the load allocation result, so that the target storage node sends target information to the request node according to the data request.

[0005] According to a second aspect of the present invention, an allocation device is provided, including: a result determination module, configured to, in response to receiving a data request from a request node, an allocation node determines a load allocation result of multiple storage nodes based on the data request, the load information of the multiple storage nodes, and a preset load policy associated with the load information; a request allocation module, configured to the allocation node allocates the data request to at least one target storage node among the multiple storage nodes based on the load allocation result, so that the target storage node sends target information to the request node according to the data request.

[0006] A third aspect of the present invention provides a storage system, including: storage nodes; an allocation node, configured to execute the above allocation method according to the load information determined by the storage nodes.

[0007] A fourth aspect of the present invention provides an electronic device, including: one or more processors; a memory, configured to store one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0008] The fifth aspect of the present invention further provides a computer-readable storage medium, on which a computer program or instruction is stored, and when the computer program or instruction is executed by a processor, the steps of the above method are implemented.

[0009] The sixth aspect of the present invention further provides a computer program product, including a computer program or instruction, and when the computer program or instruction is executed by a processor, the steps of the above method are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Through the following description of the embodiments of the present invention with reference to the accompanying drawings, the above content and other objects, features and advantages of the present invention will become clearer. In the drawings:

[0011] Figure 1 The application scenario diagram of the allocation method, system, device, medium and program product according to the embodiment of the present invention is shown;

[0012] Figure 2 The flowchart of the allocation method according to the embodiment of the present invention is shown;

[0013] Figure 3A The example schematic diagram of the data interaction process between the storage system and the request system according to the embodiment of the present invention is shown;

[0014] Figure 3B The example schematic diagram of the interaction process between the allocation node and the storage node in the storage system according to the embodiment of the present invention is shown;

[0015] Figure 4 The structural block diagram of the allocation device according to the embodiment of the present invention is shown;

[0016] Figure 5 The block diagram of the storage system according to the embodiment of the present invention is shown;

[0017] Figure 6 The block diagram of the electronic device suitable for implementing the allocation method according to the embodiment of the present invention is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary and are not intended to limit the scope of the present invention. In the following detailed description, for the sake of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present invention. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, the descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present invention.

[0019] The terms used herein are for describing specific embodiments only and are not intended to limit the present invention. The terms "comprising", "including" and the like used herein indicate the presence of the described features, steps, operations and / or components, but do not preclude the presence or addition of one or more other features, steps, operations or components.

[0020] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0021] In cases where expressions similar to "at least one of A, B, and C, etc." are used, generally, it should be interpreted according to the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0022] In some examples, by allowing multiple hosts to connect to the same storage device using the PCIe interface, each host has an independent memory space and bandwidth allocation to achieve direct data access and the lowest latency, which improves the data transfer rate and system density to a certain extent.

[0023] However, the bandwidth of each host is fixed and cannot be dynamically adjusted according to the actual IO pressure. This means that even if the input / output (I / O) demand of a certain host suddenly increases, it cannot dynamically obtain additional bandwidth from other hosts. Under high I / O pressure, some hosts may experience performance bottlenecks due to insufficient bandwidth, while other hosts cannot help share the pressure even if they have remaining bandwidth.

[0024] In some examples, each host is connected to the storage device through an independent PCIe interface, achieving data transfer isolation and security to a certain extent. However, each host is connected to the storage device through an independent PCIe interface, and these interfaces are independently managed, and resources cannot be dynamically allocated between different interfaces. Therefore, even if the I / O pressure of a certain host is low, its unused bandwidth cannot be utilized by other hosts, resulting in low overall resource utilization.

[0025] Therefore, in the scenario of multi-host shared storage resources, the I / O demands of different hosts may change over time. Due to the inability to dynamically adjust the bandwidth allocation, the system appears inflexible in dealing with these changes. In scenarios where dynamic resource allocation is required, such as high-concurrency and cloud computing environments, this limitation will lead to performance bottlenecks and resource waste.

[0026] In view of the above technical problems, the present invention provides an allocation method, including: in response to receiving a data request from a requesting node, an allocation node determines a load allocation result of a plurality of storage nodes based on the data request, the load information of the plurality of storage nodes, and a preset load policy associated with the load information; the allocation node allocates the data request to at least one target storage node among the plurality of storage nodes based on the load allocation result, so that the target storage node sends target information to the requesting node according to the data request.

[0027] According to an embodiment of the present invention, by flexibly determining the load allocation results of different storage nodes based on the data requests of different requesting nodes, the load information of the plurality of storage nodes, and the preset load policy, and through reasonable load allocation results, the resource utilization rate is optimized and resource waste is avoided. Since the data requests are dynamically allocated according to the real-time load allocation results of the storage nodes, it adapts to the changing requirements of the node environment and ensures the load balance of each node. Especially in the scenario of high-concurrency requests, through the load balance of each storage node, the overall reliability and availability of the system can be improved.

[0028] Figure 1 The application scenario diagram of the allocation method, system, device, medium, and program product according to an embodiment of the present invention is shown.

[0029] As Figure 1 shown, the application scenario according to this embodiment may include a terminal device 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0030] A user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. For example, the terminal device 101 generates and sends a data request to the server 103 according to a user operation. For another example, the terminal device 101 receives response data from the server 103 and presents it to the user in a relevant form (such as a picture, digital form).

[0031] The terminal device 101 may be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc.

[0032] Server 103 can be a server that provides various services. For example, server 103 receives a data request sent by a user through terminal device 101, parses the content of the data request, and executes corresponding business logic. For another example, server 103 dynamically allocates computing and storage resources according to the type and load of the request. For example, server 103 generates response data and sends it back to terminal device 101 through network 102.

[0033] It should be noted that the allocation method provided by the embodiments of the present invention can generally be executed by server 103. Correspondingly, the allocation device provided by the embodiments of the present invention can generally be set in server 103. The allocation method provided by the embodiments of the present invention can also be executed by a server or a server cluster different from server 103 and capable of communicating with terminal device 101 and / or server 103. Correspondingly, the allocation device provided by the embodiments of the present invention can also be set in a server or a server cluster different from server 103 and capable of communicating with terminal device 101 and / or server 103.

[0034] It should be understood that Figure 1 the numbers of terminal devices, networks, and servers in

[0035] Figure 2 shows a flowchart of the allocation method according to an embodiment of the present invention.

[0036] As Figure 2 shown, the allocation method of this embodiment includes operation S210 to operation S220.

[0037] In operation S210, in response to receiving a data request from a request node, an allocation node determines a load allocation result of multiple storage nodes based on the data request, the load information of multiple storage nodes, and a preset load policy associated with the load information.

[0038] In an embodiment of the present invention, a request node can represent a node that generates and sends a data request according to a user operation. The content and form of the data request can be determined according to the actual input of the user. An allocation node can be a node in a storage system that dynamically allocates resources for the received data request; the allocation node can be configured based on remote direct memory access technology and shared network technology over Ethernet, and is used to simultaneously process data requests from multiple external request nodes, and can also satisfy the distribution of data access requests among multiple storage nodes within the storage system.

[0039] In an embodiment of the present invention, a storage node may be a node in a storage system that stores data and sends response data to a request node in real time according to the load distribution result. The preset load policy may be a variety of processing policies corresponding to load information.

[0040] For example, the load analysis module of the storage node is used to determine the load information of each storage node, and synchronize the obtained load information to the allocation node; the allocation node determines the load distribution results of multiple storage nodes according to the load information, data requests, and corresponding processing policies.

[0041] In operation S220, the allocation node distributes the data request to at least one target storage node among multiple storage nodes based on the load distribution result, so that the target storage node sends target information to the request node according to the data request.

[0042] In an embodiment of the present invention, the target storage node may be the storage node allocated for the data request at the current moment, and there may be multiple target storage nodes, which are based on the criteria of meeting the data request and load balancing. The target information may be the response data sent to the request node according to the data request.

[0043] For example, the allocation node converts the data request according to the load distribution result and the location information of the target storage node, obtains the converted data request that meets the location information conversion condition, and sends the converted data request to the target storage node; the target storage node thus sends the response data to the request node according to the converted data request, realizing the remote direct access of data between different system nodes, and reducing the participation of other intermediate nodes and network latency.

[0044] According to the embodiment of the present invention, by flexibly determining the load distribution results of different storage nodes based on the data requests of different request nodes, the load information of multiple storage nodes, and the preset load policy, and through reasonable load distribution results, the resource utilization rate is optimized, and resource waste is avoided. Since the data requests are dynamically allocated according to the real-time load distribution results of the storage nodes, it adapts to the changing requirements of the node environment, ensuring the load balance of each node. Especially in the scenario of high-concurrency requests, through the load balance of each storage node, the overall reliability and availability of the system can be improved.

[0045] Figure 3A FIG. shows an example schematic diagram of the data interaction process between the storage system and the request system according to an embodiment of the present invention.

[0046] As Figure 3AAs shown, the storage system 30 includes a plurality of storage nodes 301 and a plurality of allocation nodes 302. The plurality of storage nodes 301 may include storage node A 301a, storage node B 301b, storage node C 301c, and storage node D 301d. The plurality of allocation nodes 302 may include allocation node A 302a and allocation node B 302b. There is a redundancy relationship between allocation node A 302a and allocation node B 302b. In the case where one of the nodes fails to store data, the other node can still meet the allocation requirements of the system. The request system 31 may include a plurality of request nodes 311, such as request node A 311a, request node B 311b, and request node C 311c.

[0047] The request system 31 is connected to the storage system 30 through a plurality of allocation nodes. The user sends a data request to the storage system 30 through the request node 311. After receiving the data request from the request node, the allocation node 302 may determine the load allocation results of storage node A 301a, storage node B 301b, storage node C 301c, and storage node D 301d based on the data request, the load information of the storage node 301, and a preset load policy.

[0048] After determining the load allocation results of different storage nodes 301, the allocation node 302 may allocate the data request to target storage nodes (such as storage node C 301c and storage node D 301d) among storage node A 301a, storage node B 301b, storage node C 301c, and storage node D 301d based on the load allocation results, so that the target storage nodes send target information to the request node 311 according to the data request.

[0049] It can be understood that the numbers of the above storage system, storage nodes, allocation nodes, request system, and request nodes are only for illustration and can be determined according to actual requirements.

[0050] According to an embodiment of the present invention, the data request includes the location information of the request node and the initial location information of the storage node. The allocation node allocating the data request to at least one target storage node among a plurality of storage nodes based on the load allocation results includes: determining the target location information of the target storage node based on the load allocation results and the initial location information; using the target location information to update the initial location information to obtain an updated data request and sending the updated data request to the target storage node; and the target storage node sending target information to the request node based on the updated data request and the location information, where the location information includes physical location information and logical location information.

[0051] In an embodiment of the present invention, the physical location information may represent the physical address information (MAC address) of a network device. The logical location information may represent the logical address information (IP address) of a network device, and is used to identify the device in a network. The initial location information may be the location information of multiple storage nodes for an external node, and the target location information may be the location information when multiple storage nodes interact with an allocation node inside the system. The updated data request may be the data request obtained by converting the initial location information into the target location information in the data request.

[0052] It can be understood that the use of the target location information can achieve the interaction between different types of nodes inside the storage system, and meet the settings where the location information of different nodes is different for each other. The use of the initial location information can achieve the interaction between different types of nodes in the storage system and an external node (request node), and meet the setting where multiple internal nodes use the same node location information to be the same for the outside.

[0053] For example, the IP addresses of storage node A 301a, storage node B 301b, storage node C 301c, and storage node D 301d are IP-302a, IP-302b, IP-302c, and IP-302d respectively, and the MAC addresses of the storage nodes are MAC-302a, MAC-302b, MAC-302c, and MAC-302d respectively; the allocation node is allocation node A 302a; the request nodes are request node A 311a, request node B 311b, and request node C 311c respectively, the IP addresses of the request nodes are IP-R-311a, IP-R-311b, and IP-R-311c respectively, and the MAC addresses of the request nodes are MAC-R-311a, MAC-R-311b, and MAC-R-311c respectively.

[0054] In the case of interaction between storage node A 301a, storage node B 301b, storage node C 301c, and storage node D 301d inside the storage system and allocation node A 302a, the allocation node can interact with different storage nodes through IP-302a, IP-302b, IP-302c, and IP-302d.

[0055] When the A storage node 301a, B storage node 301b, C storage node 301c, and D storage node 301d send the target data to the external request nodes in response to the data requests of the external A request node 311a, B request node 311b, and C request node 311c, the IP addresses corresponding to the A storage node 301a, B storage node 301b, C storage node 301c, and D storage node 301d can be unified as IP-301, and the target data is sent to IP-R-311a, IP-R-311b, and IP-R-311c.

[0056] Through the initial position information and the target position information, each storage node in the internal network has its own private IP address, which is convenient for internal communication and management. Multiple private IP addresses in the internal network can share a public IP address for external use. When an internal node needs to communicate with the external network, the allocation node can convert the internal private IP address into the public IP address for external use.

[0057] According to an embodiment of the present invention, the internal network uses private IP addresses, which can avoid IP address conflicts with the external network. At the same time, only one public IP address is exposed externally, reducing the number of IP addresses that need to be managed. The external network only needs to communicate with the internal network through one public IP address without paying attention to the topology of the internal network, realizing simplified routing configuration, reducing the complexity of network management, and reducing the need for physical wiring.

[0058] According to an embodiment of the present invention, the allocation node is configured with a set of position information for the storage nodes. The initial position information includes initial physical position information and initial logical position information; determining the target position information of the target storage node based on the load distribution result and the initial position information includes: determining the target physical position information and the target logical position information in the target position information from the set of position information based on the load distribution result, the initial physical position information, and the initial logical position information.

[0059] In an embodiment of the present invention, the set of position information is a set of physical position information and logical position information of all storage nodes configured in the allocation node, and can be used to send data requests to the corresponding target storage nodes based on the position information in the set subsequently.

[0060] For example, the allocation node is configured with a set of location information of storage node A 301a, storage node B 301b, storage node C 301c, and storage node D 301d. The set of location information may include IP addresses (IP-302a, IP-302b, IP-302c, and IP-302d) and MAC addresses (MAC-302a, MAC-302b, MAC-302c, and MAC-302d). When the load distribution result indicates that the load pressure values of storage node B 301b and storage node C 301c are the smallest at the current moment, storage node B 301b and storage node C 301c are determined as target storage nodes, and the location information of the target storage nodes can be determined from the set of location information as IP -302b, IP-302c, and MAC-302b, MAC-302c; thus, the data request is sent to the location information MAC-302b, MAC-302c and / or IP-302b, IP-302c of the target storage node.

[0061] According to an embodiment of the present invention, updating the initial location information with the target location information to obtain an updated data request and sending the updated data request to the target storage node includes: updating the initial physical location information and the initial logical location information with the target physical location information and the target logical location information respectively to obtain an updated data request recognized by the interface of the allocated node; sending the updated data request to the target storage node using the interface according to the target physical location information and the target logical location information, where the interface is correspondingly set with the request node.

[0062] In an embodiment of the present invention, the interface may be a virtual interface (Virtual Function, VF) configured by the allocation node based on the conversion requirements between different location information. The VF interface may be multiple logical interfaces obtained by virtualizing a physical network interface, and each VF interface can be independently allocated to a corresponding node.

[0063] In an embodiment of the present invention, the updated data request may be obtained by converting the physical address and the logical address corresponding to the data request. In the network layer 2, the allocation node may convert the MAC / IP layer address and convert the MAC / IP header of the new request into the MAC / IP header to be forwarded, thereby realizing the efficient forwarding of data. The MAC address is the physical address in the layer 2 network and can identify the network interface card. In the layer 2 network, the data frame can be forwarded through the MAC address. The IP address is the logical address in the layer 3 network and can identify the host or device in the network.

[0064] For example, through the Address Resolution Protocol, a distribution node can map an IP address to a MAC address in a layer-2 network so that data frames can be correctly forwarded in the layer-2 network. When a new request arrives at the distribution node, the data frame can contain information such as the source MAC address, destination MAC address, source IP address, and destination IP address. The distribution node can modify the header information of the data frame according to the MAC / IP address of the target node. In the layer-2 network, the distribution node can forward the data frame to the target storage node according to the destination MAC address.

[0065] In an embodiment of the present invention, considering that the distribution node needs to interact with nodes (such as storage nodes) within the internal storage system and external nodes (such as request nodes) at the same time, it is necessary to perform conversion between physical location information and logical location information. Thus, the distribution node can implement the conversion between physical location information and logical location information by configuring corresponding VFs. The number of VF interfaces can be in one-to-one correspondence with the number of request nodes. For example, the A distribution node 302a is configured with a corresponding VF interface, and the VF interface is configured with a corresponding MAC address, IP address, and queue.

[0066] For example, using hardware virtualization technology on a physical network interface to create multiple virtual interfaces, configuring corresponding resources (such as the number of queues, interrupts, etc.) for each virtual interface; allocating the virtual interfaces to corresponding nodes so that each node can independently access the virtual interface allocated to it; independent MAC addresses and IP addresses can be configured for each virtual interface to make these addresses unique in the network. Further, virtual local area network tags can be used to isolate the traffic of different nodes, and service quality policies can be configured to ensure the communication priority of critical nodes.

[0067] In the related art, data interaction is performed by multiple nodes sharing the same physical interface. When multiple nodes send data simultaneously, it will cause insufficient bandwidth. Especially for packets of multiple nodes that may arrive at the physical interface at the same time, conflicts and retransmissions will occur. The contention for shared resources will lead to increased latency and decreased throughput.

[0068] According to an embodiment of the present invention, by configuring an independent virtual interface for each allocation node, and each virtual interface is configured with corresponding transmission and reception queues, different data requests can be prevented from interfering with each other. At the same time, each virtual interface is configured with its own MAC / IP address to avoid address conflicts, and the traffic between different virtual interfaces is isolated, so that the communication of each node will not be affected by other nodes. Therefore, by configuring a corresponding virtual interface for each allocation node, each node can have independent communication resources, avoiding data conflicts. Especially in high-load scenarios, the throughput and reliability of the network can be improved.

[0069] According to an embodiment of the present invention, the method further includes any one of the following: determining load information based on at least one of the waiting duration, queue length, request duration, and device status of multiple storage nodes at a previous moment; determining load information based on at least one of the historical load information of the storage node at a previous moment and the fluctuation characteristics, trend characteristics, and periodic characteristics of the historical load information.

[0070] In an embodiment of the present invention, the load information can be determined according to different performance metric values of the storage node at the current moment. The performance metric values can include at least one of the waiting duration, queue length, request duration, and device status. The waiting duration can represent the duration for which a request waits in the queue for processing; the queue length can be determined according to the number of requests currently waiting for processing; the request duration can represent the duration required to process a request; the device status can be the load pressure status of the device (such as processor usage rate, memory usage rate, disk input / output status).

[0071] For example, when the waiting duration, queue length, request duration, and device status of each storage node at the current moment are obtained, a comprehensive metric result of each storage node is obtained by weighted averaging according to the waiting duration, queue length, request duration, device status, and the weights of each performance metric value, and thus the load information of the storage node is determined according to the comprehensive metric result.

[0072] In an embodiment of the present invention, the fluctuation characteristics can represent the change amplitude and frequency of the load information. The trend characteristics can represent the long-term change direction of the load information. The periodic characteristics can represent the periodic change pattern of the load information.

[0073] For example, the fluctuation characteristics, trend characteristics, and periodic characteristics of the load information are extracted from the obtained historical load information, and the above characteristics are analyzed; according to the influence degree of each characteristic on the load, weights are assigned to each characteristic, and the comprehensive load characteristics are calculated to obtain the load information, or the above-extracted historical fluctuation characteristics are input into a trained load prediction model to predict the load information of the current storage node.

[0074] According to an embodiment of the present invention, by analyzing the load information of storage nodes and the fluctuation characteristics, trend characteristics, and periodic characteristics of the load information, the load conditions of the nodes can be comprehensively understood. Thus, combined with the load balancing strategy, the performance and resource utilization rate of the storage system can be optimized. Especially for distributed storage systems and cloud storage services, the efficient operation of the system and the improvement of user experience can be achieved.

[0075] According to an embodiment of the present invention, the preset load strategy includes the storage node weights that characterize the loads of the storage nodes; the allocation node determines the load allocation results of multiple storage nodes based on the data request, the load information of multiple storage nodes, and the preset load strategy associated with the load information, including: determining the load allocation results based on the resource occupancy information, load information, and storage node weights corresponding to the data request.

[0076] In an embodiment of the present invention, the storage node weights can assign weights or priorities to each storage node according to the preset load strategy. The resource occupancy information can be obtained by evaluating the consumption amounts of various resources associated with the data request.

[0077] For example, high water marks and low water marks can be set for different load metrics according to the actual application scenario and performance requirements. For example, it is set that when the processor utilization rate is higher than 80% is high load (high water mark), and lower than 30% is low load (low water mark). An initial weight value is initially assigned to each storage node. During the operation of the system, the load metrics of each node are monitored at regular time intervals (such as every second or every few seconds). For high-load nodes, if the load metric of the node exceeds the high water mark, its weight is multiplied by 0.5 to reduce its weight and reduce the probability of subsequent new input / output requests being allocated. For low-load nodes, if the load metric of the node is lower than the low water mark, its weight is multiplied by 1.2 to increase its weight, making it more likely to be preferentially selected in subsequent request allocations.

[0078] For example, the resource occupancy amount is obtained by calculating the amount of resources required for the data request (such as the input / output of the processor, memory, and disk); thus, based on the resource occupancy amount, load information, and weights of the storage nodes, the load allocation results of each storage node are obtained through a weighted average model or a machine learning model.

[0079] According to an embodiment of the present invention, the method further includes: updating the storage node weights based on the current load information of the storage nodes at the current moment to obtain updated node weights; and updating the load allocation results using the updated node weights to obtain updated allocation results.

[0080] In an embodiment of the present invention, updating the node weight may be determined by dynamically updating the storage node weight according to the actual situation and the load information of the storage node at the current moment. Considering that the performance of the storage node may change over time (such as hardware aging, temporary performance degradation, etc.), dynamic weight adjustment can monitor the performance change in real time and dynamically adjust the weight to ensure the optimal allocation of system performance.

[0081] For example, the weight of storage node A 301a is w1, and the load allocation result is L r , after obtaining the current load information L of S-302a n , compare the current load information L of storage node A 301a n with the previous load information L n-1 , in the case of L n > L n-1 , reduce the weight value w1 of storage node A 301a, update w1 to w’1, and then use w’1 to update L r to obtain the updated allocation result L’ r .

[0082] According to an embodiment of the present invention, through the dynamic adjustment of the weight of the storage node, resources can be allocated to the node with the smallest load pressure value, avoiding resource waste. Dynamic weight adjustment can prevent node overload, ensure that the load of each node is within its processing capacity, and thus improve the overall resource utilization rate.

[0083] According to an embodiment of the present invention, the load information includes the number of connections between the storage node and the request node at the previous moment, and the preset load policy further includes a connection load policy based on the connection number information; the allocation node determines the load allocation results of multiple storage nodes based on the data request, the load information of multiple storage nodes, and the preset load policy associated with the load information, including: determining the load allocation result based on the data request, the connection number, and the connection load policy.

[0084] In an embodiment of the present invention, the connection load policy may be a policy for allocating data requests according to the number of connections between the storage node and the request node, and the connection number may be inversely proportional to the storage node weight.

[0085] For example, use a network monitoring tool to monitor the number of connections between each storage node and the request node in real time, and determine the load allocation result of each storage node according to the connection number, the resource occupancy information corresponding to the data request, and the least connection number policy.

[0086] It can be understood that how to determine the load allocation result according to the connection load policy has been described above, and below will describe how to update the load allocation result.

[0087] According to an embodiment of the present invention, the method further includes: updating the load distribution result based on the current connection number between the storage node and the request node at the current moment to obtain an updated distribution result.

[0088] In an embodiment of the present invention, the connection number of the storage node at the current moment is determined by the connection number regularly sent by the storage node to the distribution node. When it is detected that the difference between the connection number of the storage node at the current moment and the previous number exceeds the difference threshold, the current weight of the storage node is adjusted in real time, and the distribution result of the storage node is updated according to the current weight.

[0089] For example, at the previous moment (the t n-1 th moment), the connection number between the storage node 301a of A and the request node is m + 7, and the weight is w n-1 . When it is detected that the connection number between the storage node 301a of A and the request node at the current moment (the t n th moment) decreases to m - 7, and when the difference 14 between the connection number of the storage node at the current moment and the previous number exceeds the difference threshold (for example, z = 10), the weight value of the storage node at the current moment is updated from w n-1 to w n ; and then the load distribution result is updated according to w n to obtain an updated load distribution result.

[0090] It can be understood that the method of updating the distribution result based on the change of the connection number of multiple storage nodes is similar to the method of updating the distribution result using the connection number of a single storage node, and will not be elaborated here.

[0091] According to an embodiment of the present invention, the distribution node includes an information buffer; the method further includes: when the processing volume of the data request is greater than or equal to the processing threshold of the storage node, storing the data request using the information buffer; reading the data requests from the information buffer in sequence based on the priority of the data requests and allocating them to the target storage node, where the priority is determined based on the data type and data source of the data request.

[0092] In an embodiment of the present invention, the information buffer may be a memory buffer or a disk buffer in the distribution node corresponding to the burst data traffic from multiple request nodes. The priority of the data request can be determined according to any one or more of the request type, data source, request urgency, request time, and request frequency.

[0093] Considering the scenarios of high concurrency and burst traffic in data centers, cloud computing, and high-performance computing environments, in order to avoid data request loss, the burst traffic can be cached through the information buffer.

[0094] For example, a buffer queue is configured on the distribution node for temporarily storing excess requests. The size of the buffer is set according to the processing capacity and available resources of the storage node; the buffer management system is started to monitor and manage the requests in the buffer; when the request volume exceeds the processing capacity of the node, the excess requests are temporarily cached in the buffer. The cached requests can be sorted according to the priority of the requests, so that the requests with higher priority are processed first.

[0095] According to an embodiment of the present invention, through the information buffer, burst traffic can be effectively processed, request loss can be avoided, requests with higher priority are processed first, and the response speed of critical services is improved. By gradually distributing requests, the storage system can still operate stably under high load, improving resource utilization while reducing system resource waste.

[0096] According to an embodiment of the present invention, the information buffer includes a first buffer and a second buffer; storing data requests by using the information buffer includes: based on the available storage space of the first buffer and the second buffer respectively, storing the data requests into the first buffer or the second buffer according to the data volume of the data requests.

[0097] In an embodiment of the present invention, the first buffer and the second buffer may be two parallel buffers in the information buffer, and the cache breakdown and data loss of the information buffer can be avoided by alternately using the two buffers.

[0098] Taking the information buffer including buffer B1 and buffer B2 as an example. Storing the data requests into the first buffer or the second buffer based on the data volume of the data requests may include: calculating the current available storage space of buffer B1 and buffer B2; for each data request, comparing the data volume of the data request with the available space of buffer B1 and buffer B2, and determining to allocate the data request to the corresponding buffer according to the comparison result.

[0099] For example, if the data volume of the data request is less than or equal to the available space of buffer B1, and the available space of buffer B1 is greater than or equal to the available space of buffer B2, the data request can be stored in B1. If the data volume of the data request is less than or equal to the available space of buffer B2, and the available space of buffer B2 is greater than the available space of buffer B1, the data request is stored in buffer B2. If the available space of both buffers is less than the data volume, the request can be discarded or wait for the buffer to release space.

[0100] According to an embodiment of the present invention, by comparing the available space and data volume of different buffer areas, it is ensured that data requests are stored in the most suitable buffer area, avoiding premature filling of a single buffer area, improving the overall buffer utilization rate, enabling the storage system to still operate stably under high load conditions, and improving the stability and efficiency of the system.

[0101] According to an embodiment of the present invention, the method further includes: synchronizing the load information to the allocation node by using synchronization tools corresponding to the storage node and the allocation node respectively.

[0102] In an embodiment of the present invention, the synchronization tool can be a hardware interface, such as a General Purpose Input / Output (GPIO) interface. Based on different synchronization objects, the synchronization tool can include a first synchronization tool corresponding to the storage node and a second synchronization tool corresponding to the allocation node.

[0103] For example, after using the load analysis module of the storage node to determine the load information of the storage node, the first synchronization tool can be used to synchronize the load information to the storage control module of the storage node, and then the storage control module uses the second synchronization tool corresponding to the allocation tool to synchronize the load information to the allocation control module of the allocation node.

[0104] Figure 3B An example schematic diagram showing the interaction process between the allocation node and the storage node in the storage system according to an embodiment of the present invention is shown.

[0105] As Figure 3B shown, the storage system 30 includes a storage node 301 and an allocation node 302. The A storage node 301a, B storage node 301b, C storage node 301c, D storage node 301d and the A allocation node 302a, B allocation node 302b are interconnected by using their respective corresponding control modules, and the control module can be a programmable digital integrated circuit.

[0106] The A storage node 301a, B storage node 301b, C storage node 301c, D storage node 301d each include a load analysis module, and their respective corresponding load analysis modules are the load analysis module 3011a, load analysis module 3011b, load analysis module 3011c and load analysis module 3011d respectively. Multiple storage nodes monitor the input / output pressure of the storage node in real time through their respective load analysis modules to determine their respective load information; after determining the load information, the load analysis module synchronizes the load information to the corresponding storage control module (storage control module 3012a, storage control module 3012b, storage control module 3012c and storage control module 3012d) through the synchronization tool.

[0107] The storage control modules of multiple storage nodes respectively use the synchronization tools of allocation node 302a and allocation node 302b to synchronize the load information to the allocation control modules of each allocation node (allocation control module 3021a and allocation control module 3021b); the allocation control module 3021a and the allocation control module 3021b calculate and analyze the load pressure of each storage node to obtain the load pressure results of each node. The forwarding modules (forwarding module 3022a and forwarding module 3022b) in the allocation nodes send the current data request to one or more nodes with a smaller load pressure result according to the load pressure results, so as to balance the load of each storage node.

[0108] In a feasible embodiment, the storage system constructs a load balancing resource pool using multiple storage nodes. When an abnormality occurs in a certain storage node among the multiple storage nodes, the abnormal node is deleted from the load balancing pool and the uncompleted data requests are migrated to healthy nodes. When the abnormal node returns to normal, the node is re-added to the load balancing pool.

[0109] Based on the above allocation method, the present invention also provides an allocation device. The following will be combined with Figure 4 to describe this device in detail.

[0110] Figure 4 shows a structural block diagram of the allocation device according to an embodiment of the present invention.

[0111] As Figure 4 shown, the allocation device of this embodiment includes a result determination module 410 and a request allocation module 420.

[0112] The result determination module 410 is configured to, in response to receiving a data request from a request node, the allocation node determines a load allocation result of multiple storage nodes based on the data request, the load information of the multiple storage nodes, and a preset load policy associated with the load information. In one embodiment, the result determination module 410 may be used to perform the operation S210 described above, which will not be elaborated here.

[0113] The request allocation module 420 is configured to, based on the load allocation result, the allocation node allocates the data request to at least one target storage node among the multiple storage nodes, so that the target storage node sends target information to the request node according to the data request. In one embodiment, the request allocation module 420 may be used to perform the operation S220 described above, which will not be elaborated here.

[0114] According to an embodiment of the present invention, through the result determination module 410 and the request allocation module 420 in the allocation device, by flexibly determining the load allocation results of different storage nodes based on the data requests of different request nodes, the load information of multiple storage nodes, and a preset load policy, through reasonable load allocation results, the resource utilization rate is optimized, and resource waste is avoided. Since the data requests are dynamically allocated according to the real-time load allocation results of the storage nodes, it adapts to the continuously changing requirements of the node environment, ensuring the load balance of each node. In a high-concurrency request scenario, through the load balance of each storage node, the overall reliability and availability of the system are improved.

[0115] According to an embodiment of the present invention, the data request includes the location information of the request node and the initial location information of the storage node; the request allocation module 420 includes: an information determination sub-module, a request sending sub-module, and an information sending sub-module. The information determination sub-module is used to determine the target location information of the target storage node based on the load allocation result and the initial location information; the request sending sub-module is used to update the initial location information with the target location information to obtain an updated data request and send the updated data request to the target storage node; the information sending sub-module is used for the target storage node to send target information to the request node based on the updated data request and the location information, where the location information includes physical location information and logical location information.

[0116] According to an embodiment of the present invention, the allocation node is configured with a location information set for the storage node, and the initial location information includes initial physical location information and initial logical location information; the information determination sub-module includes: an information determination unit, which is used to determine the target physical location information and the target logical location information in the target location information from the location information set based on the load allocation result, the initial physical location information, and the initial logical location information.

[0117] According to an embodiment of the present invention, the request sending sub-module includes: an information update unit and a request sending unit. The information update unit is used to update the initial physical location information and the initial logical location information with the target physical location information and the target logical location information respectively to obtain an updated data request recognized by the interface of the node to be allocated; the request sending unit is used to send the updated data request to the target storage node through the interface according to the target physical location information and the target logical location information, where the interface is correspondingly set with the request node.

[0118] According to an embodiment of the present invention, the apparatus further includes any one of the following: a first load information determination module and a second load information determination module. The first load information determination module is configured to determine load information based on at least one of the waiting duration, queue length, request duration, and device status of multiple storage nodes at a previous moment; the second load information determination module is configured to determine load information based on the historical load information of the storage node at a previous moment and at least one of the fluctuation characteristics, trend characteristics, and periodic characteristics of the historical load information.

[0119] According to an embodiment of the present invention, the preset load policy includes a storage node weight that characterizes the load of the storage node; the result determination module 410 includes: an allocation result determination sub-module, configured to determine a load allocation result based on the resource occupancy information, load information, and storage node weight corresponding to the data request.

[0120] According to an embodiment of the present invention, the apparatus further includes: a mapping relationship update module and an allocation result update module. The mapping relationship update module is configured to update the storage node weight based on the current load information of the storage node at the current moment to obtain an updated node weight; the allocation result update module is configured to update the load allocation result using the updated node weight to obtain an updated allocation result.

[0121] According to an embodiment of the present invention, the load information includes the number of connections between the storage node and the request node at a previous moment, and the preset load policy further includes a connection load policy based on the connection number information; the result determination module 410 includes: a load allocation result determination sub-module, configured to determine a load allocation result based on the data request, the connection number information, and the connection load policy.

[0122] According to an embodiment of the present invention, the apparatus further includes: a load allocation result update sub-module, configured to update the load allocation result based on the current number of connections between the storage node and the request node at the current moment to obtain an updated allocation result.

[0123] According to an embodiment of the present invention, the allocation node includes an information buffer; the apparatus further includes: a request storage module and a node allocation module. The request storage module is configured to store the data request using the information buffer when the processing volume of the data request is greater than or equal to the processing threshold of the storage node; the node allocation module is configured to sequentially read the data request from the information buffer based on the priority of the data request and allocate it to the target storage node, where the priority is determined based on the data type and data source of the data request.

[0124] According to an embodiment of the present invention, the information buffer includes a first buffer and a second buffer; the request storage module includes: a request storage sub-module, configured to store the data request in the first buffer or the second buffer based on the data volume of the data request according to the available storage space of the first buffer and the second buffer respectively.

[0125] According to an embodiment of the present invention, the apparatus further includes a synchronization module, configured to synchronize the load information to the allocation node by using synchronization tools respectively corresponding to the storage node and the allocation node.

[0126] According to an embodiment of the present invention, any plurality of modules among the result determination module 410 and the request allocation module 420 may be combined and implemented in one module, or any one of them may be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the result determination module 410 and the request allocation module 420 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable manner that can integrate or package circuits, etc., implemented by hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, at least one of the result determination module 410 and the request allocation module 420 may be at least partially implemented as a computer program module, which can execute corresponding functions when the computer program module is run.

[0127] Figure 5 A block diagram of a storage system according to an embodiment of the present invention is shown.

[0128] As Figure 5 shown, a storage system according to an embodiment of the present invention includes a storage node 301 and an allocation node 302. The allocation node 302 is configured to execute the above allocation method according to the load information determined by the storage node 301.

[0129] Figure 6 A block diagram of an electronic device suitable for implementing the allocation method according to an embodiment of the present invention is shown.

[0130] As Figure 6As shown, an electronic device according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage section 608 into a random access memory (RAM) 603. The processor 601 can include, for example, a general microprocessor (such as a CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (such as an application-specific integrated circuit (ASIC)), etc. The processor 601 can also include on-board memory for caching purposes. The processor 601 can include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0131] In the RAM 603, various programs and data required for the operation of the electronic device are stored. The processor 601, ROM 602, and RAM 603 are connected to each other via a bus 604. The processor 601 performs various operations of the method flow according to an embodiment of the present invention by executing the programs in the ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than the ROM 602 and RAM 603. The processor 601 can also perform various operations of the method flow according to an embodiment of the present invention by executing the programs stored in the one or more memories.

[0132] According to an embodiment of the present invention, the electronic device may further include an input / output (I / O) interface 605, and the input / output (I / O) interface 605 is also connected to the bus 604. The electronic device may further include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read from it can be installed into the storage section 608 as needed.

[0133] The present invention also provides a computer-readable storage medium, which can be included in the device / device / system described in the above embodiments; or can exist separately without being assembled into the device / device / system. The above computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to an embodiment of the present invention is implemented.

[0134] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the above-described ROM 602 and / or RAM 603 and / or one or more memories other than ROM 602 and RAM 603.

[0135] An embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product runs in a computer system, the program code is used to enable the computer system to implement the allocation method provided by the embodiment of the present invention.

[0136] When the computer program is executed by the processor 601, it executes the above functions defined in the system / apparatus of the embodiment of the present invention. According to an embodiment of the present invention, the above-described systems, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0137] In one embodiment, the computer program can rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program can also be transmitted and distributed in the form of a signal on a network medium, and is downloaded and installed through the communication part 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0138] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, it executes the above functions defined in the system of the embodiment of the present invention. According to an embodiment of the present invention, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0139] According to embodiments of the present invention, program code for executing the computer programs provided by the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, such as Java, C++, Python, the "C" language, or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).

[0140] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0141] Those skilled in the art can understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0142] The above describes the embodiments of the present invention. However, these embodiments are only for illustrative purposes and not for limiting the scope of the present invention. Although the embodiments are described separately above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of the present invention, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present invention.

Claims

1. A distribution method, characterized in that: The method comprises: In response to receiving a data request from a requesting node, the allocating node determines a load allocation result of the plurality of storage nodes based on the data request, load information of the plurality of storage nodes, and a preset load strategy associated with the load information; The distribution node distributes the data request to at least one target storage node among the plurality of storage nodes based on the load distribution result, so that the target storage node sends target information to the requesting node according to the data request.

2. The method according to claim 1, characterized in that The data request includes the location information of the requesting node and the initial location information of the storage node; The allocating node allocates the data request to at least one target storage node among the plurality of storage nodes based on the load distribution result, including: Determine the target location information of the target storage node based on the load distribution result and the initial location information; Using the target location information to update the initial location information to obtain an update data request and sending the update data request to the target storage node; The target storage node sends the target information to the requesting node based on the update data request and the location information, wherein the location information includes physical location information and logical location information.

3. The method according to claim 2, characterized in that The allocation node is configured with a location information set for the storage node, the initial location information comprising initial physical location information and initial logical location information; Determining the target location information of the target storage node based on the load distribution result and the initial location information includes: Based on the load distribution result, the initial physical location information and the initial logical location information, target physical location information and target logical location information in the target location information are determined from the location information set.

4. The method according to claim 3, characterized in that Using the target location information to update the initial location information to obtain an update data request and sending the update data request to the target storage node includes: Using the target physical location information and the target logical location information to update the initial physical location information and the initial logical location information respectively, to obtain an update data request that satisfies the interface identification of the allocation node; The update data request is sent to the target storage node using the interface according to the target physical location information and the target logical location information, wherein the interface is set corresponding to the request node.

5. The method according to claim 1, characterized in that The method further includes any of the following: Determine the load information based on at least one of the waiting time, queue length, request time, and device status of the plurality of storage nodes at a previous moment; The load information is determined based on historical load information of the storage node at a previous moment and at least one of a fluctuation feature, a trend feature, and a period feature of the historical load information.

6. The method according to claim 1, characterized in that The preset load strategy includes a storage node weight that characterizes the load of the storage node; The allocation node determines the load allocation result of the plurality of storage nodes based on the data request, the load information of the plurality of storage nodes and a preset load strategy associated with the load information, including: The load distribution result is determined based on the resource occupancy information corresponding to the data request, the load information and the storage node weight.

7. The method according to claim 6, characterized in that The method further comprises: Based on the current load information of the storage node at the current moment, the storage node weight is updated to obtain an updated node weight; The load distribution result is updated using the updated node weight to obtain an updated distribution result.

8. The method according to claim 1, characterized in that: The load information includes the number of connections between the storage node and the requesting node at a previous moment, and the preset load strategy also includes a connection load strategy based on the number of connections; The allocation node determines the load allocation result of the plurality of storage nodes based on the data request, the load information of the plurality of storage nodes and a preset load strategy associated with the load information, including: The load distribution result is determined based on the data request, the number of connections, and the connection load strategy.

9. The method according to claim 8, characterized in that The method further comprises: The load distribution result is updated based on the current number of connections between the storage node and the request node at the current moment to obtain an updated distribution result.

10. The method according to any one of claims 1 to 9, characterized in that The distribution node includes an information buffer area; the method also includes: When the processing amount of the data request is greater than or equal to the processing threshold of the storage node, using the information cache area to store the data request; The data requests are read from the information cache area in sequence based on the priority of the data request and distributed to the target storage node, wherein the priority is determined based on the data type and data source of the data request.

11. The method according to claim 10, characterized in that The information buffer area includes a first buffer area and a second buffer area; Utilizing the information buffer area to store the data request includes: According to the available storage space of the first cache area and the second cache area respectively, the data request is stored in the first cache area or the second cache area based on the data amount of the data request.

12. The method according to claim 1, characterized in that The method further comprises: The load information is synchronized to the distribution node using synchronization tools corresponding to the storage node and the distribution node respectively.

13. A storage system, characterized in that: include: Storage nodes; The allocation node is configured to execute the method according to one of claims 1 to 12 according to the load information determined by the storage node.

14. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

15. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Citation Information

Patent Citations

  • Data processing method and device and readable storage medium

    CN116471586A

  • Load balancing method and device for storage cluster, equipment and medium

    CN117675822A

  • Load balancing method and device based on computing cluster, equipment and storage medium

    CN118897730A

  • Request allocation method, system and device and storage medium

    CN119718619A

  • Request processing method and apparatus

    WO2024174821A1