Network load balancing method and device, equipment and storage medium
By determining the target transmission path based on the five-tuple of data packets in the network load balancing system and performing data transformation at intermediate nodes, the load imbalance problem caused by the ECMP hash algorithm is solved, achieving more efficient resource utilization and improved service quality.
Patent Information
- Application Number
- CN202310596994.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-24
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-05-24
AI Technical Summary
In existing technologies, load balancing methods based on the ECMP hash algorithm are prone to uneven distribution of data streams across multiple paths when there are few data streams, resulting in hash polarization, which leads to load imbalance and low resource utilization, affecting service quality.
The target transmission path is determined by the five-tuple of the data packet, and data transformation is performed at intermediate nodes. The target sub-path index is determined by using a preset mapping record to ensure that the data packet is accurately transmitted along the target transmission path, thereby achieving load balancing.
It effectively prevents uneven distribution of data streams across multiple paths and hash polarization problems, thereby improving network resource utilization and service quality.
Smart Images

Figure CN119030928A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of network technology, and in particular to a network load balancing method, device, equipment and storage medium. Background Art
[0002] Load balancing means that network devices balance and distribute the load (work tasks) to multiple operation units for operation. For example, the operation units can be lower-level network devices such as servers that support File Transfer Protocol (FTP servers), Web servers, switches, enterprise core application servers, and other major task servers, so as to collaboratively complete work tasks.
[0003] In the related art, the method for load balancing for a virtual private cloud network (VPN) is to set a random equal cost routing (ECMP) hash algorithm in the network device of the VPN network, and the network device uses the ECMP hash algorithm to randomly and evenly distribute each data stream to multiple paths to send the data stream to the lower network device, where the path is the data transmission path between the network device and the lower network device. In this way, based on the randomness of the ECMP hash algorithm, the network device can achieve load balancing of each data stream in multiple paths, effectively improving service throughput, network availability, and data transmission efficiency. However, this method is very likely to cause uneven distribution of data streams in multiple paths when there are fewer data streams, and this method has a hash polarization problem, and load imbalance will also occur.
[0004] For example, when there is less data flow, if the network device has four paths, namely 1, 2, 3, and 4, the network device may allocate all data flows to path 1, causing path 1 to be overloaded, while the transmission resources in paths 2, 3, and 4 are idle, which in turn leads to a decrease in the quality of the services provided and low resource utilization. Figure 1As shown, in hash polarization, if the first-level network device uses the ECMP hash algorithm to determine the path for data streams 1-6, and sends data streams 1, 2, and 3 to the second-level network device 1, and sends data streams 4, 5, and 6 to the second-level network device 2, the second-level network device 1 uses the same ECMP hash algorithm to determine the path for data streams 1-3, and sends data streams 1, 2, and 3 to the third-level network device 1, and the second-level network device 2 uses the same ECMP hash algorithm to determine the path for data streams 4-6, and sends data streams 4, 5, and 6 to the third-level network device 2. It can be seen that in the lower-level network devices, the load balancing method based on the ECMP hash algorithm has failed, and the links from the second-level network device 1 to the third-level network device 2, and from the second network device 2 to the third-level network device 1 are idle, which will also lead to a decrease in service quality and low resource utilization.
[0005] Therefore, there is an urgent need to redesign a network load balancing method and overcome the above-mentioned defects. Summary of the invention
[0006] The embodiments of the present application provide a network load balancing method, apparatus, device and storage medium for performing network load balancing, improving service quality and network resource utilization.
[0007] In a first aspect, an embodiment of the present application provides a network load balancing method, applicable to a load balancing system, the method comprising:
[0008] Based on the five-tuple in the data packet generated by the source node, determine the target transmission path of the data packet and its target path identifier, wherein the target transmission path connects the source node and the target node respectively, includes at least one intermediate node, and is formed by connecting multiple target sub-paths in series;
[0009] Based on the target transmission path, the data packet is sent to the target node through the at least one intermediate node; wherein, during the transmission process, each of the intermediate nodes performs the following operations:
[0010] Based on the target data conversion result of the target path identifier, in combination with a first mapping record preset corresponding to an intermediate node, determine a target sub-path index associated with the target data conversion result, wherein the first mapping record includes: at least one data conversion result associated with each sub-path index, and each sub-path is a path between the intermediate node and a corresponding next-hop node;
[0011] Based on the target sub-path corresponding to the target sub-path index, the data packet is sent to a next hop node.
[0012] In a second aspect, an embodiment of the present application provides a network load balancing device, applicable to a load balancing system, the device comprising:
[0013] A first processing unit is configured to determine, based on a quintuple in a data packet generated by a source node, a target transmission path of the data packet and a target path identifier thereof, wherein the target transmission path connects the source node and the target node respectively, includes at least one intermediate node, and is formed by connecting a plurality of target sub-paths in series;
[0014] The second processing module is configured to send the data packet to the target node through the at least one intermediate node based on the target transmission path; wherein, during the transmission process, each of the intermediate nodes performs the following operations:
[0015] A path selection unit, configured to determine a target sub-path index associated with the target data conversion result based on the target data conversion result identified by the target path, in combination with a first mapping record preset corresponding to an intermediate node, wherein the first mapping record includes: at least one data conversion result associated with each sub-path index, and each sub-path is a path between the intermediate node and a corresponding next-hop node;
[0016] The transceiver unit is configured to send the data packet to a next hop node based on the target subpath corresponding to the target subpath index.
[0017] Optionally, the network topology structure includes the one intermediate node and other nodes, the other nodes are nodes with the same network distance from the one intermediate node to the source node, and a preset conversion method is set in the one intermediate node and the other nodes;
[0018] The path selection unit is also used for:
[0019] Based on the preset conversion method, data conversion is performed on each path identifier in the source node to obtain a data conversion result of each path identifier;
[0020] Evenly distribute the data conversion results to the sub-path indexes and other sub-path indexes, where other sub-paths are paths between other nodes and corresponding next-hop nodes;
[0021] storing an association relationship between each subpath index and at least one assigned data conversion result, and an association relationship between each other subpath index and at least one assigned data conversion result, in the first mapping record;
[0022] The first mapping record is set in the one intermediate node and the other nodes.
[0023] Optionally, also include:
[0024] A third processing unit is configured to determine, based on the target node, a target data operation result of the five-tuple in the data packet by using a thread selection unit, and determine a target thread index associated with the target data operation result in combination with a second mapping record preset in the target node, wherein the second mapping record includes: at least one data operation result associated with each thread index;
[0025] The transceiver unit is further configured to send the data packet to a target processing thread corresponding to the target thread index, so that the target processing thread processes the data packet.
[0026] Optionally, the network topology structure includes the target node and the receiving node, the receiving node is a node with the same network distance from the target node to the source node, and a preset operation method is set in the target node and the receiving node;
[0027] The third processing unit is further configured to:
[0028] Based on the preset calculation method, data conversion is performed on each quintuple existing in the network topology structure to obtain data calculation results of each quintuple;
[0029] Evenly distribute the data operation results to the thread indexes and receiving thread indexes, where the receiving thread is a processing thread in a receiving node;
[0030] storing the association relationship between each thread index and at least one allocated data operation result, and the association relationship between each receiving thread index and at least one allocated data operation result, in the second mapping record;
[0031] The second mapping record is set in the target node and the receiving node.
[0032] Optionally, the first processing unit is specifically configured to:
[0033] Based on the quintuple in the data packet generated by the source node, a preset transformation method is adopted to obtain the target data transformation result corresponding to the quintuple, and according to the target transmission path index corresponding to the target data transformation result, the target path identifier associated with the target transmission path is determined, there are multiple transmission paths between the source node and the target node, and the transmission path index of each of the multiple transmission paths is associated with at least one path identifier.
[0034] Optionally, the first processing unit is specifically configured to:
[0035] Based on the routing record, the target transmission path index corresponding to the target data transformation result is obtained, and each transmission path index in the routing record is cyclically repeated multiple times according to a preset order, and each transmission path index with a new sorting has its own sorting number, which is the data transformation result.
[0036] Optionally, the first processing unit is further used to, if a node in the transmission path fails, determine a faulty transmission path index of the faulty transmission path of the faulty node, and replace the faulty transmission path index in the routing record with another transmission path index.
[0037] Optionally, the preset conversion method is an XOR hash function, and the XOR hash function and hash seed in the one intermediate node are the same as those in the other nodes.
[0038] In a third aspect, an embodiment of the present application provides a computer device, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes any one of the network load balancing methods in the first aspect.
[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which includes a computer program. When the computer program is run on a computer device, the computer program is used to enable the computer device to execute any one of the network load balancing methods in the above-mentioned first aspect.
[0040] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium; when a processor of a computer device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the computer device executes any one of the network load balancing methods in the above-mentioned first aspect.
[0041] The beneficial effects of this application are as follows:
[0042] A network load balancing method, device, computer equipment and storage medium provided by the embodiment of the present application are applicable to a load balancing system. The network topology structure applicable to the load balancing system in the present application includes multiple nodes. When data is transmitted between nodes, routing is generally performed according to a five-tuple. The five-tuple includes a source IP, a destination IP, a source port, a destination port and a data transmission protocol identifier used. In a data transmission process, the transmission path selected for data transmission can be called a target transmission path. The target transmission path involves a source node, at least one intermediate node, and a target node that receives and processes a data packet in a network topology structure. The path between two nodes is a target sub-path (e.g., between two intermediate nodes, between a source node and an intermediate node, and between an intermediate node and a target node), and at least one target sub-path is connected in series to form a target transmission path. During the data transmission process, after the source node generates a five-tuple in a data packet, the target transmission path of the data packet can be selected according to the five-tuple. When the source node selects the target transmission path according to the five-tuple, it generally performs preliminary load balancing so that the resource utilization of each transmission path in the source node is equivalent.
[0043] Afterwards, after receiving the data packet, each intermediate node involved in the target transmission path performs data conversion for the target path identifier of the target transmission path to obtain the target data conversion result, and then determines the target sub-path index associated with the target data conversion result from the first mapping record according to the target data conversion result, and then sends the data packet to the next hop node based on the target sub-path corresponding to the target sub-path index until the data packet is transmitted to the target node. In this way, the intermediate node corresponding to each target sub-path will obtain the target sub-path under the target transmission path under the determined target path identifier, so that the data packet can accurately pass through each target sub-path of the target transmission path in sequence, ensuring the consistency of the route, and effectively preventing the network equipment in the related technology from using the ECMP hash algorithm to randomly assign paths to data flows, resulting in uneven distribution of data flows in multiple paths when there are fewer data flows, as well as the load imbalance phenomenon of the hash polarization problem.
[0044] Other features and advantages of the present application will be described in the subsequent description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0046] Figure 1 A schematic diagram of a data flow routing method in a related art provided in an embodiment of the present application;
[0047] Figure 2 A simple schematic diagram of a network topology structure provided in an embodiment of the present application;
[0048] Figure 3 A simple schematic diagram of a data center network topology with a three-layer network hierarchical design provided in an embodiment of the present application;
[0049] Figure 4 An optional schematic diagram of an application scenario provided in an embodiment of the present application;
[0050] Figure 5 An optional schematic diagram of an application scenario provided in an embodiment of the present application;
[0051] Figure 6 An optional schematic diagram of an application scenario provided in an embodiment of the present application;
[0052] Figure 7 An optional schematic diagram of another application scenario provided in an embodiment of the present application;
[0053] Figure 8 A flowchart of a network load balancing method provided in an embodiment of the present application;
[0054] Fig. 9 A schematic diagram of the structure of a data packet provided in an embodiment of the present application;
[0055] Fig.10 A schematic diagram of the structure of a data packet provided in an embodiment of the present application;
[0056] Fig.11 A schematic diagram of the structure of another data packet provided in an embodiment of the present application;
[0057] Fig.12 A schematic diagram of a packet header structure of a data packet of each transmission path in the same data transmission tunnel provided by an embodiment of the present application;
[0058] Fig.13 A flowchart of a data transmission method provided in an embodiment of the present application;
[0059] Fig.14 A flowchart of a network load balancing method provided in an embodiment of the present application;
[0060] Fig.15 A schematic diagram of a transmission path planning method provided in an embodiment of the present application;
[0061] Fig.16 A schematic diagram of a transmission path provided in an embodiment of the present application;
[0062] Fig.17 A schematic diagram of a data flow routing method provided in an embodiment of the present application;
[0063] Fig.18 A schematic diagram of a data flow routing method when a sending node fails provided in an embodiment of the present application;
[0064] Fig.19 A transmission diagram of a data stream of a currently faulty transmission path provided in an embodiment of the present application;
[0065] Fig. 20 A transmission schematic diagram of a data stream of a faulty transmission path provided in an embodiment of the present application;
[0066] Fig.21 A schematic diagram of the relationship between each sub-path and the transmission path in an intermediate node provided in an embodiment of the present application;
[0067] Fig. 22 A schematic diagram of the configuration of an intermediate node and other nodes provided in an embodiment of the present application;
[0068] Fig.23 A flowchart of a data packet processing method provided in an embodiment of the present application;
[0069] Fig.24 A schematic diagram of network traffic distribution to a target node provided in an embodiment of the present application;
[0070] Fig.25 A schematic diagram of network traffic distribution to a target node provided in an embodiment of the present application;
[0071] Fig.26 A schematic diagram of a configuration in a receiving node provided in an embodiment of the present application;
[0072] Fig. 27 A schematic diagram of source port number allocation provided in an embodiment of the present application;
[0073] Fig.28 A schematic diagram of a network load balancing method provided in an embodiment of the present application;
[0074] Fig.29A schematic diagram of a network load balancing application in a server provided in an embodiment of the present application;
[0075] Fig.30 A schematic diagram of a network load balancing device provided in an embodiment of the present application;
[0076] Fig.31 A schematic diagram of a hardware structure of a computer device to which an embodiment of the present application is applied;
[0077] Fig.32 A schematic diagram of the hardware composition structure of another computer device to which the embodiment of the present application is applied. DETAILED DESCRIPTION
[0078] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.
[0079] It is understandable that in the following specific implementations of the present application, data packets, quintuples and other related data are involved. When the various embodiments of the present application are applied to specific products or technologies, relevant licenses or consents need to be obtained, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, when it is necessary to obtain relevant data, relevant volunteers can be recruited and relevant agreements for volunteer authorization data can be signed, and then the data of these volunteers can be used for implementation; or, by implementing within the scope of the authorized organization, the following implementation methods are implemented by using the data of internal members of the organization to make relevant recommendations to internal members; or, the relevant data used in the specific implementation are all simulated data, such as simulated data generated in a virtual scene.
[0080] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:
[0081] Network topology structure: refers to the physical layout of various devices interconnected by transmission media. It refers to the specific physical, real, or logical, virtual arrangement between the members that constitute the network. It refers to the physical layout of connecting various devices such as computers to each other with transmission media. It refers to the geometric shape formed in the interconnection process. It can represent the network configuration of network servers and workstations and the connection between each other. Network topology structures can be classified by shape, including: star, ring, bus, tree, bus / star and mesh topologies. For example, the star topology connects each node to the central node, presenting a radial arrangement, and controls the communication of the entire network through the central node. The bus-type computer network topology mainly connects the surrounding nodes through a high-speed trunk cable. Among them, if the connection structure of two networks is the same, it can be said that their network topologies are the same, although their internal physical wiring and the distance between nodes may be different. For example Figure 2 The figure is a simple schematic diagram of a network topology structure provided in an embodiment of the present application.
[0082] Source node and destination node: Based on the network topology, in the transmission network, service transmission also has a starting point and an end point, but they are replaced by the standard source and sink, which are the source node and the destination node respectively. Figure 2 As shown in the figure, if host A sends a data packet to host B, host A is the source node and host B is the destination node.
[0083] Intermediate node: Based on the network topology, in the transmission network, it is the "intermediate station" between the starting point and the end point of the service transmission. Based on the above example, Figure 2 As shown, if host A sends a data packet to host B through switch 3 and switch 2, then switch 3 and switch 2 are intermediate nodes.
[0084] Next hop node: Based on the network topology, in the transport network, the next node of the service transmission route. If the router is not directly connected to the destination network, it will have a neighbor router (next hop node) that provides the next hop route to pass the data to the destination (target node). Based on the above example, if Figure 2 As shown, the next hop node of host A is switch 3, the next hop node of switch 3 is switch 2, and the next hop node of switch 2 is host B.
[0085] Network distance: Based on the network topology, the minimum number of edges between two nodes in the interconnected network. Based on the above example, Figure 2 As shown, the network distance between host A and host B is 3, the network distance between switch 3 and host B is 2, and the network distance between switch 1 and host C is 2.
[0086] OSI model: Open System Interconnection Reference Model (OSI), referred to as OSI model, is a conceptual model proposed by the International Organization for Standardization, a standard framework that attempts to interconnect various computers into a network worldwide. It includes the physical layer, data link layer, transport layer, session layer, presentation layer and application layer.
[0087] Transport protocol: refers to the protocol used when sending data packets, such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), etc.
[0088] Transmission Control Protocol (TCP): is a connection-oriented, reliable, byte-stream-based transport layer communications protocol defined by IETF RFC 793[1]. It is designed to accommodate a layered protocol hierarchy that supports multiple network applications. TCP is used to provide reliable communications services between pairs of processes in host computers connected to different but interconnected computer communications networks. TCP assumes that it can obtain simple, possibly unreliable datagram services from lower-level protocols. In principle, TCP should be able to operate on a variety of communications systems, from hard-wired connections to packet-switched or circuit-switched networks.
[0089] User Datagram Protocol (UDP): A connectionless transport protocol supported by the Internet protocol suite. UDP provides a way for applications to send encapsulated IP data packets without establishing a connection. RFC768 describes UDP. The Internet's transport layer has two main protocols that complement each other. The connectionless one is UDP, which gives applications the ability to send data packets and allows them to structure their own protocols at the required level. The connection-oriented one is TCP, which does almost everything.
[0090] Quintuple: It is a communication term, usually referring to a set of five quantities: source IP address, source port, destination IP address, destination port and transport layer protocol. For example, 192.168.1.1, 10000, TCP, 121.14.88.76, 80 constitute a quintuple. Its meaning is that a terminal with an IP address of 192.168.1.1 connects to a terminal with an IP address of 121.14.88.76 and a port of 80 through port 10000 and TCP protocol. Quintuples can distinguish different sessions, and the corresponding sessions are unique.
[0091] Source port: This field occupies the first 16 bits of UDP and generally contains the UDP port used by the program in the source node that sends the datagram. The program in the target node that receives the datagram uses the value of this field as the target address to send the response. This field is optional. The program in the source node that sends the datagram does not have to write its port in this field. If the program in the source node that sends the datagram does not write its port number, this field should be set to all 0s. If this field does not contain a valid port address, the program in the target node that receives the datagram cannot send a response.
[0092] Source IP address: refers to the IP address of the source node that sends the data packet, which is used to identify the source of the data packet.
[0093] Destination IP address: refers to the IP address of the target node that receives the data packet, which is used to determine the destination of the data packet.
[0094] Destination port: The UDP port used by the program in the target node to receive the data message.
[0095] Source port number: refers to the port number used by the source node that sends the data packet, which is used to identify which application in the source node the data packet is sent from.
[0096] Destination port number: refers to the port number used by the target node to receive the data packet, which is used to identify which application in the target node the data packet should be handed over to for processing.
[0097] Target transmission path: The path between the source node and the target node for transmitting data packets. Based on the above example, Figure 2 As shown, the target transmission path is host A->switch 3->switch 2->host B.
[0098] Data transmission tunnel: A data transmission tunnel determined by the source IP of the source node and the destination IP of the target node. The data transmission tunnel can contain multiple transmission paths. Based on the above example, Figure 2As shown, the data transmission tunnel determined by the source IP of host A and the destination IP of host B includes multiple transmission paths, such as, transmission path: host A->switch 3->switch 2->host B, transmission path: host A->switch 3->switch 4->host B, transmission path: host A->switch 1->switch 4->host B, transmission path: host A->switch 1->switch 2->host B.
[0099] Virtual eXtensible Local Area Network (VXLAN): It is a network virtualization technology that can improve the expansion problem of large-scale cloud computing during deployment. It is an extension of VLAN. VXLAN is a tunnel technology that can establish a Layer 2 Ethernet network tunnel on the basis of a Layer 3 network, thereby realizing cross-regional Layer 2 interconnection and extending the Layer 2 through a Layer 3 network. It can solve the portability limitation of VMS (Virtual Memory System) by encapsulating traffic and extending it to the Layer 3 gateway, so that it can access servers on external IP subnets. The main principle is to introduce an outer tunnel in UDP format as the data link layer, and encapsulate the original Ethernet message in the UDP data packet. The original Layer 2 data frame plus the VXLAN header are encapsulated in a UDP data packet, and the original data message content is transmitted as the tunnel payload. Since the outer layer uses UDP as the transmission means, the payload data can be easily transmitted in the Layer 2 and Layer 3 networks.
[0100] Layer 2 Tunneling Protocol (Generic Routing Encapsulation, GRE) is an IP encapsulation protocol that allows a protocol in a network to be encapsulated into another protocol. It is a network tunneling technology that can be used to establish a network tunnel, where one end sends data and the other end receives data and encapsulates the data in another protocol. It can establish tunnels between different types of networks, allowing data packets to be sent from one network through another network without using application layer protocols. Data in another network can be sent to another network through a network tunnel. GRE Layer 2 tunneling refers to the use of the GRE protocol to establish a Layer 2 tunnel between two independent networks, allowing the two parties to communicate data. The GRE Layer 2 Tunneling Protocol can be used to establish tunnels between different networks, or to implement tunnels in the same network. It can also be used to implement network virtual private networks (VPNs), multi-layer routing, and other network applications.
[0101] The three-layer network hierarchical design includes: core layer switches, aggregation layer switches and access layer switches, as well as a layer of servers. The access layer switches and the servers connected to them are collectively called a rack, and the access layer switches and all the aggregation layer switches connected to them, as well as all the servers connected to them, are collectively called a module (Pod). Figure 3 As shown, it is a simple schematic diagram of a data center network topology of a three-layer network hierarchical design provided by an embodiment of the present application. Assuming that in the three-layer network hierarchical design, there are 8 core switches C0-C7 in the core layer, 16 switches S0-S15 in the aggregation layer, 16 switches L0-L15 in the access layer, and 16 servers H0-H15 under L0-L15. The access switches L0, L1 and the servers H0, H1 under it form a rack (Rack0); and all the devices in the first large dotted box form a network module (Pod0). It should be noted that the schematic diagram here is only used to clearly explain the architecture of the three-layer network hierarchical design. In the actual data center with a three-layer network hierarchical design, there can be more core layer switches, aggregation layer switches, access layer switches and servers. There can be more servers and access layer switches in a Pod, and there can be more servers, access layer switches and aggregation layer switches in a Rack. In order to increase the communication bandwidth and connection reliability between servers, the server generally uses two links to connect to two access layer switches (there are two network ports on a network card of the server, which are connected to two switches respectively).
[0102] Core layer: The core layer is the high-speed switching backbone of the network, and plays a vital role in the connectivity of the entire network. The core layer should have the following characteristics: reliability, efficiency, redundancy, fault tolerance, manageability, adaptability, low latency, etc. In the core layer, high-bandwidth switches of Gigabit or above should be used. Because the core layer is the hub of the network, it is of outstanding importance. It is very necessary for the core layer equipment to use dual-machine redundant hot backup, and the load balancing function can also be used to improve network performance.
[0103] Aggregation layer: The aggregation layer is the "intermediary" between the network access layer and the core layer. It aggregates the workstations before they access the core layer to reduce the load on the core layer equipment. The aggregation layer has many functions, such as implementing policies, security, workgroup access, routing between virtual local area networks (VLANs), and filtering source or destination addresses. In the aggregation layer, switches that support Layer 3 switching technology and VLANs should be selected to achieve network isolation and segmentation.
[0104] Access layer: The access layer provides workstation access to the local network segment. In the access layer, the number of workstations in the same network segment is reduced, and high-speed bandwidth can be provided to the workgroup. The access layer can choose ordinary switches that do not support VLAN and Layer 3 switching technology.
[0105] Network card: is a piece of computer hardware designed to allow computers to communicate on a computer network. Because it has a MAC address, it belongs between layers 1 and 2 of the OSI model. It allows users to connect to each other through cables or wirelessly. Each network card has a unique 48-bit serial number called a MAC address, which is written in a ROM on the card. Every computer on the network must have a unique MAC address. No two network cards produced have the same address. This is because the Institute of Electrical and Electronics Engineers (IEEE) is responsible for assigning unique MAC addresses to network interface controller (network card) vendors.
[0106] Index: In a relational database, an index is a separate, physical storage structure that sorts the values of one or more columns in a database table. It is a collection of values of one or more columns in a table and a corresponding list of logical pointers to the physical identifiers in the table and the data pages of these values. It acts like a catalog of books, and you can quickly find the required content based on the page numbers in the catalog. The index provides pointers to the data values stored in the specified columns of the table, and then sorts these pointers according to the specified sort order. The database uses the index to find a specific value, and then follows the pointer to find the row containing the value. This allows the SQL statement corresponding to the table to execute faster and quickly access specific information in the database table.
[0107] Pointer is the memory address. Pointer variable is used to store the memory address. Under the same CPU architecture, different types of pointer variables occupy the same storage unit length, while the variables storing data occupy different storage space lengths due to different data types. With pointers, you can operate not only on the data itself, but also on the variable address storing the data. Pointers describe the location of data in memory, marking an entity occupying storage space and the relative distance value of the starting position of this space. In C / C++ language, pointers are generally considered to be pointer variables. The content of pointer variables stores the first address of the object they point to. The object pointed to can be a variable (pointer variables are also variables), array, function and other entities occupying storage space.
[0108] The technical solution of the embodiment of the present application involves cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network in a wide area network or a local area network to realize the calculation, storage, processing, transmission, and sharing of data. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model application, which can form a resource pool, which is used on demand and is flexible and convenient. Among them, the background service of the technical network system requires a large amount of computing and storage resources, such as video websites, picture websites, and more portal websites. With the high development and application of the Internet industry, each item may have its own identification mark in the future, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately, and all kinds of industry data need strong system backing support, which can only be achieved through cloud computing. Cloud computing technology will become an important support for cloud technology.
[0109] Cloud computing: refers to the delivery and use mode of IT infrastructure, which means obtaining required resources through the network in an on-demand and easily scalable manner; in a broad sense, cloud computing refers to the delivery and use mode of services, which means obtaining required services through the network in an on-demand and easily scalable manner. This service can be related to IT and software, the Internet, or other services. Cloud computing is the product of the development and integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing. With the development of the Internet, real-time data streams, and the diversification of connected devices, as well as the promotion of search services, social networks, mobile commerce, and open collaboration, cloud computing has developed rapidly. For example, in the transmission of data streams, the load balancing technology of cloud computing is applied. Based on the load balancing technology, multiple data streams are evenly distributed to each transmission path, so that the load conditions of each transmission path are similar, the utilization rate of network resources is improved, and the quality of service provided is improved.
[0110] The following is a brief introduction to the design concept of the embodiment of the present application:
[0111] The data center network topology is mainly divided into three layers of topology, namely the core switching layer that interconnects the data center with external operators, the data center network topology user layer or access layer, and the aggregation layer that connects the two to achieve data aggregation.
[0112] At present, data center networks generally use a routing method based on the ECMP hash algorithm to route and address data packets. Among them, the ECMP hash algorithm is set in the access layer switch and the aggregation layer switch in the data center network, and the general switch contains multiple output ports, and the multiple output ports correspond to multiple paths of equal network distance to the same destination IP. When the data packet directed to the destination IP arrives at the switch, the switch uses the ECMP hash algorithm to select one from the multiple output ports to send the data packet out from the output port. In this way, based on the randomness of the ECMP hash algorithm, the switch can evenly distribute each data flow to each output port, use the path of each output port, and distribute the data flow to the corresponding next-hop node, effectively improving the service throughput and network availability. However, this method is prone to uneven distribution of data flows when there are fewer data flows or fewer paths, and this method has the problem of hash polarization and load imbalance.
[0113] For example, if there are fewer data flows or fewer paths, if the switch contains 4 egress ports, and the egress port numbers of the 4 egress ports are 1, 2, 3, and 4 respectively, the switch may distribute all data flows to egress port 1, resulting in an excessive load on the path corresponding to egress port 1, while the resources of the paths corresponding to egress ports 2, 3, and 4 are idle, which in turn leads to a decrease in the quality of the services provided and low resource utilization. For another example, in hash polarization, if Figure 1 As shown, a data packet transmission path has been cited above and will not be repeated here. Based on the data packet transmission path of the above example, it can be seen that in the next-hop node (lower-layer network device), the load balancing method based on the ECMP hash algorithm has failed, resulting in idle link resources, which will also lead to a decline in service quality and low resource utilization.
[0114] In view of this, the embodiments of the present application provide a network load balancing method, apparatus, computer equipment and storage medium, which are applicable to a load balancing system. The network topology structure applicable to the load balancing system in the present application includes multiple nodes. In a data transmission process, the transmission path selected for data transmission can be called a target transmission path. The target transmission path involves a source node, at least one intermediate node, and a target node that receives and processes a data packet in the network topology structure, wherein the path between two nodes is a target sub-path, and at least one target sub-path is connected in series to form a target transmission path. During the data transmission process, after the source node generates a quintuple in the data packet, the target transmission path of the data packet can be selected according to the quintuple. When the source node selects the target transmission path according to the quintuple, it generally performs preliminary load balancing so that the resource utilization of each transmission path in the source node is equivalent.
[0115] Afterwards, after receiving the data packet, each intermediate node involved in the target transmission path performs data conversion for the target path identifier of the target transmission path to obtain the target data conversion result, and then determines the target sub-path index associated with the target data conversion result from the first mapping record according to the target data conversion result, and then sends the data packet to the next hop node based on the target sub-path corresponding to the target sub-path index until the data packet is transmitted to the target node. In this way, the intermediate node corresponding to each target sub-path will obtain the target sub-path under the target transmission path under the determined target path identifier, so that the data packet can accurately pass through each target sub-path of the target transmission path in sequence, ensuring the consistency of the route, and effectively preventing the network equipment in the related technology from using the ECMP hash algorithm to randomly assign paths to data flows, resulting in uneven distribution of data flows in multiple paths when there are fewer data flows, as well as the load imbalance phenomenon of the hash polarization problem.
[0116] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.
[0117] like Figure 4 As shown, it is a schematic diagram of an application scenario of an embodiment of the present application, which is applicable to a load balancing system. The application scenario diagram includes multiple servers 410, an access layer switch 420, a convergence layer switch 430, and a core layer switch 440.
[0118] It should be noted that the number of devices of the multiple servers 410, access layer switches 420, aggregation layer switches 430 and core layer switches 440 in this application scenario is not specifically limited and can be set as needed. The transmission paths between the upper network devices and the lower network devices (e.g., servers and access layer switches, access layer switches and aggregation layer switches, aggregation layer switches and core layer switches) can also be set as needed, such as Figure 5 As shown in the figure, one path from the server to the access layer switch in the left dotted box is closed, indicated by "×", and one path from the access layer switch to the aggregation layer switch in the left dotted box is closed, indicated by "×". In addition, Figure 4 The network topology levels (multiple servers, access layer switches, aggregation layer switches, and core layer switches) in the application scenario shown in the figure are only one embodiment of the present application and do not limit the implementation of the present application scheme. For example, a user access switch may also be included between the multiple servers 410 and the access layer switch 420, such as Figure 6As shown in the figure, the application scenario includes multiple servers 610, an access layer switch 620, a convergence layer switch 630, a core layer switch 640 and a user access switch 650.
[0119] Among them, the above Figure 4 , Figure 5 , Figure 6 The source node in the example may be a server running a related business system, and may autonomously generate data packets for transmission. For example, the server cluster where the source node is located may be a server cluster that provides account management services, and the server cluster where the target node is located may be a server cluster that provides related payment services. The source node may generate an account verification request at a set time and send it to the target node, so that the target node performs account verification processing. For another example, the server cluster where the source node is located may be a server cluster that provides intelligent transportation services, and the server cluster where the target node is located may be a server cluster that provides road condition management services. The source node may generate a map update request at a set time and send it to the target node, so that the target node obtains the map of the corresponding location and returns it to the source node. It should be noted that there is no specific restriction on the related business systems running in the source node and the target node, and they may be set as needed. The above examples are only used to clearly illustrate this solution.
[0120] The process of the source node sending a data packet can also be triggered by the terminal device to generate and send a data packet. Figure 4 As an example, the present application embodiment also provides an application scenario, such as Figure 7 As shown, the application scenario includes multiple servers 710, access layer switches 720, aggregation layer switches 730, core layer switches 740, and terminal devices 750. For example, the server cluster where the source node is located can be a server cluster that provides account management services, and the server cluster where the target node is located can be a server cluster that provides related payment services. After receiving the bill acquisition request from the terminal device 750, the source node can generate a bill pull request and send it to the target node, so that the target node returns the bill of the account of the terminal device 750 to the source node. For another example, the server cluster where the source node is located can be a server cluster that provides intelligent transportation services, and the server cluster where the target node is located can be a server cluster that provides road condition management services. After receiving the map acquisition request sent by the terminal device 750, the source node generates a map pull request and sends it to the target node, so that the target node obtains the map of the corresponding location and returns it to the source node. It should be noted that there is no specific restriction on the relevant business systems running in the source node and the target node, which may be set as needed. The above examples are only used to clearly illustrate this solution.
[0121] In the embodiment of the present application, the terminal device 750 includes but is not limited to mobile phones, tablet computers, laptop computers, desktop computers, e-book readers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals and other devices; the terminal device 750 may be installed with a client related to the business in the source node of the multiple servers 710, and the client may be software (such as a browser, communication software, etc.), or a web page, a small program, etc., and the source node is a background service corresponding to the software or web page, small program, etc., or a background service specifically used to recommend multimedia information to the client, which is not specifically limited in the present application. The source node in the multiple servers 710 may be a server cluster or a distributed system composed of multiple physical servers, and the source node in the multiple servers 710 may also be an independent physical server, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0122] It should be noted that the network load method in the embodiment of the present application can be executed by a computer device. Figure 4 The application scenario in is described, the computer device can be a source node and a target node in multiple servers 410, or an intermediate node in an access layer switch 420, or an intermediate node in an aggregation layer switch 430, or an intermediate node in a core layer switch 440, that is, the method can be jointly executed by the source node and the target node in multiple servers 410, the intermediate node in the access layer switch 420, the intermediate node in the aggregation layer switch 430, and the intermediate node in the core layer switch 440, then the network load balancing method includes:
[0123] The source node generates a data packet containing a five-tuple. The target transmission path and its target path identifier of the data packet are determined according to the five-tuple of the data packet. Assume Figure 4 The bold lines in the figure are each a target sub-path. Multiple target sub-paths are connected in series to form a target transmission path, so that the target transmission path connects the source node and the target node respectively, and the target transmission path contains at least one intermediate node.
[0124] After receiving the data packet, the intermediate node in the access layer switch 420 performs data conversion on the target path identifier, obtains the target data conversion result, combines the first mapping record in the corresponding intermediate node, determines the target sub-path index associated with the target data conversion result, and sends the data packet to the next hop node (the intermediate node in the aggregation layer switch 430) based on the target sub-path corresponding to the target sub-path index.
[0125] After receiving the data packet, the intermediate node in the aggregation layer switch 430 performs data conversion on the target path identifier, obtains the target data conversion result, combines the first mapping record in the corresponding intermediate node, determines the target sub-path index associated with the target data conversion result, and sends the data packet to the next hop node (the intermediate node in the core layer switch 440) based on the target sub-path corresponding to the target sub-path index.
[0126] After receiving the data packet, the intermediate node in the core layer switch 440 performs data conversion on the target path identifier, obtains the target data conversion result, combines the first mapping record in the corresponding intermediate node, determines the target sub-path index associated with the target data conversion result, and sends the data packet to the next hop node (target node) based on the target sub-path corresponding to the target sub-path index.
[0127] After receiving the data packet, the target node determines the processing thread of the data packet and processes the data packet based on the processing thread.
[0128] It should be noted that Figure 4 The above is only an example. In fact, the number of servers, the number of switches, the number of intermediate nodes, and the communication methods between them are not limited. They are not specifically limited in the embodiments of the present application. For example, in general, the load capacity of the core layer switch 440 is very strong, and the intermediate nodes of the core layer switch 440 may not adopt the network load balancing method in the present application. In addition, it can also be combined with Figure 5 In the application scenario in the network load balancing method, the computer device can be the source node and the target node in the multiple servers 510, or the intermediate node in the access layer switch 520, or the intermediate node in the aggregation layer switch 530, or the intermediate node in the core layer switch 540, that is, the method can be jointly executed by the source node and the target node in the multiple servers 510, the intermediate node in the access layer switch 520, the intermediate node in the aggregation layer switch 530, and the intermediate node in the core layer switch 540. It can also be combined with Figure 6 In the application scenario in the network load balancing method, the computer device can be the source node and the target node in multiple servers 610, or the intermediate node in the access layer switch 620, or the intermediate node in the aggregation layer switch 630, or the intermediate node in the core layer switch 640, or the intermediate node in the user access switch 650, that is, the method can be jointly executed by the source nodes and the target nodes in multiple servers 610, the intermediate nodes in the access layer switch 620, the intermediate nodes in the aggregation layer switch 630, the intermediate nodes in the core layer switch 640, and the intermediate nodes in the user access switch 650. Figure 7 Application scenarios and Figure 4 and Figure 5 The application scenarios are similar to those in Figure 7 The source node, target node and intermediate node in the figure have been shown in the figure and will not be repeated here. Figure 5 , Figure 6 and Figure 7 The network load balancing method in this application is implemented in the application scenario in the same way as above Figure 4 Similar, I won’t go into details here.
[0129] In addition, the network load balancing of the embodiments of the present application can be applied to various scenarios, such as payment services, smart transportation services, related mini-program services in conversational software, advertising recommendation services, and online shopping services, etc., which involve data transmission between servers or between terminal devices or between servers and terminal devices.
[0130] The following describes the network load balancing method provided by the exemplary embodiment of the present application in combination with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this regard.
[0131] See also Figure 8 As shown, it is a flow chart of a network load balancing method provided in an embodiment of the present application. Here, a load balancing system is used as an example for illustration. The specific implementation process of the method is as follows:
[0132] Step 801: based on a five-tuple in a data packet generated by a source node, determine a target transmission path of the data packet and its target path identifier, wherein the target transmission path connects the source node and the target node respectively, includes at least one intermediate node, and is formed by connecting a plurality of target sub-paths in series;
[0133] In one embodiment, when sending a data packet containing a five-tuple, the source node may carry a target path identifier.
[0134] In one embodiment, the OSI model can be used in the source node, and the application layer, presentation layer and session layer in the OSI model can generate original data. Specifically, the application layer: generates relevant information of the request according to the set transmission protocol (format); the presentation layer: encrypts and compresses the relevant information of the request according to a specific encryption method and compression method, and obtains the encrypted and compressed request as the original data; the session layer: establishes, maintains and releases user-oriented connections, and manages and controls sessions to ensure reliable transmission of session data; further, the transport layer is used to determine the source port number, destination port number and data transmission protocol identifier of the original data, and the source port number, destination port number, and data transmission protocol identifier are used as the source port number, destination port number, and data transmission protocol identifier. and data transmission protocol identifier as the packet header information of the original data; using the network layer to determine the source IP and destination IP of the source node corresponding to the original data, and generating the inner packet header of the data packet according to the source port number, destination port number, source IP, destination IP and data transmission protocol identifier of the original data; using the data link layer; calculating the inner five-tuple of the data packet based on the hash function to obtain the target path and its target path identifier, generating the outer packet header of the data packet according to the target path identifier, destination port number, source IP, destination IP and data transmission protocol identifier, encapsulating the data packet to obtain the final data packet, which can be sent to the next hop node through the data link layer and the physical layer. Here, a structural schematic diagram of a data packet is provided in an embodiment of the present application, such as Fig. 9 As shown, it is a data packet including an inner header and an outer header.
[0135] In one embodiment, the data packet may also include network service information, such as Fig.10 As shown, a schematic diagram of the structure of a data packet is provided in an embodiment of the present application, which includes network service information.
[0136] In one embodiment, the present application may use the Transmission Control Protocol (TCP) and the User Datagram Protocol (UDP) to carry service traffic.
[0137] In one embodiment, the present application adopts the tunnel protocol VxLAN-GPE, and the embodiment of the present application provides a structural diagram of another data packet, such as Fig.11As shown in the figure, the Outer headers are all used for tunnel encapsulation, the Inner header is the original data Payload, Outer-Eth represents the outer Ethernet header, Outer-IP represents the outer IP header (outer header), Outer UDP represents the outer UDP header (outer header), VxLAN-GPE represents that the tunnel protocol uses VxLAN-GPE, NSH is the network service header of the tunnel, Inner-Eth represents the inner Ethernet header (inner header), which is the original data, and Inner-IP represents the inner IP header (inner header), which is also the original data.
[0138] In one embodiment, the source node includes multiple egress ports, each of which can be connected to a next-hop node, and the next-hop node can also include multiple egress ports, and each egress port in the next-hop node can be connected to a next-down-hop node. Then, each egress port in the source node can correspond to multiple transmission paths, and the multiple transmission paths in each egress port in the source node have their own path identifiers, such as Figure 4 As shown in , assuming that there are only two switches in the access layer switch 420, only two switches in the aggregation layer switch 430, and only two switches in the core layer switch 440, there are 8 transmission paths between the source node and the target node. The network card of the source node includes two egress ports, which are respectively connected to the two switches in the access layer switch 420. Then, one egress port in the source node has 4 transmission paths to the target node. The source node determines the egress port in the source node to send the data packet according to the quintuple, selects one of the 4 transmission paths in the egress port as the target transmission path, and determines the target path identifier of the target transmission path.
[0139] In one embodiment, based on the above embodiment, the present application uses a data transmission tunnel with multiple transmission paths to carry all traffic between an IP pair (source IP and destination IP). The present application can be part of a Highly Available and Reliable Protocol (HARP). Multiple paths in the same data transmission tunnel have the same quadruple (source / destination IP, destination port number and protocol number), but have different source port numbers (path identifiers of transmission paths), such as Fig.12 As shown, it is a schematic diagram of the packet header structure of data packets of each transmission path in the same data transmission tunnel provided by an embodiment of the present application.
[0140] In one embodiment, in order to clearly illustrate the present solution, as Fig.13 As shown, it is a flowchart of a data transmission method provided in an embodiment of the present application, including:
[0141] Step 1301: When sending a data packet, the source node will check whether there is a data transmission tunnel leading to the destination IP.
[0142] Step 1302: If it does not exist, create a data transmission tunnel to the destination IP. After the creation is completed, configure the path identification information of each transmission path in the data transmission tunnel.
[0143] Step 1303: If it exists, it can be used directly. According to the five-tuple in the inner header of the data packet, a hash algorithm is used to obtain the target path index of the target transmission path, and then the target path identifier is obtained. Based on the target path identifier and related information such as the destination port, source IP, destination IP, etc., the outer header of the data packet is obtained and the data packet is encapsulated.
[0144] Step 1304: Transmit the encapsulated data packet including the target path identifier to the target node via the target transmission path.
[0145] Step 1305: When receiving a data packet, the target node will also first check whether there is a data transmission tunnel leading to the source IP.
[0146] Step 1306: If it does not exist, create a data transmission tunnel leading to the source IP. After the creation is completed, configure the path identification information of each transmission path in the data transmission tunnel.
[0147] Step 1307: If it exists, it can be used directly to decapsulate the data packet.
[0148] based on Figure 8 The method flow in this application embodiment provides a network load balancing method, such as Fig.14 As shown, in step 801, the determination of the target transmission path of the data packet and its target path identifier based on the five-tuple in the data packet generated by the source node includes:
[0149] Step 1401: Based on the quintuple in the data packet generated by the source node, a preset transformation method is used to obtain a target data transformation result corresponding to the quintuple;
[0150] Step 1402: determine the target path identifier associated with the target transmission path according to the target transmission path index corresponding to the target data transformation result, there are multiple transmission paths between the source node and the target node, and the transmission path index of each of the multiple transmission paths is associated with at least one path identifier.
[0151] In one embodiment, the preset transformation method adopts a hash algorithm, or a mapping function, etc. There is no limitation on the specific manner in which the preset transformation method is adopted, and it can be set as needed.
[0152] In one embodiment, the target data transformation result can be associated with the target transmission path index in a one-to-one correspondence, and each quintuple has a corresponding transmission path index of its own transmission path. For example, only the source IP is different in two quintuples, and the remaining 4 tuples are the same. The two quintuples can also be associated with different transmission path indexes respectively, or, if any tuple in the two quintuples is different, and the remaining 4 tuples are the same, the two quintuples can also be associated with different transmission path indexes respectively, that is, different quintuples are associated with different transmission paths. In other words, if there are enough transmission paths to ensure that each quintuple can be assigned a dedicated transmission path, each quintuple can have a dedicated transmission path. If the transmission path is insufficient, multiple quintuples can also share a transmission path, such as two data packets with the same source IP, destination IP, source port, and destination port can also use two transmission paths in the same data transmission tunnel respectively.
[0153] In one embodiment, the source node can use the ECMP hash algorithm to perform data transformation on the quintuple, obtain the target data transformation result, obtain the target transmission path index of the target data transformation result, and thus obtain the target transmission path and the target path identifier. Then, the ECMP hash algorithm can randomly transform each quintuple to obtain the target data transformation result and the target path identifier. In this way, the probability of each quintuple being assigned to each transmission path is substantially equal. For example, based on the above Figure 4 In the example, there are 8 transmission paths from the source node to the target node. Based on the ECMP hash algorithm in the source node, the probability of each quintuple being assigned to the 8 transmission paths is equal, ensuring network load balancing.
[0154] In one embodiment, the present application provides a schematic diagram of a transmission path planning method, such as Fig.15As shown, assuming that there are 64 source port numbers in total, there are 1 source node, 2 access layer switches (intermediate nodes), and 8 aggregation layer switches (intermediate nodes) in the application scenario of the network load balancing method, and assuming that the core layer switch does not apply the network load balancing method in this application, a full interconnection mode is adopted (a network connection form, that is, all nodes are directly connected and traffic is not transferred through a third-party node), then there are 16 physical transmission paths from the source node to the access layer switch to the aggregation layer switch; assuming that there are 64 logical path identifiers in the source node (which can be 64 source port numbers respectively, and the source port number is used to illustrate the path identifier later), then the 16 physical transmission paths are used as 16 network groups, and the 64 source port numbers are allocated to the 16 network groups, so that each group contains 4 source port numbers, so that each access layer switch has 32 source port numbers (logical path identifiers of logical transmission paths), and each aggregation layer switch has 4 source port numbers (logical path identifiers of logical transmission paths). Here, the reason why the network load balancing method in this application is not applied to the core layer switch is that the number of core layer switches is very large, and the core layer switches have enough resources for the traffic load, so only the source node to the access layer switch to the aggregation layer switch can be considered, and the above 16 physical transmission paths correspond to 64 logical path identifiers, that is, each physical transmission path corresponds to 4 logical transmission paths-4 source port numbers, then the data packet can be made from the source node-access layer switch-aggregation layer switch 1 physical transmission path, but can be dispersed at the aggregation layer switch-core switch according to 4 different source port numbers, further load balancing, using a relatively simple way to improve the network load balancing capability, such as Fig.16 As shown in the figure, the bold transmission path, when the data packet sent by the source node reaches the aggregation layer switch, because this transmission path has 4 different source port numbers, when the outer packet header of the data packet carries different source port numbers, different core layer switches can be used to transmit it to the target node. It should be noted that the transmission path from the aggregation layer switch to the core layer switch in the embodiment here is a divergent process, and the core layer switch downlink to the aggregation layer switch downlink to the target node is a convergent process. However, this embodiment does not limit the process from the core layer switch downlink to the aggregation layer switch downlink to the target node. The core layer switch from the aggregation layer switch to the target node can also be a divergent process, that is, the selection can be Fig.16 Other transmission paths except the bold transmission path on the right. Ideally, there are 65535 source port numbers, so each access layer switch can be allocated 32767.5 (source port numbers that cannot be integers can be randomly allocated to one of the two access layer switches), and each access layer switch can be allocated 8191 (the remaining 7 source port numbers can be randomly allocated to any 7 of the 8 aggregation layer switches).
[0155] In one embodiment, based on the above embodiment, the source node adopts the ECMP hash algorithm to perform data transformation on the quintuple, obtain the target data transformation result, obtain the target transmission path index of the target data transformation result, and thus obtain the target transmission path and the target path identifier (one of the 64 source port numbers). The ECMP hash algorithm can randomly transform each quintuple to obtain the target data transformation result and the target path identifier. In this way, the probability of each quintuple being assigned to the 64 source port numbers is basically equal, so that the probability of each quintuple being assigned to the 64 logical transmission paths is equal, thereby ensuring network load balancing.
[0156] In one embodiment, based on the above embodiment, the source port number can be used as the target path identifier, and the destination port number, source IP, destination IP, and transmission protocol identifier can be used as the outer header of the data packet. Correspondingly, different source port numbers identify different transmission paths as the path identifier of the transmission path.
[0157] Based on the above Fig.14 In the method flow, the embodiment of the present application provides a method for obtaining a target transmission path index. In step 1402, the target transmission path index corresponding to the target data transformation result includes:
[0158] Based on the routing record, the target transmission path index corresponding to the target data transformation result is obtained, and each transmission path index in the routing record is cyclically repeated multiple times according to a preset order, and each transmission path index with a new sorting has its own sorting number, which is the data transformation result.
[0159] In one embodiment, if Fig.17 As shown, assuming that there are 4 transmission paths from the source node to the target node, the preset order of each transmission path index is repeated multiple times (4 times), and a new order (1-16) of each transmission path index that is repeated multiple times is obtained. The source node performs data transformation on the five-tuple in the inner header of the data packet according to the hash function to obtain the target data transformation result (the new order (1-16) of each transmission path index). According to the routing record, the target transmission path index corresponding to the target data transformation result is obtained. Fig.17 The dotted line in the figure can represent the corresponding relationship between the data flow and the transmission path index, and the corresponding transmission path. Here, by repeatedly recording the preset sequence of each transmission path index in the routing record, and obtaining a new order of each transmission path index that is repeatedly repeated, and numbering the order as the data transformation result, the randomness of the hash function can be enhanced, and the load balancing capability can be further improved.
[0160] Based on the above-mentioned target transmission path index acquisition method, an embodiment of the present application also provides a method for processing a transmission path index when a node fails, including: if a node fails in a transmission path, determining the faulty transmission path index of the faulty transmission path of the faulty node, and replacing the faulty transmission path index in the routing record with other transmission path indexes.
[0161] In one embodiment, based on Fig.17 , if a node in transmission path 1 fails, the transmission path index 1 in the path record is replaced with other transmission path indexes, such as Fig.18 As shown in FIG. 1 , replacing transmission path index 1 with transmission path index 2 can solve the problem that the data packets of the data flow of the transmission path of the faulty node randomly select the transmission path, resulting in the data packets of the same data flow arriving at the target node out of order, making this data flow unable to be processed normally, such as Fig.19 FIG. 1 is a schematic diagram of a data flow transmission of a current faulty transmission path provided by an embodiment of the present application, and Fig. 20 As shown, a transmission diagram of a data stream of a fault transmission path provided by an embodiment of the present application. Fig.19 From the transmission diagram of the data flow of the current faulty transmission path, it can be seen that transmission path 2, transmission path 3, and transmission path 4 respectively send a portion of the data packets in data flow 1. Due to the inconsistent performance of different transmission paths or the performance jitter of the transmission paths, the order of the data packets in data flow 1 received by the target node will be disordered, and the information in data flow 1 cannot be obtained. Fig. 20 It can be seen from the transmission diagram of the data flow of the fault transmission path of the embodiment of the present application that the present application retains the sequence information of each data packet in the data stream 1, so that the sequence of each data packet of the data stream 1 received by the target node remains unchanged, and the information in the data stream 1 can be obtained.
[0162] Step 802: Based on the target transmission path, the data packet is sent to the target node through the at least one intermediate node; wherein, during the transmission process, each of the intermediate nodes performs the following operations:
[0163] Step 8021: Based on the target data conversion result of the target path identifier, in combination with a first mapping record preset corresponding to an intermediate node, determine a target sub-path index associated with the target data conversion result, wherein the first mapping record includes: at least one data conversion result associated with each sub-path index, and each sub-path is a path between the intermediate node and a corresponding next-hop node;
[0164] In one embodiment, the five-tuple of the data packet determines a data transmission tunnel, which includes multiple transmission paths. When the source node sends the data packet, it passes through at least one intermediate node to the destination node, for example, Figure 4 The target transmission path with bold lines is shown as the intermediate node-access layer switch, and the intermediate node-aggregation layer switch; when the source node transmits the data packet to the intermediate node through at least one target sub-path, it selects the target sub-path from the intermediate node to the next hop node from each sub-path of the intermediate node according to the target path identifier of the data packet. Among them, the sub-path identifier of each sub-path of the intermediate node can correspond to multiple path identifiers, so as Fig.21 As shown, it is a schematic diagram of the relationship between each sub-path and the transmission path in an intermediate node provided by an embodiment of the present application. If there are 64 transmission paths from the source node to the target node, the source node has 2 next-hop nodes-access layer switches, and each access layer switch has 8 next-hop nodes-aggregation layer switches, then the intermediate node is an access layer switch, corresponding to 8 next-hop nodes-aggregation layer switches, then the intermediate node-access switch has 8 sub-paths in the data transmission tunnel between the source node and the target node, and each sub-path corresponds to 4 path identifiers, Subpath 1-path identifier (4, 8, 7, 9), subpath 2-path identifier (20, 64, 42, 50), subpath 3-path identifier (13, 45, 21, 46), subpath 4-path identifier (15, 17, 23, 56), subpath 5-path identifier (31, 35, 48, 58), subpath 6-path identifier (29, 27, 60, 14), subpath 7-path identifier (39, 38, 37, 34), subpath 8-path identifier (11, 12, 14, 16). Assuming that the target path identifier is 15, the target subpath from the intermediate node-access layer switch to the next hop node is subpath 4. Then the first mapping record may include at least one data conversion result associated with each subpath index of the intermediate node.
[0165] In one embodiment, the intermediate node includes a first mapping record, and the first mapping record may include data transmission tunnels corresponding to multiple receiving nodes corresponding to multiple sending nodes, sub-paths of each transmission path in the corresponding data transmission tunnel in the intermediate node, and path identifiers corresponding to the sub-paths. For example, assuming that there are 5 sending nodes and 5 receiving nodes, there are 25 data transmission tunnels. Assuming that each data transmission tunnel has 64 transmission paths, there are 25*64 transmission paths in total. If there are 1000 transmission paths passing through the intermediate node, the first mapping record of the intermediate node may include 1000 data conversion results, and the 1000 data conversion results are distributed in the corresponding sub-paths from the intermediate node to each next-hop node, and the association between the sub-path index and the multiple data conversion results of the sub-path is recorded in the first mapping record.
[0166] Step 8022: Send the data packet to a next hop node based on the target subpath corresponding to the target subpath index.
[0167] In one embodiment, the next hop node may be a target node, or may be a next hop intermediate node of a current intermediate node. It should be noted that the number of intermediate nodes included in the target transmission path of the present application is not specifically limited.
[0168] In one embodiment, the intermediate node may include multiple egress ports, and a subpath corresponding to each egress port leads to a next hop node. The target subpath index may also be the target egress port index of the intermediate node.
[0169] In the above method, the network topology structure of the load balancing system includes multiple nodes. In a data transmission process, the transmission path selected for data transmission can be called a target transmission path. The target transmission path involves a source node, at least one intermediate node, and a target node that receives and processes data packets in the network topology structure, wherein the path between two nodes is a target sub-path, and at least one target sub-path is connected in series to form a target transmission path.
[0170] During the data transmission process, after the source node generates a five-tuple in the data packet, the target transmission path of the data packet can be selected according to the five-tuple, wherein the source node can obtain the transmission path between the source IP and the destination IP in advance, and further, assign a path identifier to each transmission path, and then obtain the corresponding target path identifier according to the data packet five-tuple, and generate the data packet outer header according to the target path identifier, and the first mapping record of the intermediate node already has an association relationship between the target path identifier and the target sub-path index, so that the intermediate node selects the corresponding target sub-path in the target transmission path according to the target path identifier in the data packet outer header, so as to achieve the purpose of controlling the transmission path of the data packet to follow the target transmission path determined by the source node. The load balancing system enables the source node, the intermediate node and the target node to cooperate to ensure the consistency of the routing, so that the network traffic load is balanced, and effectively prevents the network equipment in the related technology from using the ECMP hash algorithm to randomly assign paths to the data stream, resulting in uneven distribution of data streams in multiple paths when there are fewer data streams, and the load imbalance phenomenon of the hash polarization problem.
[0171] Based on the above Figure 8 , Fig.13 and Fig.14 The method in the embodiment of the present application provides a first mapping record generation method, wherein the network topology structure includes the one intermediate node and other nodes, the other nodes are nodes with the same network distance from the one intermediate node to the source node, and the one intermediate node and the other nodes are provided with a preset conversion method;
[0172] In step 8021, before determining the target sub-path index associated with the target data conversion result, the method further includes:
[0173] Step 1: Based on the preset conversion method, data conversion is performed on each path identifier in the source node to obtain a data conversion result of each path identifier;
[0174] Step 2: Evenly distribute the data conversion results to the sub-path indexes and other sub-path indexes, where other sub-paths are paths between other nodes and corresponding next-hop nodes;
[0175] Step 3: storing the association relationship between each subpath index and at least one assigned data conversion result, and the association relationship between each other subpath index and at least one assigned data conversion result, in the first mapping record;
[0176] Step 4: Set the first mapping record to the one intermediate node and the other nodes.
[0177] In one embodiment, the preset conversion method may be a mapping function or a hash algorithm, and the specific setting of the preset conversion method is not limited here.
[0178] In one embodiment, the network topology structure includes an intermediate node and other nodes, and the other nodes are nodes that have the same network distance from the intermediate node to the source node, such as: Figure 4 The access layer switches in the access layer switches are the access layer switches of the intermediate nodes and other access layer switches. For example, Figure 4 The aggregation layer switches in the aggregation layer are the aggregation layer switches of the intermediate nodes and other aggregation layer switches.
[0179] In one embodiment, based on a preset conversion method, the path identifier in the source node is input into the preset conversion method, the data conversion results of each path identifier are obtained, the association relationship between the path identifier and the data conversion result is obtained, and each data conversion result is evenly distributed to each sub-path index of the intermediate node, which is equivalent to evenly distributing each path identifier in the source node to each sub-path, and then equivalent to evenly distributing the data packets of each quintuple in the source node to each sub-path, so that each data packet received by the intermediate node can be evenly distributed to each sub-path for transmission, and the purpose of load balancing is achieved in the intermediate node. That is to say, in the source node, when determining the target transmission path according to the quintuple, after preliminary load balancing has been performed, after setting the first mapping record (path identifiers are evenly distributed to each sub-path), further load balancing is performed, and then load balancing is performed in each serially connected intermediate node in the target transmission path, so as to achieve maximum network traffic load balancing, effectively improve resource utilization, and improve service quality.
[0180] In one embodiment, each data conversion result is evenly distributed to each sub-path index of the intermediate node, that is, each path identifier is evenly distributed to each sub-path of the intermediate node, that is, each quintuple is evenly distributed to each sub-path of the intermediate node, which can further balance the network traffic load.
[0181] Based on the above methods, an embodiment of the present application provides a preset conversion method, wherein the preset conversion method is an XOR hash function, and the XOR hash function and hash seed in the one intermediate node are the same as those in the other nodes.
[0182] In one embodiment, the first mapping record is uniformly set in the intermediate node and other nodes with the same network distance from the source node, and the first mapping record only needs to be generated once, that is, based on a preset conversion method (the same in the intermediate node and other nodes), the data conversion results of each path identifier can be obtained, and then each data conversion result is assigned to each sub-path of the intermediate node, and each data conversion result is assigned to each other sub-path index of other nodes, and the first mapping record recording the association relationship between each sub-path index and at least one assigned data conversion result, and the association relationship between each other sub-path index and at least one assigned data conversion result is obtained. There is no need to generate the first mapping record for the intermediate node and other nodes respectively, which improves the efficiency of mapping record generation.
[0183] In one embodiment, if Fig. 22As shown, in the present application, the intermediate nodes and other nodes with the same network distance from the source node use a routing hash method based on XOR and use the same hash seed (hash key). In addition, the conversion method set in the nodes with different network distances from the source node can be the same as or different from the preset conversion method of the intermediate node, and there is no specific limitation here.
[0184] Based on the above Figure 8 The method flow in this application provides a data packet processing method, such as Fig.23 As shown, in step 8022, after sending the data packet to the target node through the at least one intermediate node, it also includes:
[0185] Step 2301: Based on the target node, determine the target data operation result of the five-tuple in the data packet, and determine the target thread index associated with the target data operation result in combination with a second mapping record preset in the target node, wherein the second mapping record includes: at least one data operation result associated with each thread index;
[0186] Step 2302: Send the data packet to the target processing thread corresponding to the target thread index, so that the target processing thread processes the data packet.
[0187] In one embodiment, the target node includes multiple processing threads, such as Fig.24 As shown, a schematic diagram of network traffic distribution to a target node provided by an embodiment of the present application is provided. The network card in the target node receives the data packet, and based on the target data operation result of the five-tuple in the data packet and in combination with the second mapping record, determines the target thread index associated with the target data operation result, and sends the data packet to the corresponding target processing thread for processing based on the virtual machine of the application. That is, the network card in the target node distributes the received network traffic to multiple CPU threads for shared processing (Receiver Side Scaling, RSS), then after receiving the data packet, the network card of the target node needs to determine the queue of the processing thread to which the data packet is allocated, so that the processing thread processes the data packet.
[0188] In one embodiment, the five-tuple of the data packet obtained by the target node is the five-tuple in the inner header of the data packet, and a preset operation method such as a hash algorithm or a mapping method can be used to obtain the target data operation result of the five-tuple, such as Fig.25 As shown, it is a schematic diagram of allocating network traffic to a target node provided in an embodiment of the present application.
[0189] In one embodiment, the five-tuple of the data packet obtained by the target node may also be the five-tuple in the outer header of the data packet, and a preset operation method such as an XOR hash algorithm or a mapping method may be used to obtain the target data operation result of the five-tuple, such as Fig.26 As shown, it is a schematic diagram of the configuration in a receiving node provided in an embodiment of the present application.
[0190] Based on the above Fig.23 The method flow in the embodiment of the present application provides a second mapping record generation method, wherein the network topology structure includes the target node and the receiving node, the receiving node is a node with the same network distance from the target node to the source node, and the target node and the receiving node are provided with a preset operation method;
[0191] In step 2301, before determining the target data operation result of the quintuple in the data packet, the method further includes:
[0192] Step 1: Based on the preset calculation method, data conversion is performed on each quintuple existing in the network topology structure to obtain data calculation results of each quintuple;
[0193] Step 2: Evenly distribute the data operation results to the thread indexes and receiving thread indexes, where the receiving thread is a processing thread in a receiving node;
[0194] Step 3: storing the association relationship between each thread index and at least one allocated data operation result, and the association relationship between each receiving thread index and at least one allocated data operation result, in the second mapping record;
[0195] Step 4: Set the second mapping record to the target node and the receiving node.
[0196] In one embodiment, the preset operation method may be a mapping function or a hash algorithm, and the specific setting of the preset conversion method is not limited here.
[0197] In one embodiment, the network topology structure includes a target node and a receiving node, and the receiving node is a node that has the same network distance from the target node to the source node, such as: Figure 4 The target node and other servers (receiving nodes) among multiple servers.
[0198] In one embodiment, based on a preset operation method, the path identifier in the source node is input into the preset operation method, the data operation results of each path identifier are obtained, the association relationship between the path identifier and the data operation result is obtained, and each data operation result is evenly distributed to each thread index of the target node, which is equivalent to evenly distributing each path identifier in the source node to each processing thread, and then equivalent to evenly distributing the data packets of each quintuple in the source node to each processing thread, so that each data packet received by the target node can be evenly distributed to each processing thread for processing, and the purpose of load balancing is achieved in each processing thread of the target node. That is to say, in the source node, when the target transmission path is determined according to the quintuple, after preliminary load balancing has been performed, after the first mapping record (path identifiers are evenly distributed to each sub-path) is set in the intermediate node, load balancing is further performed, and load balancing is performed in each serial intermediate node in the target transmission path. Finally, in each processing thread of the target node, load balancing is finally performed, and thread resources are evenly distributed to achieve maximum network traffic load balancing, effectively improve resource utilization, and improve service quality.
[0199] Based on the above embodiments, in one embodiment, the preset operation method is an XOR hash function, and the XOR hash function and the hash seed in the target node and the receiving node are the same.
[0200] In one embodiment, the second mapping record is uniformly set in the target node and the receiving node with the same network distance from the source node, and the second mapping record only needs to be generated once, that is, based on a preset operation method (the same in the target node and the receiving node), the data operation results of each path identifier can be obtained, and then each data operation result is assigned to each processing thread of the target node, and each data operation result is assigned to each receiving processing thread of the receiving node, and the second mapping record recording the association relationship between each thread index and at least one assigned data operation result, and the association relationship between each receiving thread index and at least one assigned data conversion result is obtained. There is no need to generate the second mapping record for the target node and the receiving node respectively, which improves the efficiency of mapping record generation.
[0201] In one embodiment, as in the hash algorithm in the source node, the target transmission path of the data packet can be obtained according to the five-tuple in the inner header of the data packet. On a macro level, each data stream of the source node can be evenly distributed to each transmission path. The hash algorithm in the target node can also obtain the target thread index of the data packet (the target data operation result of the five-tuple of the data packet is associated with the target thread index) according to the five-tuple in the inner header of the data packet. On a macro level, each data stream of the target node can be evenly distributed to each processing thread for processing. In this way, in the source node, when the target transmission path is determined according to the five-tuple, after preliminary load balancing has been performed, after setting the first mapping record (each sub-path is evenly distributed with the path identifier) at the intermediate node, further load balancing is performed, and load balancing is performed in each serially connected intermediate node in the target transmission path. Finally, load balancing is performed in each processing thread of the target node to achieve maximum network traffic load balancing, effectively improve resource utilization, and improve service quality. That is, the present application performs load balancing at each node of the data packet to achieve maximum load balancing.
[0202] In one embodiment, if Fig. 27 As shown in the figure, if the source node contains 64 source port numbers (path identifiers of the transmission path), based on Figure 4 In the application scenario, there are 8 aggregation layer switches, 2 access layer switches, and 32 processing threads in the target node. A greedy algorithm can be used to assign path identifiers to each switch and processing thread, so that each network group has 4 source port numbers, and these 4 source port numbers are distributed in different RSS groups (processing threads), and each RSS group has 2 port numbers.
[0203] include:
[0204] Step 1: Get 16 network groups based on the aggregation layer switches and access layer switches.
[0205] Step 2: Traverse the network groups and record the current network group as N, where N≤16 and is an integer.
[0206] Step 3: traverse the source port number of network packet N, record the current source port number as P, and traverse the RSS packet, record the current RSS packet as R.
[0207] If the following conditions are met, select P and mark R as used by N;
[0208] (a) The current port number P is in R;
[0209] (b) R is not used by N (the four source ports selected by N must be distributed in different RSS groups);
[0210] (c) The number of port numbers selected in N is not greater than 4 (64 / 16).
[0211] (d) The number of port numbers selected in R is not greater than 2 (64 / 32).
[0212] If the number of source port numbers obtained in the above embodiment is less than 64, condition b, c or d in the above condition may be appropriately relaxed to perform incremental selection until 64 source port numbers are selected.
[0213] In addition, for a data center network, the typical configuration is that each pod has 8 aggregation layer switches, each rack has 2 access layer switches, and the number of RSS group-processing threads is 32 or 16, 8, 4, etc. For 65535 source port numbers, each access layer switch has 8192 source port numbers, each access layer switch has 32768 source port numbers, and each RSS group has 2048 source port numbers. In fact, the selection of source port numbers (path identifiers) is very sufficient, and the above embodiment can basically complete the selection of source port numbers.
[0214] Based on the above-mentioned method processes and the embodiments involved, the embodiment of the present application provides a network load balancing method, which only needs to balance the upstream direction of the access layer switch (for the load balancing of each aggregation layer switch) and the downstream direction of the aggregation layer switch (for the load balancing of each access layer switch). The reason is that the route from the aggregation layer switch (upstream) to the peer aggregation layer switch (downstream) is highly deterministic, that is, Figure 3 As shown in the figure, the aggregation switch at a certain position in the same pod will only reach the aggregation switch at the same position in other pods through a group of access switches. For example, the traffic from S0 to Pod 1 will only reach S4 regardless of which core switch it passes through. At the same time, you don’t need to pay attention to the load balancing of the upstream of the aggregation switch, because there are many core switches, and the traffic bottleneck across Pods is often on the aggregation switch. Fig.28 As shown, the sending node assigns a path identifier to each transmission path, and uses a hash algorithm and a routing record to determine the target transmission path index based on the five-tuple in the inner header of the data packet, and balances the network traffic in each transmission path. After receiving the data packet, the intermediate node uses a preset conversion method and a first mapping record based on the target path identifier carried in the outer header of the data packet to determine the target sub-path index, and balances the network traffic in each sub-path. After receiving the data packet, the target node uses a preset calculation method and a second mapping record based on the five-tuple in the inner header of the data packet to balance the network traffic in each processing thread. Among them, the network load balancing method of the present application can be based on the data link layer application in the server, such as Fig.29 shown.
[0215] Based on the same concept, the embodiment of the present application provides a network load balancing device 3000, which is applicable to a load balancing system, such as Fig.30 As shown, including:
[0216] The first processing unit 3010 is used to determine a target transmission path of the data packet and its target path identifier based on a quintuple in the data packet generated by the source node, wherein the target transmission path connects the source node and the target node respectively, includes at least one intermediate node, and is formed by connecting a plurality of target sub-paths in series;
[0217] The second processing module 3020 is configured to send the data packet to the target node through the at least one intermediate node based on the target transmission path; wherein, during the transmission process, each of the intermediate nodes performs the following operations:
[0218] The path selection unit 3021 is used to determine the target sub-path index associated with the target data conversion result based on the target data conversion result identified by the target path, in combination with a first mapping record preset corresponding to an intermediate node, wherein the first mapping record includes: at least one data conversion result associated with each sub-path index, and each sub-path is a path between the intermediate node and the corresponding next-hop node;
[0219] The transceiver unit 3022 is configured to send the data packet to a next hop node based on the target sub-path corresponding to the target sub-path index.
[0220] Optionally, the network topology structure includes the one intermediate node and other nodes, the other nodes are nodes with the same network distance from the one intermediate node to the source node, and a preset conversion method is set in the one intermediate node and the other nodes;
[0221] The path selection unit 3021 is also used for:
[0222] Based on the preset conversion method, data conversion is performed on each path identifier in the source node to obtain a data conversion result of each path identifier;
[0223] Evenly distribute the data conversion results to the sub-path indexes and other sub-path indexes, where other sub-paths are paths between other nodes and corresponding next-hop nodes;
[0224] storing an association relationship between each subpath index and at least one assigned data conversion result, and an association relationship between each other subpath index and at least one assigned data conversion result, in the first mapping record;
[0225] The first mapping record is set in the one intermediate node and the other nodes.
[0226] Optionally, also include:
[0227] The third processing unit 3030 is used to determine the target data operation result of the five-tuple in the data packet based on the target node, and determine the target thread index associated with the target data operation result in combination with a second mapping record preset in the target node, wherein the second mapping record includes: at least one data operation result associated with each thread index;
[0228] The transceiver unit 3022 is further configured to send the data packet to a target processing thread corresponding to the target thread index, so that the target processing thread processes the data packet.
[0229] Optionally, the network topology structure includes the target node and the receiving node, the receiving node is a node with the same network distance from the target node to the source node, and a preset operation method is set in the target node and the receiving node;
[0230] The third processing unit 3030 is further configured to:
[0231] Based on the preset calculation method, data conversion is performed on each quintuple existing in the network topology structure to obtain data calculation results of each quintuple;
[0232] Evenly distribute the data operation results to the thread indexes and receiving thread indexes, where the receiving thread is a processing thread in a receiving node;
[0233] storing the association relationship between each thread index and at least one allocated data operation result, and the association relationship between each receiving thread index and at least one allocated data operation result, in the second mapping record;
[0234] The second mapping record is set in the target node and the receiving node.
[0235] Optionally, the first processing unit 3010 is specifically configured to:
[0236] Based on the quintuple in the data packet generated by the source node, a preset transformation method is adopted to obtain the target data transformation result corresponding to the quintuple, and according to the target transmission path index corresponding to the target data transformation result, the target path identifier associated with the target transmission path is determined, there are multiple transmission paths between the source node and the target node, and the transmission path index of each of the multiple transmission paths is associated with at least one path identifier.
[0237] Optionally, the first processing unit 3010 is specifically configured to:
[0238] Based on the routing record, the target transmission path index corresponding to the target data transformation result is obtained, and each transmission path index in the routing record is cyclically repeated multiple times according to a preset order, and each transmission path index with a new sorting has its own sorting number, which is the data transformation result.
[0239] Optionally, the first processing unit 3010 is further used to, if a node in the transmission path fails, determine a faulty transmission path index of the faulty transmission path of the faulty node, and replace the faulty transmission path index in the routing record with another transmission path index.
[0240] Optionally, the preset conversion method is an XOR hash function, and the XOR hash function and hash seed in the one intermediate node are the same as those in the other nodes.
[0241] Based on the same inventive concept as the above method embodiment, the present application embodiment also provides a computer device. In one embodiment, the computer device may be a server, such as Figure 4 , Figure 5 , Figure 6 and Figure 7 The server 410, server 510, server 610, and server 710 are shown. In this embodiment, the structure of the computer device can be as follows: Fig.31 As shown, it includes a memory 3101, a communication module 3103 and one or more processors 3102.
[0242] The memory 3101 is used to store computer programs executed by the processor 3102. The memory 3101 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.
[0243] The memory 3101 may be a volatile memory, such as a random-access memory (RAM); the memory 3101 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 3101 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 3101 may be a combination of the above memories.
[0244] The processor 3102 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 3102 is configured to implement the above network load balancing method when calling the computer program stored in the memory 3101 .
[0245] The communication module 3103 is used to communicate with terminal devices and other servers.
[0246] The specific connection medium between the memory 3101, the communication module 3103 and the processor 3102 is not limited in the embodiment of the present application. Fig.31 In the embodiment, the memory 3101 and the processor 3102 are connected via a bus 3104. The bus 3104 is Fig.31 The connections between the other components are only for illustration and are not intended to be limiting. The bus 3104 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Fig.31 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or only one type of bus.
[0247] The memory 3101 stores a computer storage medium, and the computer storage medium stores computer executable instructions, and the computer executable instructions are used to implement the network load balancing method of the embodiment of the present application. The processor 3102 is used to execute the above-mentioned network load balancing method, such as Figure 8 or Fig.13 or Fig.14 or Fig.23 shown.
[0248] In another embodiment, the computer device may also be other computer devices, such as Figure 7 The terminal device 750 shown in FIG. 1 is a terminal device 750 shown in FIG. 1 . In this embodiment, the structure of the computer device can be as follows: Fig.32As shown, it includes: a communication component 3210, a memory 3220, a display unit 3230, a camera 3240, a sensor 3250, an audio circuit 3260, a Bluetooth module 3270, a processor 3280 and other components.
[0249] The communication component 3210 is used to communicate with the server. In some embodiments, a circuit wireless fidelity (Wireless Fidelity, WiFi) module may be included. The WiFi module belongs to a short-range wireless transmission technology. The computer device can help the user to send and receive information through the WiFi module.
[0250] The memory 3220 can be used to store software programs and data. The processor 3280 executes various functions and data processing of the terminal device 750 by running the software programs or data stored in the memory 3220. The memory 3220 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The memory 3220 stores an operating system that enables the terminal device 750 to run. In the present application, the memory 3220 can store an operating system and various application programs, and can also store a computer program for executing the network load balancing method in the embodiment of the present application.
[0251] The display unit 3230 can also be used to display information input by the user or information provided to the user and a graphical user interface (GUI) of various menus of the terminal device 750. Specifically, the display unit 3230 may include a display screen 3232 disposed on the front of the terminal device 750. The display screen 3232 may be configured in the form of a liquid crystal display, a light emitting diode, etc. The display unit 3230 may be used to display the network load balancing user interface in the embodiment of the present application, etc.
[0252] The display unit 3230 can also be used to receive input digital or character information, and generate signal input related to user settings and function control of the terminal device 750. Specifically, the display unit 3230 may include a touch screen 3231 arranged on the front of the terminal device 750, which can collect user touch operations on or near it, such as clicking a button, dragging a scroll box, etc.
[0253] The touch screen 3231 can be covered on the display screen 3232, or the touch screen 3231 and the display screen 3232 can be integrated to realize the input and output functions of the terminal device 750, and the integrated touch screen can be referred to as a touch display screen. In this application, the display unit 3230 can display the application and the corresponding operation steps.
[0254] Camera 3240 can be used to capture still images, and users can post comments on images taken by camera 3240 through the application. Camera 3240 can be one or more. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the processor 3280 to convert it into a digital image signal.
[0255] The terminal device may further include at least one sensor 3250, such as an acceleration sensor 3251, a distance sensor 3252, a fingerprint sensor 3253, and a temperature sensor 3254. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.
[0256] The audio circuit 3260, the speaker 3261, and the microphone 3262 can provide an audio interface between the user and the terminal device 750. The audio circuit 3260 can transmit the electrical signal converted from the received audio data to the speaker 3261, which is converted into a sound signal for output. The terminal device 750 can also be configured with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 3262 converts the collected sound signal into an electrical signal, which is received by the audio circuit 3260 and converted into audio data, and then the audio data is output to the communication component 3210 to be sent to, for example, another terminal device 750, or the audio data is output to the memory 3220 for further processing.
[0257] The Bluetooth module 3270 is used to exchange information with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the terminal device can establish a Bluetooth connection with a wearable computer device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 3270 to exchange data.
[0258] The processor 3280 is the control center of the terminal device. It uses various interfaces and lines to connect various parts of the entire terminal. It executes various functions of the terminal device and processes data by running or executing software programs stored in the memory 3220 and calling data stored in the memory 3220. In some embodiments, the processor 3280 may include one or more processing units; the processor 3280 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes an operating system, a user interface, and an application program, and the baseband processor mainly processes wireless communications. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 3280. In the present application, the processor 3280 can run an operating system, an application program, a user interface display and a touch response, as well as a network load balancing method of an embodiment of the present application. In addition, the processor 3280 is coupled to the display unit 3230.
[0259] In some possible implementations, various aspects of the network load balancing method provided by the present application may also be implemented in the form of a program product, which includes a computer program. When the program product is run on a computer device, the computer program is used to enable the computer device to execute the steps of the network load balancing method according to various exemplary embodiments of the present application described above in this specification. For example, the computer device may execute the following steps: Figure 8 or Fig.13 or Fig.14 or Fig.23 Follow the steps shown in .
[0260] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0261] The program product of the embodiment of the present application may adopt a portable compact disk read-only memory (CD-ROM) and include a computer program, and can be run on a computer device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with a command execution system, apparatus, or device.
[0262] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.
[0263] The computer program embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0264] The computer program for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The computer program may be executed entirely on the user's computer device, partially on the user's device, as a separate software package, partially on the user's computer device and partially on a remote computer device, or entirely on a remote computer device or server. In the case of a remote computer device, the remote computer device may be connected to the user's computer device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer device (e.g., via the Internet using an Internet service provider).
[0265] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.
[0266] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.
[0267] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0268] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0269] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0270] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0271] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A network load balancing method, characterized in that: Applicable to a load balancing system, the method includes: Determine, based on a quintuple in a data packet generated by a source node, a target transmission path for the data packet and its target path identifier, wherein the target transmission path connects the source node and the target node, includes at least one intermediate node, and is formed by a plurality of target sub-paths connected in series; Based on the target transmission path, the data packet is sent to the target node through the at least one intermediate node; wherein, during the transmission process, each of the intermediate nodes performs the following operations: Determining, based on the target data conversion result identified by the target path and in combination with a first mapping record preset corresponding to an intermediate node, a target sub-path index associated with the target data conversion result, wherein the first mapping record includes: at least one data conversion result associated with each sub-path index, and each sub-path is a path between the intermediate node and a corresponding next-hop node; Based on the target subpath corresponding to the target subpath index, the data packet is sent to the next hop node.
2. The method according to claim 1, wherein The network topology structure includes the intermediate node and other nodes, the other nodes are nodes at the same network distance from the source node as the intermediate node, and the intermediate node and the other nodes are provided with a preset conversion method; Before determining the target subpath index associated with the target data conversion result, the method further includes: Based on the preset conversion method, data conversion is performed on each path identifier in the source node to obtain a data conversion result for each path identifier; Evenly distributing the data conversion results to the sub-path indexes and other sub-path indexes, where other sub-paths are paths between other nodes and corresponding next-hop nodes; storing an association between each subpath index and at least one assigned data conversion result, and an association between each other subpath index and at least one assigned data conversion result, in the first mapping record; The first mapping record is set in the one intermediate node and the other nodes.
3. The method according to claim 1, wherein After sending the data packet to the target node through the at least one intermediate node, the method further includes: Determining, based on the target node, a target data operation result of the quintuple in the data packet, and determining, in combination with a second mapping record preset in the target node, a target thread index associated with the target data operation result, the second mapping record including: at least one data operation result associated with each thread index; The data packet is sent to a target processing thread corresponding to the target thread index, so that the target processing thread processes the data packet.
4. The method according to claim 3, wherein The network topology structure includes the target node and a receiving node, the receiving node is a node at the same network distance from the target node to the source node, and the target node and the receiving node are provided with a preset operation method; Before determining the target data operation result of the quintuple in the data packet, the method further includes: Based on the preset calculation method, data conversion is performed on each quintuple in the network topology structure to obtain a data calculation result of each quintuple; Evenly distributing the data operation results to the thread indexes and the receiving thread indexes, where the receiving thread is a processing thread in the receiving node; storing an association between each thread index and at least one allocated data operation result, and an association between each receiving thread index and at least one allocated data operation result, in the second mapping record; The second mapping record is set in the target node and the receiving node.
5. The method according to claim 1, wherein The determining, based on the quintuple in the data packet generated by the source node, a target transmission path of the data packet and its target path identifier, includes: Based on the quintuple in the data packet generated by the source node, a preset transformation method is used to obtain the target data transformation result corresponding to the quintuple, and according to the target transmission path index corresponding to the target data transformation result, the target path identifier associated with the target transmission path is determined. There are multiple transmission paths between the source node and the target node, and the transmission path index of each of the multiple transmission paths is associated with at least one path identifier.
6. The method according to claim 5, wherein The target transmission path index corresponding to the target data transformation result includes: Based on the routing record, the target transmission path index corresponding to the target data transformation result is obtained, and each transmission path index in the routing record is repeated multiple times in a preset order, and each transmission path index with a new sort has its own sort number, which is the data transformation result.
7. The method according to claim 6, wherein If a node in a transmission path fails, a faulty transmission path index of the faulty transmission path of the faulty node is determined, and the faulty transmission path index in the routing record is replaced with another transmission path index.
8. The method according to claim 2, wherein The preset conversion method is an XOR hash function, and the XOR hash function and hash seed in the one intermediate node are the same as those in the other nodes.
9. A network load balancing device, characterized in that: Applicable to a load balancing system, the device includes: a first processing unit, configured to determine, based on a quintuple in a data packet generated by a source node, a target transmission path of the data packet and a target path identifier thereof, wherein the target transmission path connects the source node and the destination node respectively, includes at least one intermediate node, and is formed by a plurality of target sub-paths connected in series; A second processing module is configured to send the data packet to the target node through the at least one intermediate node based on the target transmission path; wherein, during the transmission process, each of the intermediate nodes performs the following operations: a path selection unit, configured to determine, based on the target data conversion result identified by the target path and in combination with a first mapping record preset corresponding to an intermediate node, a target sub-path index associated with the target data conversion result, wherein the first mapping record includes: at least one data conversion result associated with each sub-path index, and each sub-path is a path between the intermediate node and a corresponding next-hop node; The transceiver unit is configured to send the data packet to a next hop node based on the target subpath corresponding to the target subpath index.
10. The device according to claim 9, wherein The network topology structure includes the intermediate node and other nodes, the other nodes are nodes at the same network distance from the source node as the intermediate node, and the intermediate node and the other nodes are provided with a preset conversion method; The path selection unit is further configured to: Based on the preset conversion method, data conversion is performed on each path identifier in the source node to obtain a data conversion result for each path identifier; Evenly distributing the data conversion results to the sub-path indexes and other sub-path indexes, where other sub-paths are paths between other nodes and corresponding next-hop nodes; storing an association between each subpath index and at least one assigned data conversion result, and an association between each other subpath index and at least one assigned data conversion result, in the first mapping record; The first mapping record is set in the one intermediate node and the other nodes.
11. The device according to claim 9, wherein Also includes: a third processing unit, configured to determine, based on the target node, a target data operation result of the quintuple in the data packet, and determine a target thread index associated with the target data operation result in combination with a second mapping record preset in the target node, wherein the second mapping record includes at least one data operation result associated with each thread index; The transceiver unit is further configured to send the data packet to a target processing thread corresponding to the target thread index, so that the target processing thread processes the data packet.
12. The device according to claim 11, wherein The network topology structure includes the target node and a receiving node, the receiving node is a node at the same network distance from the target node to the source node, and the target node and the receiving node are provided with a preset operation method; The third processing unit is further configured to: Based on the preset calculation method, data conversion is performed on each quintuple in the network topology structure to obtain a data calculation result of each quintuple; Evenly distributing the data operation results to the thread indexes and the receiving thread indexes, where the receiving thread is a processing thread in the receiving node; storing an association between each thread index and at least one allocated data operation result, and an association between each receiving thread index and at least one allocated data operation result, in the second mapping record; The second mapping record is set in the target node and the receiving node.
13. A computer-readable non-volatile storage medium, characterized in that: The computer-readable non-volatile storage medium stores a program, and when the program is run on a computer, the computer is enabled to implement the method according to any one of claims 1 to 8.
14. A computer device, characterized in that: include: Memory for storing computer programs; A processor is configured to call a computer program stored in the memory and execute the method according to any one of claims 1 to 8 according to the obtained program.
15. A computer program product, characterized in that The method comprises a computer program stored in a computer-readable storage medium; when a processor of a computer device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the computer device performs the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Method for forwarding messages based on equal cost multi-path routing and network devices
CN106559324A
Load balancing method and device for network equipment
CN110430143A
Flow equalization method and device
CN111147381A
Load sharing method, device and equipment in BIER (bit indexed explicit replication) network
CN112787934A
Load balancing method, device and system
CN114024969A
Cited By
Message processing method and network equipment
CN119544586A
