Task-level load balancing method for data transmission system

Abstract

The invention relates to a task-level load balancing method for a data transmission system. The method comprises the following steps that a receiving end monitors the software state, the task number, the CPU occupancy rate, the memory occupancy rate, the disk utilization rate and the network bandwidth occupancy rate of a resource pool server in real time; the receiving end calculates the comprehensive service capability of each server based on a preset algorithm, and generates an IP sequence of the server; and the sending end performs data transmission according to the priority of the IP sequence, and automatically switches to the server corresponding to the next IP when the transmission fails. According to the invention, the software and hardware capability and task condition of the server in the resource pool can be comprehensively evaluated, and the real-time balancing result is transmitted through transmission interaction between the sending end and the receiving end, so that task-level load balancing is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and particularly to a task-level load balancing method for a data transmission system. Background Art

[0002] In the current rapid development of modern communication technologies, the field of ground satellite data transmission plays a crucial role in many industries, such as meteorological monitoring, geographical mapping, military defense, and radio and television signal transmission. With the explosive growth of data volume and the increasing complexity of business requirements, the efficiency, stability, and reliability of data transmission have become key indicators for the development of this field, and load balancing technology is the core support to achieve these goals.

[0003] In the early days, the scale of ground satellite data transmission systems was small, the data volume was relatively limited, and the load balancing requirements were not prominent. A single server or a simple server cluster could meet the basic business requirements. However, with the progress of satellite technology, high-resolution satellites have emerged continuously, and the data volume collected by satellites has increased exponentially. Taking meteorological satellites as an example, the data volume generated by each observation has increased from the GB level in the past to the TB level today. The real-time transmission and processing of massive data pose a huge challenge to ground data transmission systems.

[0004] To address this challenge, an architecture consisting of multiple servers and a network load balancing controller has emerged. Load balancing is achieved by providing a virtual IP externally. This architecture has achieved certain results at the network level. It can evenly distribute network traffic to each server, avoiding performance bottlenecks caused by excessive traffic on a single server, and significantly improving the concurrent processing ability of the system. In some early radio and television signal transmission systems, through this method, stable signal transmission services can be provided to a large number of users simultaneously, ensuring the basic operation of the business.

[0005] However, with the continuous expansion and deepening of application scenarios, this method of only performing load balancing at the network level has gradually revealed many drawbacks. In actual data transmission tasks, the service performance differences of servers have a crucial impact on transmission efficiency and quality. When the network load balancing controller distributes tasks, it often only relies on network traffic indicators and ignores the software and hardware status and task load of the servers themselves.

[0006] Therefore, designing a task-level load balancing method for a data transmission system to achieve task-level load balancing is an urgent problem to be solved. Summary of the Invention

[0007] To solve the technical problems existing in the above-mentioned prior art, the object of the present invention is to provide a task-level load balancing method for a data transmission system, which can comprehensively evaluate the software and hardware capabilities and task conditions of the servers in the resource pool, and transmit the real-time balancing result through the transmission interaction between the sender and the receiver, so as to achieve task-level load balancing.

[0008] To achieve the above object of the invention, the present invention provides a task-level load balancing method for a data transmission system, including the following steps:

[0009] Step S1: The receiver monitors in real time the software status, task quantity, CPU occupancy rate, memory occupancy rate, disk usage rate, and network bandwidth occupancy rate of the resource pool servers.

[0010] Step S2: The receiver calculates the comprehensive service capabilities of each server based on a preset algorithm and generates an IP sequence of the servers.

[0011] Step S3: The sender performs data transmission according to the priority of the IP sequence, and automatically switches to the server corresponding to the next IP when the transmission fails.

[0012] According to a technical solution of the present invention, the task-level load balancing method for a data transmission system further includes:

[0013] Step S4: If the software status of all servers is abnormal, an error code is returned to the sender and the current transmission process is terminated.

[0014] According to a technical solution of the present invention, before the step S1, it further includes:

[0015] The receiver registers the servers in the resource pool, assigns a unique resource ID to each server, and configures the service priority.

[0016] According to a technical solution of the present invention, the step S2 includes:

[0017] The sender initiates a data transmission application to the receiver;

[0018] The receiver responds to the data transmission application, filters out the servers with normal software status, and calculates the comprehensive service capabilities of the filtered servers based on a preset algorithm;

[0019] The receiver sorts the servers from high to low according to the comprehensive service capabilities to generate an IP sequence and feeds it back to the sender.

[0020] According to a technical solution of the present invention, in the step S3, the sender preferentially selects the server ranked at the top of the IP sequence for data transmission;

[0021] If the transmission fails, it will automatically switch to the next server in the sequence until the transmission is successful.

[0022] According to one technical solution of the present invention, the preset algorithm is at least one of weighted, polling, and custom ratio algorithms.

[0023] According to one technical solution of the present invention, the preset algorithm is expressed as:

[0024] Comprehensive service capacity = α*(1 - a) + β*(1 - b) + γ*(1 - c) + δ*(1 - d) + ε*(1 - e) + ζ*f + η*g;

[0025] Wherein, a is the proportion of the number of server tasks to the total tasks, b is the CPU occupancy rate, c is the memory occupancy rate, d is the disk occupancy rate, e is the network bandwidth occupancy rate, f is the resource ID, g is the proportion of service priority, α + β + γ + δ + ε + ζ + η = 1, and the values of ζ and η are 0 or 1.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] A task-level load balancing method for a data transmission system according to the present invention monitors the software status, task quantity, server CPU occupancy rate, memory occupancy rate, disk usage rate, and network bandwidth occupancy rate of servers in the resource pool in real time, and calculates the comprehensive service capacity of the servers by weighted calculation; the balance result is transmitted through the transmission application and response interaction between the sender and the receiver. The sender restricts the sending behavior, preferentially transmits to the IP with a higher sequence number, and automatically switches to the next IP when transmission jitters occur, realizing the fault tolerance function of load balancing. Using the performance indicators concerned by tasks to evaluate the server capabilities, compared with common network load balancing controllers, the evaluation system is more reasonable and has a higher satisfaction with actual tasks.

[0028] In the present invention, by adopting the transmission application-response interaction mechanism between the sender and the receiver, the dynamic transmission of the load balancing result is realized. The sender actively triggers the resource evaluation process of the receiver before each data transmission, and the receiver generates a real-time IP sequence according to the latest server status and feeds it back to ensure that the load distribution always matches the current environment.

[0029] In the present invention, by introducing weighted, polling, and custom ratio algorithms, the evaluation weights can be flexibly adjusted according to different service types. Brief Description of the Drawings

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the accompanying drawings required for the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0031] Figure 1 Schematically showing a flowchart of a task-level load balancing method for a data transmission system according to an embodiment of the present invention;

[0032] Figure 2 Schematically showing a calculation example of the polling algorithm according to the present invention;

[0033] Figure 3 Schematically showing a calculation example of the weighted algorithm according to the present invention;

[0034] Figure 4 Schematically showing a calculation example of the custom ratio algorithm according to the present invention. Detailed implementation manners

[0035] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0036] As Figure 1 shown, in a task-level load balancing method for a data transmission system of the present invention, the load balancing evaluates the server capabilities using the performance indicators concerned by the tasks. Compared with the common network load balancing controllers, the evaluation system is more reasonable and has a higher satisfaction degree for actual tasks.

[0037] A task-level load balancing method for a data transmission system of the present invention includes the following steps:

[0038] Step S1: The receiving end monitors the software status, task quantity, CPU occupancy rate, memory occupancy rate, disk usage rate, and network bandwidth occupancy rate of the resource pool server in real time;

[0039] Step S2: The receiving end calculates the comprehensive service capabilities of each server based on a preset algorithm and generates an IP sequence of the servers, specifically including:

[0040] The sending end initiates a data transmission request to the receiving end;

[0041] In response to the data transmission application, the receiving end filters out servers with normal software status and calculates the comprehensive service capabilities of the filtered servers based on a preset algorithm;

[0042] Among them, the proportion a of the server task quantity to the total tasks is statistically analyzed. The smaller the task quantity proportion, the stronger the service capability;

[0043] The lower the CPU occupancy rate b, the stronger the service capability;

[0044] The lower the memory occupancy rate c, the stronger the service capability;

[0045] The lower the disk occupancy rate d, the stronger the service capability;

[0046] The lower the network bandwidth occupancy rate e, the stronger the service capability;

[0047] The larger the resource ID f, the higher the priority of service allocation;

[0048] The larger the service priority proportion g, the higher the priority of service allocation;

[0049] The receiving end sorts the servers from high to low according to the comprehensive service capabilities to generate an IP sequence and feedbacks it to the sending end.

[0050] The data transmission application triggers the receiving end to customize a policy in combination with the business and the performance focus of the resource pool, and performs weighted calculation according to the real-time monitored software and hardware status of the resource pool devices to obtain a receiving end resource balance result that matches the business. This balance result is the IP sequence of the server.

[0051] Step S3: The sending end performs data transmission according to the priority of the IP sequence. When the transmission fails, it automatically switches to the server corresponding to the next IP. Specifically, it includes:

[0052] The sending end preferentially selects the server ranked at the top of the IP sequence for data transmission;

[0053] If the transmission fails, it automatically switches to the next server in the sequence until the transmission is successful.

[0054] Whenever a new sending end needs to perform data transmission, steps S1 and S2 are repeated. Due to the changes in the status of the CPU, memory, hard disk, etc. caused by the task quantity and task transmission, the load balancing will obtain a new set of IP sequences, thereby realizing the load balancing of the receiving end servers.

[0055] When the sending end encounters a sending failure, there is no need to apply again. It actively switches the IP in the sequence and transmits to the server with the second-ranked IP, realizing the fault tolerance function of load balancing

[0056] Step S4: If the software status of all servers is abnormal, return an error code to the sending end and terminate the current transmission process.

[0057] In some embodiments of the present invention, before the step S1, it further includes:

[0058] The receiving end registers the servers in the resource pool, assigns a unique resource ID to each server, and configures the service priority.

[0059] In some embodiments of the present invention, the preset algorithm is at least one of weighted, round-robin, and custom ratio algorithms.

[0060] In some embodiments of the present invention, the preset algorithm is expressed as:

[0061] Comprehensive service ability = α*(1 - a) + β*(1 - b) + γ*(1 - c) + δ*(1 - d) + ε*(1 - e) + ζ*f + η*g;

[0062] Where a is the proportion of the number of server tasks to the total tasks, α is the corresponding weight; b is the CPU occupancy rate, β is the corresponding weight; c is the memory occupancy rate, γ is the corresponding weight; d is the disk occupancy rate, δ is the corresponding weight; e is the network bandwidth occupancy rate, ε is the corresponding weight; f is the resource ID, ζ is the corresponding weight; g is the proportion of service priority, η is the corresponding weight; α + β + γ + δ + ε + ζ + η = 1, and the values of ζ and η are 0 or 1.

[0063] The above preset algorithm includes weighted, round-robin, and custom ratio algorithms. As Figure 2 shown, it is an example of round-robin algorithm calculation. Among them, except ζ = 1, the other parameters are all 0, then the IP sequence is Server 1, Server 2, and Server 3; as Figure 3 shown, it is an example of weighted algorithm calculation. α, β, γ, δ, and ε are all 0.2, then the IP sequence is Server 2, Server 3, and Server 1; as Figure 4 shown, it is an example of custom ratio algorithm calculation. Except η = 1, the other parameters are all 0. At the same time, the priority of each server is set, then the IP sequence is Server 1, Server 2, and Server 3.

[0064] For a task-level load balancing method of a data transmission system of the present invention, when the sending end needs to perform data transmission, it sends a transmission application to the receiving end. The transmission application triggers the receiving end to customize a strategy in combination with business and resource pool performance emphasis, performs weighted calculation according to the real-time monitored software and hardware status of the resource pool devices, obtains a sequence of resource balance results of the receiving end that matches the business, and the receiving end feeds back this sequence to the sending end while responding to the application. The sending end performs transmission to the specific servers in the resource pool according to the result sequence, thereby realizing task-level load balancing.

[0065] It should be noted that although the embodiments described above of the present invention are illustrative, they are not limitations of the present invention. Therefore, the present invention is not limited to the above specific embodiments. Without departing from the principle of the present invention, any other embodiments obtained by those skilled in the art under the inspiration of the present invention are deemed to be within the protection scope of the present invention.

Claims

1. A task-level load balancing method for a data transmission system, characterized in that, It includes the following steps: Step S1: The receiving end monitors the software status, task quantity, CPU occupancy rate, memory occupancy rate, disk usage rate, and network bandwidth occupancy rate of the resource pool server in real time; Step S2: The receiving end calculates the comprehensive service capabilities of each server based on a preset algorithm and generates an IP sequence of the servers; Step S3: The sending end performs data transmission according to the priority of the IP sequence, and automatically switches to the server corresponding to the next IP when the transmission fails.

2. The task-level load balancing method for the data transmission system according to claim 1, wherein It also includes: Step S4: If the software status of all servers is abnormal, an error code is returned to the sending end and the current transmission process is terminated.

3. The task-level load balancing method for a data transmission system according to claim 1, characterized in that Before the step S1, it also includes: The receiving end registers the servers in the resource pool, assigns a unique resource ID to each server, and configures the service priority.

4. The task-level load balancing method for the data transmission system according to claim 1, characterized in that The step S2 includes: The sending end initiates a data transmission application to the receiving end; In response to the data transmission application, the receiving end filters out the servers with normal software status and calculates the comprehensive service capabilities of the filtered servers based on a preset algorithm; The receiving end sorts the servers from high to low according to the comprehensive service capabilities to generate an IP sequence and feeds it back to the sending end.

5. The task-level load balancing method for a data transmission system according to claim 1, wherein In the step S3, the sending end preferentially selects the server ranked at the top of the IP sequence for data transmission; If the transmission fails, it automatically switches to the next server in the sequence until the transmission is successful.

6. The task-level load balancing method for a data transmission system according to claim 1, characterized in that The preset algorithm is at least one of the weighted, round-robin, and custom ratio algorithms.

7. The task-level load balancing method for the data transmission system according to claim 1, wherein The preset algorithm is expressed as: Comprehensive service capability = α*(1 - a) + β*(1 - b) + γ*(1 - c) + δ*(1 - d) + ε*(1 - e) + ζ*f + η*g; Where, a is the ratio of the server task quantity to the total tasks, b is the CPU occupancy rate, c is the memory occupancy rate, d is the disk occupancy rate, e is the network bandwidth occupancy rate, f is the resource ID, g is the service priority ratio, α + β + γ + δ + ε + ζ + η = 1, and the values of ζ and η are 0 or 1.

Images