A load balancing method, system and storage medium

Abstract

The present application discloses a load balancing method, system and storage medium. The method includes: passing the load balancing parameters configured at the Octavia front end to the configuration file of Nginx through the Amphora agent, and restarting and monitoring the status of Nginx; after receiving a user request, using the full-user-space F-stack to directly transmit the request data packet from the network card to the Nginx in the user space; running Nginx in multiple processes to concurrently receive and distribute the request data packets to the resource pool members providing services, so as to perform load balancing and response of request messages. In this way, the resources and time consumed by packet forwarding can be reduced, the quality and efficiency of the load balancing service can be significantly improved, the utilization rate of computing resources in the cloud environment and the response efficiency of requests in the cloud environment can be greatly enhanced, and the high-concurrency request requirements of Internet users can be met.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud computing, and particularly to a load balancing method, system and storage medium. Background Art

[0002] Today, with the rapid development of the Internet, various intelligent products implemented through the cloud have entered our daily lives. Although cloud computing technology has been rapidly adopted, there are still long-term deficiencies in the ability to reliably distribute workloads across multi-cloud platforms, multiple data centers, and hybrid infrastructures. As a result, the workload is unevenly distributed and the performance of applications deteriorates, which could be avoided if workloads were better managed globally. Currently, Octavia (a load balancing project) in OpenStack (an open-source cloud computing infrastructure project) uses HAProxy (a load balancing software) based on the kernel space protocol stack. In the traditional kernel protocol stack, there are many bottlenecks in network packet processing, seriously affecting the performance of network packet reception and transmission.

[0003] The performance bottlenecks mainly include three aspects: The first is locality failure. The processing of a data packet may span multiple Central Processing Unit (CPU) cores, cache misses, and be Non-Uniform Memory Access Architecture (NUMA)-unfriendly. A data packet may be interrupted on CPU0, processed in kernel mode on CPU1, and processed in user mode on CPU2, thus spanning multiple cores, causing locality failure and CPU cache misses, and there may also be cross-NUMA memory access, greatly affecting performance. The second is interrupt processing. When there is a large amount of data in the network, a large number of data packets generate frequent hardware interrupt requests. These hardware interrupts can interrupt the execution process of previously lower-priority soft interrupts or system calls. If such interruptions occur frequently, it will result in a high performance overhead. Context switches between user mode and kernel mode and soft interrupts all add additional overhead. The third is memory copy. The memory copy between kernel mode and user mode for network data packets from the network card to the application program needs to go through the following process: The data is transferred from the network card to the buffer opened by the kernel through Direct Memory Access (DMA) and other methods; the data is copied from the kernel space to the user space. In the Linux kernel protocol stack, this time-consuming even accounts for half of the entire data packet processing flow.

[0004] Therefore, how to solve the performance bottleneck problem of OpenStack load balancing is a technical problem that needs to be urgently solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the object of the present invention is to provide a load balancing method, system and storage medium, which can reduce the resources and time consumed by packet forwarding, and significantly improve the quality and efficiency of load balancing services. The specific scheme is as follows:

[0006] A load balancing method for an OpenStack cloud environment, comprising:

[0007] Transfer the load balancing parameters configured by Octavia front-end to the configuration file of Nginx through the Amphora agent, and restart and monitor the status of Nginx;

[0008] After receiving the user's request, use the full-user-space F-stack to directly transfer the request packet from the network card to the user-space Nginx;

[0009] Run Nginx in multiple processes, concurrently receive and distribute the request packets to the members of the resource pool providing services, so as to perform load balancing and response of request messages.

[0010] Preferably, in the above load balancing method provided by the embodiment of the present invention, the transferring the load balancing parameters configured by Octavia front-end to the configuration file of Nginx includes:

[0011] After configuring load balancing at the Octavia front-end, obtain the load balancing parameters configured by the front-end;

[0012] Extract and convert the obtained load balancing parameters;

[0013] Write and save the converted load balancing parameters to the pre-backup configuration file according to the Nginx configuration file format.

[0014] Preferably, in the above load balancing method provided by the embodiment of the present invention, before restarting and monitoring the status of Nginx, it further includes:

[0015] Use the nginx–t command to check whether the configuration file format is correct;

[0016] If so, restart and monitor the status of Nginx;

[0017] If not, return a configuration exception error, perform a rollback operation on the configuration file, and restart Nginx.

[0018] Preferably, in the above load balancing method provided by the embodiment of the present invention, the monitoring of the Nginx status includes:

[0019] Monitor whether the Nginx service is started normally;

[0020] If the startup is normal, the configuration distribution is completed;

[0021] If the startup is abnormal, a configuration exception error is returned, the configuration file is rolled back, and nginx is restarted.

[0022] Preferably, in the above load balancing method provided by the embodiments of the present invention, the concurrent receiving and distributing the request data packets to the resource pool members providing services for load balancing and response of the request messages includes:

[0023] Using nginx to select resource pool members for load balancing according to the load balancing rules, modifying the content of the concurrently received request data packets and transmitting them to the network card via F-stack;

[0024] According to the header information of the request data packet, distributing the request data packet to the resource pool members providing services through the network card;

[0025] Responding to the request content through the resource pool members and returning a response data packet;

[0026] After the returned response data packet is sent to amphora, it is returned to the user after destination address conversion, completing the load balancing and response of the request message.

[0027] Preferably, in the above load balancing method provided by the embodiments of the present invention, each of the processes is bound to an independent network card queue and CPU and uses independent memory.

[0028] The embodiments of the present invention also provide a load balancing system for the openstack cloud environment, including: Amphoraagent, the fully user-space F-stack, and nginx;

[0029] Amphora agent is used to transfer the load balancing parameters configured by Octavia in the front end to the configuration file of nginx and restart and monitor the status of nginx;

[0030] F-stack is used to directly transfer the request data packet from the network card to the user-space nginx after receiving the user's request;

[0031] nginx is used to concurrently receive and distribute the request data packets to the resource pool members providing services under multi-process operation for load balancing and response of the request messages.

[0032] Preferably, in the above load balancing system provided by the embodiments of the present invention, the Amphora agent is specifically configured to obtain the load balancing parameters configured at the front end after configuring the load balancing at the Octavia front end; extract and convert the obtained load balancing parameters; and write and save the converted load balancing parameters to the pre-backup configuration file in accordance with the configuration file format of nginx.

[0033] Preferably, in the above load balancing system provided by the embodiments of the present invention, nginx is specifically configured to select resource pool members for load balancing according to the load balancing rules, modify the content of the concurrently received request data packets and then transmit them to the network card via F-stack; distribute the request data packets to the resource pool members providing services through the network card according to the header information of the request data packets; respond to the request content through the resource pool members and return the response data packets; after the returned response data packets are sent to Amphora, they are returned to the user after destination address conversion to complete the load balancing and response of the request message.

[0034] The embodiments of the present invention further provide a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above load balancing method provided by the embodiments of the present invention.

[0035] As can be seen from the above technical solutions, a load balancing method provided by the present invention includes: passing the load balancing parameters configured at the Octavia front end to the configuration file of nginx through the Amphora agent, and restarting and monitoring the status of nginx; after receiving a user request, directly transmitting the request data packet from the network card to the nginx in the user space by using the full user space F-stack; running nginx in multiple processes to concurrently receive and distribute the request data packets to the resource pool members providing services to perform load balancing and response of the request message.

[0036] In the cloud environment based on OpenStack, the present invention utilizes the F-stack architecture and Nginx to implement full-user-mode load balancing, providing high-performance load balancing services for the OpenStack environment. By migrating packet processing to the user space, efficient processing of network packets can be achieved, reducing the resources and time consumed by packet forwarding, minimizing the memory copy from the kernel mode to the user mode, solving the problem of locality failure, eliminating the performance impact of interrupt processing, significantly improving the quality and efficiency of load balancing services, greatly enhancing the utilization rate of computing resources in the cloud environment and the response efficiency of requests in the cloud environment, and meeting the high-concurrency request needs of Internet users. In addition, the present invention also provides a corresponding system and computer-readable storage medium for the load balancing method, further making the above method more practical, and the system and computer-readable storage medium have corresponding advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or in the related art, the following will briefly introduce the drawings required for use in the description of the embodiments or the related art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0038] Figure 1 It is a flowchart of the load balancing method provided by the embodiment of the present invention;

[0039] Figure 2 It is a logical view of the load balancing method provided by the embodiment of the present invention;

[0040] Figure 3 It is a configuration conversion flowchart in the load balancing method provided by the embodiment of the present invention;

[0041] Figure 4 It is a load balancing flowchart in the load balancing method provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0043] The present invention provides a load balancing method for the OpenStack cloud environment, as Figure 1 and Figure 2 shown, which includes the following steps:

[0044] S101. Use the Amphora agent to transfer the load balancing parameters configured by Octavia front - end to the nginx configuration file, and restart and monitor the status of nginx;

[0045] It should be noted that Octavia is an open - source, carrier - scale load - balancing solution designed to work with OpenStack. Amphora is a component in the Octavia project, the load - balancing function execution unit. Amphora agent is a client running on the Amphora virtual machine, responsible for receiving and executing the instructions issued by Octavia worker. Octavia worker is a component in the Octavia project, which executes corresponding operations according to the received operation requests. Nginx is a high - performance HyperText Transfer Protocol (HTTP) and reverse proxy web server, and also provides Internet Mail Access Protocol (IMAP) / Post Office Protocol - Version 3 (POP3) / Simple Mail Transfer Protocol (SMTP) services.

[0046] S102. After receiving the user's request, use the full - user - space F - stack to directly transfer the request data packet from the network card to the nginx in the user space;

[0047] It should be noted that F - stack is a high - performance network access development package in full - user - mode (kernel bypass), based on Data Plane Development Kit (DPDK), FreeBSD (Unix - like operating system) protocol stack, micro - thread interface, etc. It is applicable to various services that require network access. Users only need to focus on the business logic and simply integrate F - Stack to achieve a high - performance network server. In specific implementation, the present invention uses the full - user - space F - stack to replace the original kernel - space protocol stack, directly receives data packets from the network card to the user mode, and completes message processing in the user space, which can significantly improve the performance of the OpenStack load - balancing service and network traffic processing performance.

[0048] S103. Run nginx in multiple processes, concurrently receive and distribute request data packets to the members of the resource pool (Pool) that provides services, for load balancing and response of request messages.

[0049] In specific implementation, nginx can be used as a load balancing execution unit; each process can be bound to an independent network card queue and CPU, and use independent memory. The resource pool is a cluster of backend servers that provide services; the resource pool members are the server members in the resource pool. There can be multiple resource pool members, such as Figure 2 shown, including Member1, Member2, …, MemberN.

[0050] In the above load balancing method provided by the embodiments of the present invention, in a cloud environment based on openstack, it mainly includes two processes: load balancing configuration conversion implementation and request message load balancing. Using the F-stack architecture and nginx, it realizes full user-mode load balancing, provides high-performance load balancing services for the openstack environment, migrates packet processing to the user space, realizes efficient processing of network packets, can reduce the resources and time consumed by packet forwarding, reduce the memory copy from the kernel mode to the user mode, solve the problem of locality failure, eliminate the performance impact of interrupt processing, significantly improve the quality and efficiency of load balancing services, greatly improve the utilization rate of computing resources in the cloud environment and the response efficiency of requests in the cloud environment, and meet the high-concurrency request needs of Internet users.

[0051] Further, in specific implementation, in the above load balancing method provided by the embodiments of the present invention, step S101 transfers the load balancing parameters configured by Octavia front-end to the configuration file of nginx, which can specifically include: after configuring load balancing at the Octavia front-end, obtaining the load balancing parameters configured by the front-end; extracting and converting the obtained load balancing parameters; writing and saving the converted load balancing parameters to a pre-backup configuration file according to the configuration file format of nginx.

[0052] Specifically, after the user configures load balancing at the front-end, the Amphora agent will receive the corresponding configuration parameters, and the load balancing related parameters follow the original parameters of octavia; after the load balancing configuration parameters are extracted, the configuration file is first backed up, and then written and saved to the configuration file according to the configuration file format of nginx.

[0053] In specific implementation, in the above load balancing method provided by the embodiments of the present invention, before restarting and monitoring the status of nginx in step S101, as Figure 3 shown, it can also include: using the nginx–t command to check whether the configuration file format is correct; if so, restart and monitor the status of nginx; if not, return a configuration exception error, perform a rollback operation on the configuration file, and restart nginx to end the configuration process.

[0054] In specific implementation, in the above load balancing method provided by the embodiments of the present invention, as Figure 3 shown, the monitoring of the nginx status may include: monitoring whether the nginx service is started normally; if the startup is normal, the configuration is issued; if the startup is abnormal, a configuration exception error is returned, the configuration file is rolled back, and nginx is restarted to end the configuration process.

[0055] In specific implementation, in the above load balancing method provided by the embodiments of the present invention, as Figure 4 shown, step S103 concurrently receives and distributes request data packets to the resource pool members providing services for load balancing and response of request messages, including: using nginx to select resource pool members for load according to the load balancing rules, modifying the content of the concurrently received request data packets and then transmitting them to the network card via F-stack; distributing the request data packets to the resource pool members providing services through the network card according to the header information of the request data packets; responding to the request content through the resource pool members and returning response data packets; after the returned response data packets are sent to Amphora, they are returned to the user after destination address conversion to complete the load balancing and response of the request messages.

[0056] Specifically, first, the Web user accesses the service provided by the cloud environment through the VIP, and the corresponding request is sent to the network card of the virtual machine where the load balancer is located; then the Amphora virtual machine network card extracts the request data packet and transmits it to nginx in the user space via f-stack; then nginx selects resource pool members for load according to the load balancing rules, modifies the data packet content and then transmits it to the network card via f-stack; the network card distributes the data packet to the resource pool members providing services according to the data packet header information; after the resource pool members receive the request data packet, they respond to the request content and return response data packets; finally, the returned response data packets are sent to Amphora and then returned to the user after destination address conversion to complete the load balancing and response of the request messages. Such a process realizes load balancing in the full user state and improves the load balancing performance.

[0057] Based on the same inventive concept, the embodiments of the present invention also provide a load balancing system. Since the principle of the system to solve problems is similar to that of the foregoing load balancing method, the implementation of the system can refer to the implementation of the load balancing method, and the repeated parts will not be described again.

[0058] In specific implementation, the load balancing system provided by the embodiments of the present invention is used for the openstack cloud environment and specifically includes: Amphora agent, F-stack in the full user space, and nginx;

[0059] Amphora agent is used to transfer the load balancing parameters configured by Octavia front - end to the nginx configuration file, and restart and monitor the status of nginx;

[0060] F - stack is used to directly transfer the request data packet from the network card to the user - space nginx after receiving the user's request;

[0061] nginx is used to concurrently receive and distribute request data packets to the resource pool members providing services for load balancing and response of request messages under multi - process operation.

[0062] In the above - mentioned load - balancing system provided by the embodiments of the present invention, the F - stack architecture and nginx are used to implement full - user - state load balancing, provide high - performance load - balancing services for the openstack environment, migrate packet processing to the user space, achieve efficient processing of network packets, reduce the resources and time consumed by packet forwarding, reduce the memory copy from the kernel state to the user state, solve the problem of locality failure, eliminate the performance impact of interrupt processing, significantly improve the quality and efficiency of load - balancing services, greatly improve the utilization rate of computing resources in the cloud environment and the response efficiency of requests in the cloud environment, and meet the high - concurrent request needs of Internet users.

[0063] Further, in specific implementation, in the above - mentioned load - balancing system provided by the embodiments of the present invention, Amphora agent can specifically be used to obtain the load - balancing parameters configured by the front - end after load - balancing configuration in Octavia; extract and convert the obtained load - balancing parameters; write and save the converted load - balancing parameters to the pre - backed - up configuration file according to the nginx configuration file format.

[0064] Further, in specific implementation, in the above - mentioned load - balancing system provided by the embodiments of the present invention, nginx can specifically be used to select resource pool members for load according to the load - balancing rules, modify the content of the concurrently received request data packets and then transmit them to the network card via F - stack; distribute the request data packets to the resource pool members providing services according to the header information of the request data packets through the network card; respond to the request content through the resource pool members and return the response data packets; after the returned response data packets are sent to amphora, they are returned to the user after destination address conversion, completing the load balancing and response of the request message.

[0065] For the more specific working processes of the above - mentioned various devices, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be elaborated herein.

[0066] Furthermore, the present invention also discloses a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, the load balancing method disclosed above is implemented.

[0067] For a more specific process of the above method, reference may be made to the corresponding content disclosed in the foregoing embodiments, which will not be elaborated herein.

[0068] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference may be made to each other. For the systems and storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and reference may be made to the description of the method part for the relevant parts.

[0069] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0070] The steps of the method or algorithm described in combination with the embodiments disclosed herein can be directly implemented by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the technical field.

[0071] In summary, a load balancing method provided by an embodiment of the present invention includes: passing the load balancing parameters configured at the front end of Octavia to the configuration file of nginx through the Amphora agent, and restarting and monitoring the status of nginx; after receiving a user request, using the full user space F-stack to directly transmit the request data packet from the network card to the user space nginx; running nginx in multiple processes to concurrently receive and distribute the request data packets to the resource pool members providing services, so as to perform load balancing and response of the request messages. In this way, in the cloud environment based on openstack, the full user-mode load balancing is realized by using the F-stack architecture and nginx, providing high-performance load balancing services for the openstack environment, migrating the data packet processing to the user space, realizing the efficient processing of network packets, reducing the resources and time consumed by packet forwarding, reducing the memory copy from the kernel mode to the user mode, solving the locality failure problem, eliminating the performance impact of interrupt processing, significantly improving the quality and efficiency of the load balancing service, greatly enhancing the utilization rate of computing resources in the cloud environment and the response efficiency of requests in the cloud environment, and meeting the high-concurrency request needs of Internet users. In addition, the present invention also provides a corresponding system and computer-readable storage medium for the load balancing method, further making the above method more practical, and the system and computer-readable storage medium have corresponding advantages.

[0072] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without more limitations, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0073] The load balancing method, system and storage medium provided by the present invention have been introduced in detail above. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A load balancing method, characterized in that, For the OpenStack cloud environment, including: Transfer the load balancing parameters configured by Octavia front-end to the nginx configuration file through the Amphora agent, and use the nginx-t command to check whether the configuration file format is correct; If so, restart and monitor the status of nginx; if not, return a configuration exception error, perform a rollback operation on the configuration file, and restart nginx; After receiving the user's request, replace the kernel space protocol stack with the full user space F-stack, and directly transfer the request data packet from the network card to the user space nginx; Run nginx in multiple processes, and use nginx to select resource pool members for load balancing according to the load balancing rules, modify the content of the concurrently received request data packets, and then transmit them to the network card via F-stack; According to the header information of the request data packet, distribute the request data packet to the resource pool members providing services through the network card; After receiving the request data packet through the resource pool member, respond to the request content and return a response data packet; After the returned response data packet is sent to Amphora, it is returned to the user after destination address conversion, completing the load balancing and response of the request message to achieve full user-state load balancing.

2. The load balancing method according to claim 1, wherein The transfer of the load balancing parameters configured by Octavia front-end to the nginx configuration file includes: After configuring load balancing at the Octavia front-end, obtain the load balancing parameters configured by the front-end; Extract and convert the obtained load balancing parameters; Write and save the converted load balancing parameters to the pre-backup configuration file according to the nginx configuration file format.

3. The load balancing method according to claim 2, wherein The monitoring of the nginx status includes: Monitor whether the nginx service starts normally; If the start is normal, complete the configuration distribution; If the start is abnormal, return a configuration exception error, perform a rollback operation on the configuration file, and restart nginx.

4. The load balancing method according to claim 3, wherein Each of the processes binds to an independent network card queue and CPU and uses independent memory.

5. A load balancing system, characterized in that, For the OpenStack cloud environment, including: Amphora agent, full user space F-stack, and nginx; Amphora agent, used to transfer the load balancing parameters configured by Octavia front-end to the nginx configuration file, and use the nginx–t command to check whether the configuration file format is correct; if so, restart and monitor the status of nginx; if not, return a configuration exception error, perform a rollback operation on the configuration file, and restart nginx; F-stack, used to replace the kernel space protocol stack after receiving the user's request, and directly transfer the request data packet from the network card to the user space nginx; Nginx is used to, under multi-process operation, select resource pool members for load balancing according to load balancing rules, modify the content of the concurrently received request data packets and then transmit them to the network card via F-stack; distribute the request data packets to the resource pool members providing services through the network card according to the header information of the request data packets; after receiving the request data packets through the resource pool members, respond to the request content and return response data packets; after the returned response data packets are sent to Amphora, they are returned to the user after destination address conversion, completing the load balancing and response of the request message to achieve full user-state load balancing.

6. The load balancing system according to claim 5, wherein The Amphora agent is specifically used to, after configuring load balancing at the Octavia front end, obtain the load balancing parameters configured at the front end; extract and convert the obtained load balancing parameters; write and save the converted load balancing parameters to a pre-backup configuration file in the format of the nginx configuration file.

7. A computer-readable storage medium, characterized in that, A computer program is stored, wherein the computer program, when executed by a processor, implements the load balancing method according to any one of claims 1 to 4.

Images