Request processing method and device, electronic equipment and computer readable storage medium

Abstract

The invention relates to a request processing method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps that under the condition that a proxy server receives a plurality of target requests, the proxy server transmits the plurality of target requests to a plurality of network load balancing modules, each network load balancing module in the plurality of network load balancing modules receives a part of target requests in the plurality of target requests, and the difference value between the numbers of the target requests received by any two network load balancing modules in the plurality of network load balancing modules is smaller than a difference value threshold value; the plurality of network load balancing modules respectively transmit the plurality of target requests to the plurality of container groups, and each container group in the plurality of container groups is used for processing part of the target requests in the plurality of target requests. According to the method and the device, the plurality of network load balancing modules are arranged in the container arrangement platform, so that the problem of relatively weak high-concurrency processing capability of the container arrangement platform in the related technology is solved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a request processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the rapid development of information technology, various Internet applications and distributed systems are facing increasingly more high-concurrency scenarios. To improve the concurrent processing capabilities of servers, traditional solutions typically rely on deploying a large number of physical machines and combining load balancing technology to distribute requests to backend services. While this model can achieve high concurrency goals, the large number of physical machines brings high deployment and maintenance costs.

[0003] Against this backdrop, container orchestration platforms are increasingly being adopted by enterprises as a solution for building and managing microservice architectures. For example, container orchestration platforms can include, but are not limited to, Kubernetes (K8s). However, in practical applications, it has been found that although container orchestration platforms can easily increase the number of pods (container components), the overall system's concurrent processing capacity does not increase linearly. After the number of pods reaches a certain scale, it becomes difficult to further improve the system's concurrent processing capacity. Therefore, container orchestration platforms suffer from a weakness in high-concurrency processing capabilities. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides a request processing method, apparatus, electronic device, and computer-readable storage medium to resolve the issue of weak high-concurrency processing capabilities in container orchestration platforms in the related technologies.

[0005] In a first aspect, embodiments of this disclosure provide a request processing method applied to a container orchestration platform. The container orchestration platform includes: a proxy server, multiple network load balancing modules, and multiple container groups. Each of the multiple network load balancing modules is connected to multiple container groups. The request processing method includes: When the proxy server receives multiple target requests, it transmits these requests to multiple network load balancing modules. Each network load balancing module receives a portion of the target requests, and the difference between the number of target requests received by any two network load balancing modules is less than a threshold. The multiple network load balancing modules then transmit these target requests to multiple container groups, where each container group processes a portion of the target requests.

[0006] In some alternative implementations, the proxy server transmits multiple target requests to multiple network load balancing modules, including: The proxy server polls multiple network load balancing modules in rounds to determine the first correspondence between the multiple network load balancing modules and multiple target requests; based on the address information of the multiple network load balancing modules, the proxy server transmits the multiple target requests to the multiple network load balancing modules respectively according to the first correspondence.

[0007] In some alternative implementations, multiple network load balancing modules respectively transmit multiple target requests to multiple container groups, including: For each of the multiple network load balancing modules, the network load balancing module obtains the load weighting value of each container group in the multiple container groups, wherein the load weighting value of each container group is used to characterize the number of requests being processed by the container group; the network load balancing module determines at least one first container group corresponding to the network load balancing module based on the load weighting value; the network load balancing module transmits the target request received by the network load balancing module to at least one first container group according to the address information of at least one first container group.

[0008] In some alternative implementations, multiple network load balancing modules respectively transmit multiple target requests to multiple container groups, including: For each of the multiple network load balancing modules, the network load balancing module polls multiple container groups in round-robin to determine a second correspondence between the network load balancing module and at least one second container group among the multiple container groups; the network load balancing module transmits the target request received by the network load balancing module to at least one second container group according to the address information of at least one second container group and the second correspondence.

[0009] In some alternative implementations, multiple network load balancing modules respectively transmit multiple target requests to multiple container groups, including: Multiple network load balancing modules sequentially perform target network address translation (NAT) and source network address translation (NAT) processing on multiple target requests; the multiple network load balancing modules then transmit the NAT-translated target requests to multiple container groups.

[0010] In some optional implementations, after multiple network load balancing modules have respectively transmitted multiple target requests to multiple container groups, the request processing method further includes: When multiple container groups generate multiple request results corresponding to multiple target requests, the multiple container groups perform source network address translation (NAT) and target network address translation (NAT) processing on the multiple request results in sequence; the multiple container groups then transmit the NAT-translated multiple request results to the proxy server.

[0011] In some optional implementations, after multiple container groups transmit the address-translated results of multiple requests to the proxy server, the request processing method further includes: The proxy server transmits the results of multiple requests to multiple target clients, where the multiple target clients are the clients that transmit multiple target requests to the proxy server.

[0012] Secondly, embodiments of this disclosure provide a request processing apparatus applied to a container orchestration platform. The container orchestration platform includes: a proxy server, multiple network load balancing modules, and multiple container groups. Each of the multiple network load balancing modules is connected to multiple container groups. The request processing apparatus includes: The first transmission unit is used to execute the following: when the proxy server receives multiple target requests, the proxy server transmits the multiple target requests to multiple network load balancing modules respectively. Each of the multiple network load balancing modules receives a portion of the target requests from the multiple target requests, and the difference between the number of target requests received by any two of the multiple network load balancing modules is less than a difference threshold. The second transmission unit is used to execute the multiple network load balancing modules to transmit the multiple target requests to multiple container groups respectively. Each of the multiple container groups is used to process a portion of the target requests from the multiple target requests.

[0013] Thirdly, embodiments of this disclosure provide an electronic device, including: A memory; a processor; and a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor to implement the request processing method as described in the first aspect.

[0014] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the request processing method described in the first aspect.

[0015] Fifthly, embodiments of this disclosure also provide a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the request processing method described above.

[0016] The request processing method, apparatus, electronic device, and computer-readable storage medium provided in this disclosure, by setting up multiple network load balancing modules in a container orchestration platform, allow the proxy server to distribute multiple target requests to different network load balancing modules when it receives multiple target requests. These different network load balancing modules then transmit the multiple requests to multiple container groups, reducing the likelihood of congestion at each network load balancing module. This ensures that the target requests received by the proxy server can be promptly transmitted to the backend container groups through multiple network load balancing modules, thereby improving the concurrent processing capability of the container orchestration platform and solving the problem of weak high-concurrency processing capability in related technologies. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart of a request processing method provided in an embodiment of this disclosure; Figure 2 A schematic diagram illustrating an application scenario provided by an embodiment of this disclosure; Figure 3 A flowchart of a request processing method provided in another embodiment of this disclosure; Figure 4 A flowchart of a request processing method provided in yet another embodiment of this disclosure; Figure 5 A flowchart of a request processing method provided in yet another embodiment of this disclosure; Figure 6 A schematic diagram of the structure of the request processing apparatus provided in the embodiments of this disclosure; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0020] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0021] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0022] This disclosure provides a request processing method, which will be described below with reference to specific embodiments.

[0023] Some of the terms or terms that appear in the description of the embodiments of this disclosure shall be interpreted as follows: Container orchestration platform: An open-source container cluster management system used for the automated deployment, scaling, and management of containerized applications. For example, a container orchestration platform may include, but is not limited to, Kubernetes (K8s). This application embodiment uses K8s as an example for description.

[0024] Proxy server (Nginx, engine x): HTTP (Hypertext Transfer Protocol) and reverse proxy web (network) server, and also provides IMAP (Internet Message Access Protocol), POP3 (Post Office Protocol version 3), and SMTP (Simple Mail Transfer Protocol) proxy services.

[0025] Pod: A deployable and manageable unit of computation in a container orchestration platform. It encapsulates an application; containers are a lightweight, portable software packaging technology that enables applications and their dependencies to run consistently in virtually any computing environment. For example, Docker containers.

[0026] Network load balancing module: An abstract service in the container orchestration platform that defines the logical way proxy servers access container groups. The network load balancing module can achieve traffic load balancing across multiple container groups by selecting backend container groups.

[0027] IP: an abbreviation for Internet Protocol Address.

[0028] Figure 1 A flowchart illustrating a request processing method provided in an embodiment of this disclosure. This method can be applied to... Figure 2The application scenario shown includes a container orchestration platform and a client. The container orchestration platform can be an open-source container cluster management system (Kubernetes, K8s); the client can be, for example, a smartphone, PDA, tablet, wearable device with a display, desktop computer, laptop computer, all-in-one computer, smart home device, etc. It is understood that the request processing method provided in this disclosure can also be applied to other scenarios. The request processing method in this disclosure can be executed by the container orchestration platform or a server deploying the container orchestration platform.

[0029] The following is combined with Figure 2 The application scenarios shown are for Figure 1 The request processing method shown is described below. This method is applied to a container orchestration platform or a server on which a container orchestration platform is deployed. The container orchestration platform includes: a proxy server, multiple network load balancing modules, and multiple container groups. Each of the multiple network load balancing modules is connected to multiple container groups. The specific steps included in this request processing method are as follows: S101. When the proxy server receives multiple target requests, the proxy server will transmit the multiple target requests to multiple network load balancing modules respectively.

[0030] In this process, each of the multiple network load balancing modules receives a portion of the target requests from the multiple target requests, and the difference between the number of target requests received by any two network load balancing modules is less than a difference threshold.

[0031] It should be noted that a partial target request refers to at least one or more of multiple target requests. In other words, when the proxy server receives multiple target requests, it will not distribute all target requests to the same network load balancing module.

[0032] In this embodiment, the container orchestration platform includes a proxy server, a network load balancing module, and container components. There are multiple network load balancing modules and multiple container components; the number of network load balancing modules and the number of container components can be the same or different.

[0033] For example, the number of container components is greater than or equal to the number of network load balancing modules.

[0034] In this embodiment, the multiple target requests received by the proxy server can originate from multiple clients or from a single client; the multiple target requests refer to the target requests received by the proxy server within a certain time period. Specifically, clients initiate target requests to the container orchestration platform's proxy server, and each client can send one or more target requests to the proxy server.

[0035] For example, the request processing method is applied to a video playback platform, where multiple target requests include multiple clients requesting playback of video streams from the video playback platform.

[0036] For example, the request processing method is applied to a ticketing platform, and the multiple target requests include purchase requests, refund requests, ticket exchange requests, etc., made by multiple clients to the ticketing platform.

[0037] For example, the data packet of the target request received by the proxy server can be [Source:ClientIP:12345]->[Dest:NginxIP:80].

[0038] Wherein, Source:ClientIP:12345 represents the client whose source address is IP:12345, and Dest:NginxIP:80 represents the proxy server whose target address is IP:80.

[0039] In related technologies, container orchestration platforms typically use only one network load balancing module, distributing target requests across multiple backend container components using a single module. When the container orchestration platform's proxy server receives multiple target requests, all requests are distributed through this single network load balancing module. Therefore, even with an increase in the number of backend container groups, once the number of container groups reaches a certain scale, the performance of this single network load balancing module limits the concurrency processing capabilities of the container orchestration platform, preventing further improvements.

[0040] In this embodiment, multiple network load balancing modules are configured in the container orchestration platform. Each network load balancing module is connected to all backend container groups, enabling each module to transmit target requests to all backend container groups. When the proxy server receives multiple target requests, it can distribute these requests to the multiple network load balancing modules. This distribution avoids concentrating a large number of target requests across only a portion of the network load balancing modules, ensuring that each module is allocated at least one target request, thus reducing the likelihood of congestion in any single module.

[0041] Specifically, the proxy server distributes target requests to multiple network load balancing modules according to preset rules, so that the difference between the number of target requests received by any two network load balancing modules is less than the difference threshold, thereby reducing the gap between the number of target requests received by multiple network load balancing modules and improving the uniformity of target request distribution to multiple network load balancing modules.

[0042] For example, the difference threshold ranges from 1 to 100.

[0043] For example, the data packet of the target request received by the network load balancing module can be [Source:NginxIP:45678]->[Dest:ServiceIP:80].

[0044] Wherein, Source:NginxIP:45678 represents the proxy server whose source address is IP:45678, and Dest:ServiceIP:80 represents the target load balancing module whose target address is IP:80.

[0045] In actual testing, with only one network load balancing module configured, the query per second (QPS) of the container orchestration platform remained consistently around 20,000, regardless of the number of container groups set up within it. However, by increasing the number of network load balancing modules and connecting multiple modules to multiple container groups, the QPS of the container orchestration platform could be increased to the millions.

[0046] The following combination Figure 2 This is illustrated by taking the example of a proxy server transmitting a target request to N network load balancing modules: The proxy server distributes the target requests one by one to Service1, Service2...ServiceN through Upstream (an upstream server group, a configuration block of a proxy server used to forward target requests), so that the number of target requests received by any two network load balancing modules in Service1, Service2...ServiceN is less than the number threshold.

[0047] In some possible implementations, the proxy server obtains the current request count for each network load balancing module, where the current request count is the number of requests being processed by the container group connected to the network load balancing module. The proxy server allocates target requests to each network load balancing module based on the current request count, wherein the number of target requests allocated to each network load balancing module is negatively correlated with the current request count.

[0048] In these implementations, the current request count can characterize the number of requests being processed by multiple container groups corresponding to the network load balancing module. To avoid continuing to allocate target requests to container groups that have already processed a large number of requests, the proxy server obtains the current request count of each network load balancing module when allocating target requests to the network load balancing module, and prioritizes transmitting target requests to network load balancing modules with fewer current request counts. This allows the network load balancing module to subsequently allocate target requests to the container groups it is connected to that are processing fewer requests, ensuring that container groups processing fewer requests receive target requests first, and that container groups processing a large number of requests receive as few target requests as possible. S102, Multiple network load balancing modules transmit multiple target requests to multiple container groups respectively.

[0049] Each of the multiple container groups is used to process a portion of the target requests from the multiple target requests.

[0050] In this embodiment, when each network load balancing module receives a target request sent by the proxy server, the network load balancing module needs to determine the corresponding container group and transmit the target request to the corresponding container group.

[0051] The following combination Figure 2 This explanation uses the example of Service1 transmitting a target request to the container group from among N network load balancing modules as an example: When Service1 receives a single target request, it chooses to forward the request to one of Pod1, Pod2, ..., PodN. When Service1 receives multiple target requests, it uses either a round-robin algorithm or a least-connections algorithm to forward the requests to one or more of Pod1, Pod2, ..., PodN.

[0052] In this embodiment of the disclosure, by setting up multiple network load balancing modules in the container orchestration platform, when the proxy server receives multiple target requests, the proxy server distributes the multiple target requests to different network load balancing modules. The multiple requests are then transmitted to multiple container groups through different network load balancing modules, reducing the possibility of congestion at each network load balancing module. This allows the target requests received by the proxy server to be transmitted to the backend container groups in a timely manner through multiple network load balancing modules, thereby improving the concurrent processing capability of the container orchestration platform and solving the problem of weak high-concurrency processing capability of container orchestration platforms in related technologies.

[0053] Figure 3 A flowchart of a request processing method provided in another embodiment of this disclosure is shown below. Figure 3As shown, the request processing method is applied to a container orchestration platform, which includes a proxy server, multiple network load balancing modules, and multiple container groups. Each of the network load balancing modules is connected to multiple container groups. The specific steps of this request processing method are as follows: S301. When the proxy server receives multiple target requests, the proxy server polls multiple network load balancing modules in round-robin fashion to determine the first correspondence between the multiple network load balancing modules and the multiple target requests.

[0054] In this embodiment, when the proxy server transmits multiple target requests to multiple network load balancing modules, it needs to determine a first correspondence between each target request and each network load balancing module. This first correspondence is used to characterize the destination of each target request in the multiple network load balancing modules. The proxy server determines the network load balancing module corresponding to each target request by polling, thereby determining the first correspondence.

[0055] In some possible implementations, the proxy server uses a default round-robin algorithm to poll the network load balancing modules. Specifically, multiple target requests are distributed to multiple network load balancing modules one by one according to the time sequence of their receipt.

[0056] The following combination Figure 2 The application scenario illustrates the process of the default round-robin algorithm transmitting target requests to the network load balancing module: When the proxy server receives M target requests, it sorts the M target requests according to their reception time to obtain their chronological order. The M target requests are then sequentially assigned to Service1 through ServiceN in chronological order. If M is greater than N, the first N target requests are assigned to Service1 through ServiceN one by one, and then starting from the (N+1)th target request, they are sequentially assigned to Service1 through ServiceN again, and this process is repeated until all M target requests have been assigned.

[0057] Since multiple target requests are distributed to multiple network load balancing modules in chronological order, the threshold for the difference between the number of target requests received by any two network load balancing modules is 1.

[0058] In some other possible implementations, the proxy server uses a weighted round-robin algorithm to poll the network load balancing modules. Specifically, the proxy server assigns a weight to each network load balancing module in a round-robin manner, where the weight is positively correlated with the number of container groups connected to each network load balancing module. The proxy server allocates target requests to each network load balancing module according to its assigned weight, assigning more target requests to network load balancing modules with higher weights and fewer target requests to network load balancing modules with lower weights.

[0059] In these implementations, the proxy server can determine the number of target requests to be allocated to each network load balancing module based on the number of container groups connected to each network load balancing module. This allows network load balancing modules connected to more container groups to receive more target requests, while network load balancing modules connected to fewer container groups can receive fewer target requests. This reduces the likelihood of more target requests being transmitted to fewer container groups, thereby improving the concurrent processing capability of the container orchestration platform.

[0060] In some other possible implementations, the proxy server uses a least-connections algorithm to poll the network load balancing modules in round-robin fashion. Specifically, the proxy server polls multiple network load balancing modules to determine the current number of requests in each module, and prioritizes sending the target request to the network load balancing module with the fewest current requests.

[0061] Step 302: The proxy server transmits multiple target requests to multiple network load balancing modules according to the address information of multiple network load balancing modules and the first correspondence relationship.

[0062] In this embodiment, after the proxy server determines the network load balancing module corresponding to each target request, the proxy server can transmit each target request to the network load balancing module based on the address information of multiple network load balancing modules.

[0063] Specifically, the proxy server forwards target requests to an upstream server group via proxy pass instructions (proxy_pass). This upstream server group has address information for multiple network load balancing modules, and uses a round-robin algorithm to forward multiple target requests to multiple network load balancing modules. For example, the address information can be the IP address, port, or a combination of port and IP address of the network load balancing module. Specifically, if the container orchestration platform's proxy mode is IPVS (IP Virtual Server), then the address information retrieved will be the service IP and port (ServiceIP + Port) of the network load balancing module.

[0064] Step 303: Multiple network load balancing modules transmit multiple target requests to multiple container groups respectively.

[0065] Each of the multiple container groups is used to process a portion of the target requests from the multiple target requests.

[0066] Specifically, the implementation process and principle of S303 and S102 are the same, and will not be repeated here.

[0067] In this embodiment of the disclosure, the proxy server can determine a first correspondence between multiple target requests and multiple network load balancing modules by polling. This first correspondence can characterize the network load balancing module that each target request needs to be transmitted to. The proxy server then transmits multiple target requests to multiple network load balancing modules accordingly. Since the target requests are allocated by polling, the uniformity of the allocation to multiple network load balancing modules can be improved, and the possibility of multiple target requests being congested in the same network load balancing module can be further reduced.

[0068] Figure 4 A flowchart of a request processing method provided in another embodiment of this disclosure is shown below. Figure 4 As shown, the request processing method is applied to a container orchestration platform, which includes a proxy server, multiple network load balancing modules, and multiple container groups. Each of the network load balancing modules is connected to multiple container groups. The specific steps of this request processing method are as follows: S401. When the proxy server receives multiple target requests, the proxy server will transmit the multiple target requests to multiple network load balancing modules respectively.

[0069] Specifically, the implementation process and principle of S401 and S101 are the same, and will not be repeated here.

[0070] S402. For each of the multiple network load balancing modules, the network load balancing module obtains the load weighting value of each container group in the multiple container groups.

[0071] The load weighting value for each container group is used to characterize the number of requests that the container group is processing. In this embodiment, when multiple network load balancing modules receive a target request, each network load balancing module needs to select which container group to transmit the target request to. Therefore, the following description takes a single network load balancing module as the execution subject: The network load balancing module obtains the load weighting value of each container group it is connected to. The more requests being processed in a container group, the higher the load weighting value; the fewer requests being processed in a container, the lower the load weighting value.

[0072] S403. The network load balancing module determines at least one first container group corresponding to the network load balancing module among multiple container groups based on the load weighting value.

[0073] In this embodiment, each network load balancing module can determine the number of requests being processed by each container group based on the load weighting value of each container group. After determining the load weighting values ​​of multiple container groups, the network load balancing module selects the first container group to which the target request needs to be transmitted based on the load weighting values ​​of multiple container groups. It can be understood that the number of target requests that the network load balancing module needs to forward can be one or more, and multiple target requests can be transmitted to the same container group or to multiple container groups respectively.

[0074] Specifically, the load weighting value can represent the number of target requests that each container group is processing. Therefore, the network load balancing module can choose to select the first container group by either threshold judgment or load weighting value sorting.

[0075] In some possible implementations, the network load balancing module sorts the container groups in ascending order of load weight, prioritizing the container groups ranked higher as the first container group. For example, if the network load balancing module has 6 target requests and connects 5 container groups, then the top 3 container groups are designated as the first container group. The first container group ranked first is assigned 3 target requests, the second-ranked first container group is assigned 2 target requests, and the third-ranked first container group is assigned 1 target request.

[0076] In some other possible implementations, the network load balancing module compares the load weight value of each container group with a weight threshold to select the first container group that meets the requirements. For example, if the target number of requests in the network load balancing module is 8 and there are 5 connected container groups, the 4 container groups with load weight values ​​lower than the weight threshold are identified as the first container groups. Then, the 8 target requests are distributed to the 4 first container groups according to the allocation method of 2 requests per first container group.

[0077] S404. The network load balancing module transmits the target request received by the network load balancing module to at least one first container group based on the address information of at least one first container group.

[0078] In this embodiment, the network load balancing module stores address information for multiple container groups. After the network load balancing module determines at least one first container group among the multiple container groups, it transmits the target request to the first container group using the address information of the first container group.

[0079] In this embodiment of the application, when the network load balancing module distributes the target request to the container group, it obtains the load weighting value of the connected container group and selects at least one first container group that can receive the target request based on the load weighting value. The process of the network load balancing module selecting the first container group takes into account the number of requests being processed in each container group, which reduces the possibility that the container group that is processing more requests will receive the target request and increases the possibility that the container group that is processing fewer requests will receive the target request, thereby reducing the occurrence of uneven distribution of target requests among multiple target container groups.

[0080] Figure 5 A flowchart of a request processing method provided in another embodiment of this disclosure is shown below. Figure 5 As shown, the request processing method is applied to a container orchestration platform, which includes a proxy server, multiple network load balancing modules, and multiple container groups. Each of the network load balancing modules is connected to multiple container groups. The specific steps of this request processing method are as follows: S501. When the proxy server receives multiple target requests, the proxy server will transmit the multiple target requests to multiple network load balancing modules respectively.

[0081] Specifically, the implementation process and principle of S501 and S101 are the same, and will not be repeated here.

[0082] S502. For each of the multiple network load balancing modules, the network load balancing module polls the multiple container groups in round-robin fashion to determine a second correspondence between the network load balancing module and at least one second container group among the multiple container groups.

[0083] In this embodiment, when multiple network load balancing modules receive a target request, each network load balancing module needs to select which container group to transmit the target request to. Therefore, the following description takes a single network load balancing module as the execution subject: After receiving the target request, when the network load balancing module transmits the target request to the container group, it needs to determine the network load balancing module and at least one second container group corresponding to the multiple container groups, and the second correspondence represents the correspondence between the network load balancing module and at least one second container group.

[0084] Specifically, the network load balancing module determines the second container group corresponding to each target request through a round-robin process, thus establishing the second correspondence. The network load balancing module obtains the time sequence of the target requests it receives, as well as the arrangement order of the multiple container groups. It identifies the last container group to receive a request and designates the container group whose arrangement order follows that of the last receiving request as the first second container group, which is the starting point of this round-robin. The network load balancing module then transmits the target requests to the second container groups one by one according to their time sequence and arrangement order.

[0085] The following combination Figure 2 The application scenario illustrates the process of the network load balancing module transmitting target requests to the container group: When the network load balancing module receives X target requests, it sorts these X target requests according to their reception time, obtaining their chronological order. If the network load balancing module determines that PodN is the last container group to receive a request, it then determines Pod1 as the starting point for round-robin distribution and allocates the X target requests to at least a portion of the Pods from Pod1 to PodN.

[0086] If X is greater than or equal to N, then Pods 1 through Pod N are all designated as the second container group. The first N target requests out of the X are then distributed to Pods 1 through Pod N one by one. Starting from the (N+1)th target request, the requests are then distributed to Pods 1 through Pod N in turn, and this process is repeated until all X target requests are distributed to the second container group. If X is less than N, then Pods 1 through Pod X are designated as the second container group, and the X target requests are distributed to Pods 1 through Pod X.

[0087] S503. The network load balancing module transmits the target request received by the network load balancing module to at least one second container group according to the address information of at least one second container group and in accordance with the second correspondence.

[0088] In this embodiment, the network load balancing module stores address information for multiple container groups. After the network load balancing module determines at least one second container group among the multiple container groups, it transmits the target request to the second container group using the address information of the first container group.

[0089] In this embodiment, when the network load balancing module distributes target requests to container groups, it determines the second container group among multiple container groups through a round-robin method, and transmits the target requests in the network load balancing module to the second container group one by one. This enables the target requests in the network load balancing module to be transmitted evenly to the second container group. Since multiple network load balancing modules all use a round-robin method to transmit target requests to multiple container groups, it can improve the uniformity of request reception for all container groups in the entire container orchestration platform.

[0090] It is understood that, in the embodiments of this application, Figures 1 to 5 The solutions shown can be used individually or in combination, provided there are no conflicts. For example: Figure 4 , Figure 5 The scheme shown above determines the container group that receives the target request and Figure 2 The scheme shown for determining the network load balancing module that receives the target request can be used in combination.

[0091] In some embodiments of this disclosure, multiple network load balancing modules transmit multiple target requests to multiple container groups, including: the multiple network load balancing modules sequentially perform target network address translation processing and source network address translation processing on the multiple target requests; the multiple network load balancing modules transmit the target requests after address translation processing to the multiple container groups respectively.

[0092] In this embodiment, when multiple network load balancing modules transmit a target request to multiple container groups, they need to modify the target address and source address of the target request data packet in sequence, that is, perform target network address translation and source network address translation on the target request in sequence.

[0093] Specifically, when a target request reaches the network load balancing module, it first performs destination network address translation (DNAT), modifying the destination IP and port of the target request packet from the network load balancing module's IP to the container group's IP and port. After the target request packet undergoes DNAT, its destination address is now the container group's IP. However, to ensure the correct return of the response packet, source network address translation (SNAT) is required. DNAT enables the address information in the target request packet to represent the destination address, while SNAT enables the address information in the target request packet to represent the source address.

[0094] For example, the data packet of the target request after the target network address translation and source network address translation is [Source:NginxIP:56789]->[Dest:PodIP:80], where Source:NginxIP:56789 represents the proxy server with the source address of the target request being IP:56789, and Dest:PodIP:80 represents the container group with the target address of the target request being IP:80.

[0095] In this embodiment, when the network load balancing module transmits a target request to the container group, target network address translation (NAT) and source network address translation (NAT) are performed sequentially. This ensures that the data packets of the target request can be stably transmitted to the corresponding container group and instructs the container group to send back the request result of the target request to the address, thereby improving the stability of the transmission of the request packets and response packets of the target request.

[0096] In some embodiments of this disclosure, after multiple network load balancing modules transmit multiple target requests to multiple container groups respectively, the request processing method further includes: When multiple container groups generate multiple request results corresponding to multiple target requests, the multiple container groups perform source network address translation (NAT) and target network address translation (NAT) processing on the multiple request results in sequence; the multiple container groups then transmit the NAT-translated multiple request results to the proxy server.

[0097] In this embodiment, the container group can process the target request and obtain the corresponding request result. Each container group needs to modify the source address and destination address of the request result data packet in sequence, that is, perform source network address translation (NAT) and destination network address translation (NAT) on the request result in sequence. The address-translated request result is then sent back to the proxy server, so that the proxy server can subsequently send the request result back to the client.

[0098] Specifically, after the container group processes the target request and obtains the request result, the source address of the data packet sent as the request result is the container group's address, and the destination address is the proxy server's address. In order for this request result data packet to be correctly received by the client, it is necessary to perform source network address translation and the reverse of destination network address translation in sequence.

[0099] For example, the data packet of the request result after source network address translation processing is [Source:ServiceIP:80]->[Dest:NginxIP:45678], where Source:ServiceIP:80 represents the network load balancing module with IP:80 as the source address of the request result, and the proxy server with IP:45678 as the destination address of the request result.

[0100] In this embodiment of the application, when the container group generates the request result corresponding to the target request, it is necessary to perform source network address translation and target network address translation in reverse order on the data packet of the request result, thereby reducing the occurrence of data packet loss during transmission.

[0101] In some embodiments of this disclosure, after multiple container groups transmit multiple request results after address translation processing to a proxy server, the request processing method further includes: the proxy server transmitting multiple request results to multiple target clients, wherein the multiple target clients are clients that transmit multiple target requests to the proxy server.

[0102] In this embodiment, after receiving multiple request results from multiple container groups, the proxy server sends the multiple request results back to the target client that sent the target request, thereby completing the processing of this target request. Since each target client sends at least one target request, the number of request results returned to the target client is the same as the number of target requests, improving the stability of the interaction between the container orchestration platform and the target client.

[0103] For example, the data packet of the request result transmitted by the proxy server to the client is [Source:NginxIP:80]->[Dest:ClientIP:12345], where Source:NginxIP:80 indicates that the source address of the request result is the proxy server with IP:80, and the destination address of the request result is the client with IP:12345.

[0104] like Figure 2 As shown, client 210 transmits multiple target requests to proxy server 221 in container orchestration platform 220. Proxy server 221 transmits the target requests to multiple network load balancing modules 222. Multiple network load balancing modules 222 forward the target requests to multiple container groups 223. Multiple container groups 223 can send the processed request results back to proxy server 221, so that proxy server sends the request results back to client 210.

[0105] Figure 6 This is a schematic diagram of the structure of a request processing apparatus provided in an embodiment of this disclosure. The request processing apparatus may be a container orchestration platform as described in the above embodiments, or it may be a server deploying a container orchestration platform. The apparatus provided in this disclosure can execute the processing flow provided in the method embodiments, such as... Figure 6As shown, the request processing device 60 is applied to a container orchestration platform, which includes: a proxy server, multiple network load balancing modules and multiple container groups. Each of the multiple network load balancing modules is connected to multiple container groups. The request processing device 60 includes: a first transmission unit 61 and a second transmission unit 62. The first transmission unit 61 is used to transmit multiple target requests to multiple network load balancing modules respectively when the proxy server receives multiple target requests. Each of the multiple network load balancing modules receives a portion of the multiple target requests, and the difference between the number of target requests received by any two network load balancing modules is less than a difference threshold. The second transmission unit 62 is used to transmit multiple target requests to multiple container groups respectively, wherein each container group is used to process a portion of the multiple target requests.

[0106] In some embodiments of this disclosure, the request processing apparatus 60 further includes: The first polling unit is used to execute the proxy server to poll multiple network load balancing modules in a round-robin fashion to determine the first correspondence between multiple network load balancing modules and multiple target requests; The first transmission unit 61 is also used to transmit multiple target requests to multiple network load balancing modules according to the address information of multiple network load balancing modules and in accordance with the first correspondence relationship.

[0107] In some embodiments of this disclosure, the request processing apparatus 60 further includes: The acquisition unit is used to acquire the load weighting value of each container group in multiple container groups for each of the multiple network load balancing modules, wherein the load weighting value of each container group is used to characterize the number of requests being processed by the container group. A determining unit is used to determine at least one first container group corresponding to the network load balancing module among multiple container groups based on the load weighting value; The second transmission unit 62 is used to transmit the target request received by the network load balancing module to at least one first container group according to the address information of at least one first container group.

[0108] In some embodiments of this disclosure, the request processing apparatus 60 further includes: The second polling unit is used to poll multiple container groups for each of the multiple network load balancing modules to determine the second correspondence between the network load balancing module and at least one second container group among the multiple container groups. The second transmission unit 62 is used to transmit the target request received by the network load balancing module to at least one second container group according to the address information of at least one second container group and in accordance with the second correspondence.

[0109] In some embodiments of this disclosure, the request processing apparatus 60 further includes: The first address translation unit is used to sequentially perform target network address translation processing and source network address translation processing on multiple target requests respectively; The third transmission unit is used to transmit the target request after address translation to multiple container groups respectively.

[0110] In some embodiments of this disclosure, the request processing apparatus 60 further includes: The second address translation unit is used to perform source network address translation processing and target network address translation processing on the multiple request results in turn when multiple container groups generate multiple request results corresponding to multiple target requests. The fourth transmission unit is used to transmit the results of multiple requests after address translation processing to the proxy server.

[0111] In some embodiments of this disclosure, the request processing apparatus 60 further includes: The fifth transmission unit is used to transmit multiple request results to multiple target clients, wherein the multiple target clients are clients that transmit multiple target requests to the proxy server.

[0112] Figure 6 The request processing apparatus of the illustrated embodiment can be used to execute the technical solution of the above-described request processing method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0113] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. The electronic device can be an upgradeable device as described in the above embodiments. The electronic device provided in this disclosure can execute the processing flow provided in the request processing method embodiments, such as… Figure 7 As shown, device 70 includes: memory 71, processor 72 and computer program; wherein the computer program is stored in memory 71 and configured to be executed by processor 72 as described above in the request processing method.

[0114] In addition, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the request processing method described in the above embodiments.

[0115] Furthermore, this disclosure also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the request processing method described above.

[0116] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0117] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A request processing method, characterized in that, This method is applied to a container orchestration platform, which includes: a proxy server, multiple network load balancing modules, and multiple container groups. Each of the multiple network load balancing modules is connected to the multiple container groups. The request processing method includes: When the proxy server receives multiple target requests, the proxy server transmits the multiple target requests to the multiple network load balancing modules respectively. Each of the multiple network load balancing modules receives a portion of the multiple target requests. The difference between the number of target requests received by any two of the multiple network load balancing modules is less than a difference threshold. The plurality of network load balancing modules respectively transmit the plurality of target requests to the plurality of container groups, wherein each of the plurality of container groups is used to process a portion of the plurality of target requests.

2. The request processing method according to claim 1, characterized in that, The proxy server transmits the multiple target requests to the multiple network load balancing modules respectively, including: The proxy server polls the plurality of network load balancing modules in a round-robin fashion to determine a first correspondence between the plurality of network load balancing modules and the plurality of target requests; The proxy server transmits the multiple target requests to the multiple network load balancing modules according to the address information of the multiple network load balancing modules and in accordance with the first correspondence.

3. The request processing method according to claim 1, characterized in that, The multiple network load balancing modules respectively transmit the multiple target requests to the multiple container groups, including: For each of the plurality of network load balancing modules, the network load balancing module obtains the load weighting value of each container group in the plurality of container groups, wherein the load weighting value of each container group is used to characterize the number of requests being processed by the container group; The network load balancing module determines at least one first container group among the plurality of container groups that corresponds to the network load balancing module, based on the load weighting value. The network load balancing module transmits the target request received by the network load balancing module to the at least one first container group according to the address information of the at least one first container group.

4. The request processing method according to claim 1, characterized in that, The multiple network load balancing modules respectively transmit the multiple target requests to the multiple container groups, including: For each of the plurality of network load balancing modules, the network load balancing module polls the plurality of container groups to determine a second correspondence between the network load balancing module and at least one second container group among the plurality of container groups; The network load balancing module transmits the target request received by the network load balancing module to the at least one second container group according to the address information of the at least one second container group and in accordance with the second correspondence.

5. The request processing method according to any one of claims 1 to 4, characterized in that, The multiple network load balancing modules respectively transmit the multiple target requests to the multiple container groups, including: The plurality of network load balancing modules respectively perform target network address translation processing and source network address translation processing on the plurality of target requests in sequence; The multiple network load balancing modules transmit the target requests after address translation to the multiple container groups respectively.

6. The request processing method according to any one of claims 1 to 4, characterized in that, After the plurality of network load balancing modules respectively transmit the plurality of target requests to the plurality of container groups, the method further includes: When the multiple container groups generate multiple request results corresponding to the multiple target requests, the multiple container groups respectively perform source network address translation processing and target network address translation processing on the multiple request results in sequence; The multiple container groups transmit the address translation results of multiple requests to the proxy server.

7. The request processing method according to claim 6, characterized in that, After the multiple container groups transmit the address-translated request results to the proxy server, the request processing method further includes: The proxy server transmits the multiple request results to multiple target clients, wherein the multiple target clients are clients that transmit the multiple target requests to the proxy server.

8. A request processing apparatus, characterized in that, An application is provided in a container orchestration platform, the container orchestration platform comprising: a proxy server, multiple network load balancing modules, and multiple container groups, each of the multiple network load balancing modules being connected to the multiple container groups, and the request processing device comprising: The first transmission unit is configured to transmit the multiple target requests to the multiple network load balancing modules respectively when the proxy server receives multiple target requests, wherein each of the multiple network load balancing modules receives a portion of the multiple target requests, and the difference between the number of target requests received by any two of the multiple network load balancing modules is less than a difference threshold. The second transmission unit is used to transmit the plurality of target requests to the plurality of container groups respectively, wherein each of the plurality of container groups is used to process a portion of the plurality of target requests.

9. An electronic device, characterized in that, include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images