Systems and methods for network load balancing using machine learning

Abstract

Technology related to network load balancing using machine learning is disclosed. Potential imbalances in some load balancing scenarios can be addressed by using a machine learning model to generate resource utilization predictions for requests and performing load balancing operations based on the resource utilization predictions. For example, requests can be stored in a plurality of queues based on the resource utilization predictions. The queues of requests can then be processed by a load balancer. For example, the requests in the plurality of queues can be assigned to a plurality' of servers independently of one another, wherein requests from one queue are assigned to a plurality of servers without regard for the way that requests from another queue are assigned to the plurality of servers, and vice versa. Thus, in at least some scenarios, imbalances in the processing loads of the plurality of the servers can be avoided.

Images