Container task scheduling method and system based on reinforcement learning technology and storage medium

Abstract

The invention relates to the field of deep learning, in particular to a container task scheduling method and system based on the reinforcement learning technology and a storage medium, and the method comprises the steps: obtaining container task scheduling data; constructing a scheduling optimization problem based on the minimum task waiting time and the minimum cluster resource imbalance degree; constructing a reinforcement learning model, and training the reinforcement learning model according to the scheduling optimization problem and the container task scheduling data to obtain a scheduling model; the method specifically comprises the following steps: filtering data obtained in a training process during training, and then training to obtain a scheduling model; and carrying out container task scheduling. According to the method, a scheduling optimization problem based on the minimum task waiting time and the minimum cluster resource imbalance degree is constructed; filtering the data obtained in the training process, and then training to obtain a scheduling model; therefore, the training effect is improved, and finally the container task scheduling effect is improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning, and more specifically, to a container task scheduling method and system based on reinforcement learning technology. Background Art

[0002] With the close integration of cloud computing and container technology, container cloud platforms have also emerged. The resource scheduler in the container cloud greatly affects the performance of the cluster. Among them, resource scheduling is the process of properly allocating, planning and coordinating various virtualized resources according to certain rules to meet user needs. In the container cloud platform, when there are a large number of containers to be managed, it is very important to allocate resources to containers and schedule containers to appropriate nodes. For complex production environments, the existing scheduling algorithms are relatively simple. Although they are easy to implement, they lack flexibility. For the large number of container instances running in the existing container platform, complex dependencies, and fast version iterations, simple scheduling algorithms cannot schedule tasks with maximum efficiency, which affects the scheduling effect.

[0003] The prior art discloses a method for optimizing resource scheduling of container clouds for multi-layer services, which includes the following steps: constructing a container cloud architecture consisting of multiple heterogeneous containers according to the demand for providing services for multiple multi-layer services; establishing the overall revenue function of the cloud service provider for the container cloud according to the architecture; providing the response status of each layer of services under the multi-layer service according to the queuing theory, combining the revenue function to obtain the overall revenue model, and setting the initial solution for the revenue model according to the service requirements of each layer of services; providing a resource allocation optimization algorithm for the optimization problem of the above model, and then allocating resources according to the solution of the algorithm, so as to maximize the benefits of the cloud service provider. However, this method cannot adjust the strategy in real time according to the specific situation, thus affecting the scheduling effect. Summary of the invention

[0004] The purpose of the present invention is to disclose a container task scheduling method, system and storage medium based on reinforcement learning technology with better scheduling effect.

[0005] In order to achieve the above object, the present invention provides a container task scheduling method based on reinforcement learning technology, comprising: S1: obtaining container task scheduling data;

[0006] S2: Based on the container task scheduling data, a scheduling optimization problem based on minimizing task waiting time and minimizing the imbalance of cluster resources is constructed;

[0007] S3: Build a reinforcement learning model, train the reinforcement learning model according to the scheduling optimization problem and the container task scheduling data to obtain a scheduling model; specifically, filter the data obtained during the training process, and then train to obtain a scheduling model;

[0008] S4: Schedule container tasks according to the scheduling model.

[0009] Furthermore, in step S2, it includes:

[0010] The task waiting time penalty and the system node resource imbalance penalty are combined to transform the scheduling process into the process of finding the optimal solution for the task node combination: the one-to-many matching process of tasks and nodes is transformed into the many-to-many matching process of tasks and nodes; in the cluster N = {N1, N2, …, N n}, there are n nodes in total, and the task is recorded as T j , where the task queue at a certain moment is recorded as Q = {T1, T2, ..., T j ,…,T m}, waiting to be scheduled task T j The resource requirement is expressed as When the task T j Scheduled to node N i When on, you need to meet the task T j The application amount for each resource is less than the node N i The amount of available resources for the cth scheduling in the current system satisfies:

[0011]

[0012] According to the characteristics of the task, the task that meets the scheduling requirements is selected from the task queue and scheduled to the appropriate node according to the scheduling strategy. Taking into account the different delay sensitivities of different tasks, different weight coefficients are given to different task types. Assume r W (c) Select task T when the scheduler performs the cth round of task scheduling j The waiting time penalty obtained by scheduling to a suitable node for execution is calculated as follows:

[0013]

[0014] Among them, W in the formula j,c Represents task T j Waiting time for node N i , in the cth round of scheduling, the overall resource utilization can be expressed as Different resource types can be set with different overall utilization calculation methods; the imbalance degree of resource usage within a node is expressed as

[0015]

[0016] The degree of imbalance in resource usage between system nodes It is expressed as:

[0017]

[0018] For selecting task T from the task queue j Schedule to node N i The reward function r executed i,j (c) is expressed as:

[0019]

[0020] Among them, γ w , γ bs and γ bn is the corresponding weight parameter.

[0021] The scheduling optimization problem based on minimizing task waiting time and minimizing the imbalance of cluster resources is defined as:

[0022]

[0023] Further, in step S3, the reinforcement learning model is an Actor-Critic network model;

[0024] The Actor-Critic network model combines the two methods of policy gradient and value function. The basic structure of the Actor-Critic network model includes:

[0025] Actor: Actor consists of a policy network and is responsible for selecting actions. Actor generates a policy based on the current state, which is the probability distribution of selecting an action from a given state.

[0026] Critic: Actor consists of a value network and is responsible for evaluating the quality of the selected action; Critic estimates the value of a state or state-action pair through a value function, that is, the expected return of choosing an action in the current state.

[0027] Further, it includes: in step S3, training the reinforcement learning model according to the scheduling optimization problem and the container task scheduling data to obtain the scheduling model includes: using the asynchronous advantage actor-critic algorithm to train the reinforcement learning model according to the scheduling optimization problem to obtain the scheduling model.

[0028] Further, including:

[0029] The asynchronous advantage actor-critic algorithm contains a main thread and multiple auxiliary threads. First, the parameters θ1 of the Actor and θ2 of the Critic in the main thread are randomly initialized;

[0030] For each auxiliary thread, synchronize the parameters of the main thread Actor and Critic; get the state s according to the container task scheduling datac ; Perform mask operation on the policy gradient output by the policy network to filter out actions that do not meet the scheduling conditions;

[0031] The policy network is based on the state s c Get action output a c =[j,i], execute action a c , that is, task T j Schedule to node N i On, complete the scheduling process and get the state s c+1 ;

[0032] Calculate the reward according to the reward function of the scheduling optimization problem, and update the parameters θ1 of the Actor and θ2 of the Critic in the auxiliary thread;

[0033] If the main thread parameters have been updated synchronously for g times, the auxiliary thread parameters are merged and the main thread parameters are updated;

[0034] Repeat the above steps until the maximum number of training rounds is reached and the scheduling model is obtained.

[0035] Furthermore, the state s obtained based on the container task scheduling data c Includes: Status c Including cluster node characteristics and schedulable task characteristics; to ensure the fixedness of the state space dimension, the number of tasks in the fixed state space is m; when the number of schedulable tasks is less than the fixed number of tasks, The way to fill the task; in terms of cluster characteristics, including the maximum available amount of each resource of each node, expressed as And the allocatable amount of each resource of each node under the cth scheduling time state is specifically expressed as The resource utilization rate of each node at the cth scheduling time is expressed as U c =[U 1,c ,…,U n,c ]; In terms of task characteristics, R c Represents the schedulable tasks, including the resource request amount of the task under the cth scheduling time state, that is, W c =[W 1,c ,…,W m,c ] contains m components, representing the waiting time of each task, Each component in represents the delay sensitivity weight coefficient corresponding to the task type.

[0036] In summary, c The status of the cth scheduling is:

[0037]

[0038] Furthermore, the policy network is based on the state s c Get action output a c =[j,i] includes: selecting one of the tasks to be scheduled from the schedulable task queue and scheduling it to a certain node; let a c =(a T ,a N ), where a T ∈{1,2,…,m} represents the task number in the task queue to be scheduled, a N ∈{1,2,…,n} represents the node number; a c =[j,i] represents selecting task T from the task queue j Schedule to node N i Execute on; the cluster has n nodes in total, maintaining a task queue of length m, then there are n·m possible actions; a c =(i,k) as an example, it means that the task numbered i in the task queue is scheduled to the cluster node numbered k.

[0039] Furthermore, filtering the container task scheduling data to obtain usable data includes: performing a mask operation on the policy gradient output by the policy network, filtering out actions that do not meet the scheduling conditions, and reassigning the policy gradient logits in the policy network to the value of conditional element multiplication of the policy gradient and a mask binary matrix M of the same dimension:

[0040]

[0041] The symbol ★ represents conditional element-wise multiplication, where the value M in the i-th row and j-th column of the mask matrix i,j :

[0042]

[0043] In addition, the present invention also provides a container task scheduling system based on reinforcement learning technology, including:

[0044] Acquisition module: obtain container task scheduling data;

[0045] Problem module: Construct a scheduling optimization problem based on minimizing task waiting time and minimizing cluster resource imbalance according to container task scheduling data;

[0046] Model training module: builds a reinforcement learning model, trains the reinforcement learning model according to the scheduling optimization problem and container task scheduling data, and obtains a scheduling model; specifically, it includes filtering the data obtained during the training process, and then training to obtain a scheduling model;

[0047] Scheduling module: Schedule container tasks according to the scheduling model.

[0048] In addition, the present invention also provides a container task scheduling storage medium based on reinforcement learning technology, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned container task scheduling method based on reinforcement learning technology is implemented.

[0049] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0050] The present invention constructs a scheduling optimization problem based on minimizing task waiting time and minimizing the imbalance of cluster resources; and filters the data obtained during the training process, and then trains to obtain a scheduling model; thereby improving the training effect and ultimately improving the container task scheduling effect. And according to the training, the scheduling strategy can be adjusted in real time to improve the scheduling effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of the container task scheduling method based on reinforcement learning technology described in Example 1;

[0052] Figure 2 This is a schematic diagram of the mask operation described in Example 2;

[0053] Figure 3 This is a block diagram of a container task scheduling system based on reinforcement learning technology as described in Example 3; DETAILED DESCRIPTION

[0054] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0055] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0056] Embodiment 1:

[0057] This embodiment provides Figure 1 The container task scheduling method based on reinforcement learning technology includes:

[0058] S1: Obtain container task scheduling data;

[0059] S2: Based on the container task scheduling data, a scheduling optimization problem based on minimizing task waiting time and minimizing the imbalance of cluster resources is constructed;

[0060] S3: Build a reinforcement learning model, train the reinforcement learning model according to the scheduling optimization problem and the container task scheduling data to obtain a scheduling model; specifically, filter the data obtained during the training process, and then train to obtain a scheduling model;

[0061] S4: Schedule container tasks according to the scheduling model.

[0062] This embodiment constructs a scheduling optimization problem based on minimizing task waiting time and minimizing the imbalance of cluster resources; and filters the data obtained during the training process, and then trains to obtain a scheduling model; thereby improving the training effect and ultimately improving the container task scheduling effect. And according to the training, the scheduling strategy can be adjusted in real time to improve the scheduling effect.

[0063] Embodiment 2:

[0064] This embodiment further discloses on the basis of the first embodiment:

[0065] In step S2, it includes:

[0066] The task waiting time penalty and the system node resource imbalance penalty are combined to transform the scheduling process into the process of finding the optimal solution for the task node combination: the one-to-many matching process of tasks and nodes is transformed into the many-to-many matching process of tasks and nodes; in the cluster N = {N1, N2, …, N n}, there are n nodes in total, and the task is recorded as T j , where the task queue at a certain moment is recorded as Q = {T1, T2, ..., T j ,…,T m}, waiting to be scheduled task T j The resource requirement is expressed as When the task T j Scheduled to node N i When on, you need to meet the task T j The application amount for each resource is less than the node N i The amount of available resources for the cth scheduling in the current system satisfies:

[0067]

[0068] According to the characteristics of the task, the task that meets the scheduling requirements is selected from the task queue and scheduled to the appropriate node according to the scheduling strategy. Taking into account the different delay sensitivities of different tasks, different weight coefficients are given to different task types. Assume r W (c) Select task T when the scheduler performs the cth round of task scheduling j The waiting time penalty obtained by scheduling to a suitable node for execution is calculated as follows:

[0069]

[0070] Among them, W in the formula j,c Represents task T j Waiting time for node Ni , in the cth round of scheduling, the overall resource utilization can be expressed as Different resource types can be set with different overall utilization calculation methods; the imbalance degree of resource usage within a node is expressed as

[0071]

[0072] The degree of imbalance in resource usage between system nodes It is expressed as:

[0073]

[0074] For selecting task T from the task queue j Schedule to node N i The reward function r executed i,j (c) is expressed as:

[0075]

[0076] Among them, γ w , γ bs and γ bn is the corresponding weight parameter.

[0077] The scheduling optimization problem based on minimizing task waiting time and minimizing the imbalance of cluster resources is defined as:

[0078]

[0079] In step S3, the reinforcement learning model is an Actor-Critic network model;

[0080] The Actor-Critic network model combines the two methods of policy gradient and value function. The basic structure of the Actor-Critic network model includes:

[0081] Actor: Actor consists of a policy network and is responsible for selecting actions. Actor generates a policy based on the current state, which is the probability distribution of selecting an action from a given state.

[0082] Critic: Actor consists of a value network and is responsible for evaluating the quality of the selected action; Critic estimates the value of a state or state-action pair through a value function, that is, the expected return of choosing an action in the current state.

[0083] The method comprises: in step S3, training a reinforcement learning model according to a scheduling optimization problem and container task scheduling data to obtain a scheduling model; and the method comprises: using an asynchronous advantage actor-critic algorithm to train a reinforcement learning model according to a scheduling optimization problem to obtain a scheduling model.

[0084] include:

[0085] The asynchronous advantage actor-critic algorithm contains a main thread and multiple auxiliary threads. First, the parameters θ1 of the Actor and θ2 of the Critic in the main thread are randomly initialized;

[0086] For each auxiliary thread, synchronize the parameters of the main thread Actor and Critic; get the state s according to the container task scheduling data c ; Perform mask operation on the policy gradient output by the policy network to filter out actions that do not meet the scheduling conditions;

[0087] The policy network is based on the state s c Get action output a c =[j,i], execute action a c , that is, task T j Schedule to node N i On, complete the scheduling process and get the state s c+1 ;

[0088] Calculate the reward according to the reward function of the scheduling optimization problem, and update the parameters θ1 of the Actor and θ2 of the Critic in the auxiliary thread;

[0089] If the main thread parameters have been updated synchronously for g times, the auxiliary thread parameters are merged and the main thread parameters are updated;

[0090] Repeat the above steps until the maximum number of training rounds is reached and the scheduling model is obtained.

[0091] The status s obtained based on the container task scheduling data c Includes: Status c Including cluster node characteristics and schedulable task characteristics; to ensure the fixedness of the state space dimension, the number of tasks in the fixed state space is m; when the number of schedulable tasks is less than the fixed number of tasks, The way to fill the task; in terms of cluster characteristics, including the maximum available amount of each resource of each node, expressed as And the allocatable amount of each resource of each node under the cth scheduling time state is specifically expressed as The resource utilization rate of each node at the cth scheduling time is expressed as U c =[U 1,c ,…,U n,c ]; In terms of task characteristics, R c Represents the schedulable tasks, including the resource request amount of the task under the cth scheduling time state, that is, W c =[W 1,c,…,W m,c ] contains m components, representing the waiting time of each task, Each component in represents the delay sensitivity weight coefficient corresponding to the task type.

[0092] In summary, c The status of the cth scheduling is:

[0093]

[0094] The policy network is based on the state s c Get action output a c =[j,i] includes: selecting one of the tasks to be scheduled from the schedulable task queue and scheduling it to a certain node; let a c =(a T ,a N ), where a T ∈{1,2,…,m} represents the task number in the task queue to be scheduled, a N ∈{1,2,…,n} represents the node number; a c =[j,i] represents selecting task T from the task queue j Schedule to node N i Execute on; the cluster has n nodes in total, maintaining a task queue of length m, then there are n·m possible actions; a c =(i,k) as an example, it means that the task numbered i in the task queue is scheduled to the cluster node numbered k.

[0095] The available data obtained by filtering the container task scheduling data include: the policy gradient output by the policy network, such as Figure 2 As shown, a mask operation is performed to filter out actions that do not meet the scheduling conditions, and the policy gradient logits in the policy network are reassigned to the value of conditional element multiplication of the policy gradient and a mask binary matrix M of the same dimension:

[0096]

[0097] The symbol ★ represents conditional element-wise multiplication, where the value M in the i-th row and j-th column of the mask matrix i,j :

[0098]

[0099] This embodiment constructs a scheduling optimization problem based on minimizing task waiting time and minimizing the imbalance of cluster resources; and filters the data obtained during the training process, and then trains to obtain a scheduling model; thereby improving the training effect and ultimately improving the container task scheduling effect. And according to the training, the scheduling strategy can be adjusted in real time to improve the scheduling effect.

[0100] Embodiment three:

[0101] This embodiment provides Figure 3 The container task scheduling system based on reinforcement learning technology shown in the figure includes:

[0102] Acquisition module: obtain container task scheduling data;

[0103] Problem module: Construct a scheduling optimization problem based on minimizing task waiting time and minimizing cluster resource imbalance according to container task scheduling data;

[0104] Model training module: builds a reinforcement learning model, trains the reinforcement learning model according to the scheduling optimization problem and container task scheduling data, and obtains a scheduling model; specifically, it includes filtering the data obtained during the training process, and then training to obtain a scheduling model;

[0105] Scheduling module: Schedule container tasks according to the scheduling model.

[0106] This embodiment constructs a scheduling optimization problem based on minimizing task waiting time and minimizing the imbalance of cluster resources; and filters the data obtained during the training process, and then trains to obtain a scheduling model; thereby improving the training effect and ultimately improving the container task scheduling effect. And according to the training, the scheduling strategy can be adjusted in real time to improve the scheduling effect.

[0107] Embodiment 4:

[0108] This embodiment also provides a container task scheduling storage medium based on reinforcement learning technology, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned container task scheduling method based on reinforcement learning technology is implemented.

[0109] This embodiment constructs a scheduling optimization problem based on minimizing task waiting time and minimizing the imbalance of cluster resources; and filters the data obtained during the training process, and then trains to obtain a scheduling model; thereby improving the training effect and ultimately improving the container task scheduling effect. And according to the training, the scheduling strategy can be adjusted in real time to improve the scheduling effect.

[0110] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A container task scheduling method based on reinforcement learning technology, characterized in that: include: S1: Obtain container task scheduling data; S2: Based on the container task scheduling data, a scheduling optimization problem based on minimizing task waiting time and minimizing the imbalance of cluster resources is constructed; S3: Build a reinforcement learning model, train the reinforcement learning model according to the scheduling optimization problem and the container task scheduling data to obtain a scheduling model; specifically, filter the data obtained during the training process, and then train to obtain a scheduling model; S4: Schedule container tasks according to the scheduling model.

2. The container task scheduling method based on reinforcement learning technology according to claim 1 is characterized in that: In step S2, it includes: The task waiting time penalty and the system node resource imbalance penalty are combined to transform the scheduling process into the process of finding the optimal solution for the task node combination: the one-to-many matching process of tasks and nodes is transformed into the many-to-many matching process of tasks and nodes; in the cluster N = {N1, N2, …, N n }, there are n nodes in total, and the task is recorded as T j , where the task queue at a certain moment is recorded as Q = {T1, T2, ..., T j ,…,T m }, waiting to be scheduled task T j The resource requirement is expressed as When the task T j Scheduled to node N i When on, you need to meet the task T j The application amount for each resource is less than the node N i The amount of available resources for the cth scheduling in the current system satisfies: According to the characteristics of the task, the task that meets the scheduling requirements is selected from the task queue and scheduled to the appropriate node according to the scheduling strategy. Taking into account the different delay sensitivities of different tasks, different weight coefficients are given to different task types. Assume r w (c) Select task T when the scheduler performs the cth round of task scheduling j The waiting time penalty obtained by scheduling to a suitable node for execution is calculated as follows: Among them, W in the formula j,c Represents task T j Waiting time for node N i , in the cth round of scheduling, the overall resource utilization can be expressed as Different resource types can be set with different overall utilization calculation methods; the imbalance degree of resource usage within a node is expressed as The degree of imbalance in resource usage between system nodes It is expressed as: For selecting task T from the task queue j Schedule to node N i The reward function r executed i,j (c) is expressed as: Among them, γ w , γ bs and γ bn is the corresponding weight parameter; The scheduling optimization problem based on minimizing task waiting time and minimizing the imbalance of cluster resources is defined as:

3. The container task scheduling method based on reinforcement learning technology according to claim 1 is characterized in that: In step S3, the reinforcement learning model is an Actor-Critic network model; The Actor-Critic network model combines the two methods of policy gradient and value function. The basic structure of the Actor-Critic network model includes: Actor: Actor consists of a policy network and is responsible for selecting actions. Actor generates a policy based on the current state, which is the probability distribution of selecting an action from a given state. Critic: Actor consists of a value network and is responsible for evaluating the quality of the selected action; Critic estimates the value of a state or state-action pair through a value function, that is, the expected return of choosing an action in the current state.

4. The container task scheduling method based on reinforcement learning technology according to claim 3 is characterized in that: include: In step S3, the reinforcement learning model is trained according to the scheduling optimization problem and the container task scheduling data to obtain the scheduling model, including: using the asynchronous advantage actor-critic algorithm to train the reinforcement learning model according to the scheduling optimization problem to obtain the scheduling model.

5. The container task scheduling method based on reinforcement learning technology according to claim 3 is characterized in that: include: The asynchronous advantage actor-critic algorithm contains a main thread and multiple auxiliary threads. First, the parameters θ1 of the Actor and θ2 of the Critic in the main thread are randomly initialized; For each auxiliary thread, synchronize the parameters of the main thread Actor and Critic; get the state s according to the container task scheduling data c ; Perform mask operation on the policy gradient output by the policy network to filter out actions that do not meet the scheduling conditions; The policy network is based on the state s c Get action output a c =[j,i], execute action a c , that is, task T j Schedule to node N i On, complete the scheduling process and get the state s c+1 ; Calculate the reward according to the reward function of the scheduling optimization problem, and update the parameters θ1 of the Actor and θ2 of the Critic in the auxiliary thread; If the main thread parameters have been updated synchronously for g times, the auxiliary thread parameters are merged and the main thread parameters are updated; Repeat the above steps until the maximum number of training rounds is reached and the scheduling model is obtained.

6. The container task scheduling method based on reinforcement learning technology according to claim 5 is characterized in that: The status s obtained based on the container task scheduling data c Includes: Status c Including cluster node characteristics and schedulable task characteristics; to ensure the fixedness of the state space dimension, the number of tasks in the fixed state space is m; when the number of schedulable tasks is less than the fixed number of tasks, The way to fill the task; in terms of cluster characteristics, including the maximum available amount of each resource of each node, expressed as And the allocatable amount of each resource of each node under the cth scheduling time state is specifically expressed as The resource utilization rate of each node at the cth scheduling time is expressed as U c =[U 1,c ,…,U n,c ]; In terms of task characteristics, R c Represents the schedulable tasks, including the resource request amount of the task under the cth scheduling time state, that is, W c =[W 1,c ,…,W m,c ] contains m components, representing the waiting time of each task, Each component in represents the delay sensitivity weight coefficient corresponding to the task type; In summary, c The status of the cth scheduling is:

7. The container task scheduling method based on reinforcement learning technology according to claim 5 is characterized in that: The policy network is based on the state s c Get action output a c =[j,i] includes: selecting one of the tasks to be scheduled from the schedulable task queue and scheduling it to a certain node; let a c =(a T ,a N ), where a T ∈{1,2,…,m} represents the task number in the task queue to be scheduled, a N ∈{1,2,…,n} represents the node number; a c =[j,i] represents selecting task T from the task queue j Schedule to node N i Execute on; the cluster has n nodes in total, maintaining a task queue of length m, then there are n·m possible actions; a c =(i,k) as an example, it means that the task numbered i in the task queue is scheduled to the cluster node numbered k.

8. The container task scheduling method based on reinforcement learning technology according to claim 5 is characterized in that: During training, the data obtained during the training process is filtered, and then training is performed to obtain a scheduling model, including: masking the policy gradient output by the policy network, filtering out actions that do not meet the scheduling conditions, and reassigning the policy gradient logits in the policy network to the value of conditional element multiplication of the policy gradient and a mask binary matrix M of the same dimension: The symbol ★ represents conditional element-wise multiplication, where the value M in the i-th row and j-th column of the mask matrix i,j :

9. A container task scheduling system based on reinforcement learning technology, characterized in that: include: Acquisition module: obtain container task scheduling data; Problem module: Construct a scheduling optimization problem based on minimizing task waiting time and minimizing cluster resource imbalance according to container task scheduling data; Model training module: builds a reinforcement learning model, trains the reinforcement learning model according to the scheduling optimization problem and container task scheduling data, and obtains a scheduling model; specifically, it includes filtering the data obtained during the training process, and then training to obtain a scheduling model; Scheduling module: Schedule container tasks according to the scheduling model.

10. A container task scheduling storage medium based on reinforcement learning technology, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the container task scheduling method based on reinforcement learning technology as described in any one of claims 1 to 8 is implemented.

Images