Resource scheduling method, device, electronic device and storage medium
By optimizing task allocation through a resource scheduling model, the problem of underutilization of resources in distributed systems is solved, and more efficient resource utilization is achieved.
Patent Information
- Application Number
- CN202110875335.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-30
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-07-30
AI Technical Summary
In distributed systems, existing resource scheduling strategies lead to a backlog of training tasks and underutilization of cluster resources.
A resource scheduling model is adopted, which is trained through a value network and a target network, and combines action rewards based on resource uniformity and saturation to optimize task allocation.
It enables the cluster to undertake more tasks with the same hardware resources, rationally allocate resources, and improve the efficiency of cluster resource utilization.
Smart Images

Figure CN113515385B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a resource scheduling method, device, electronic equipment and storage medium. Background Art
[0002] With the development and application of artificial intelligence technology, the demand for computing power has become increasingly greater. Currently, a common practice is to build a distributed system cluster and then train models based on the cluster. For a cluster, resource scheduling strategy is one of its core functions.
[0003] At present, in distributed systems, the resource scheduling strategy is still to allocate physical nodes to submitted tasks based on defined rules. In practical applications, as the amount of submitted model training tasks increases, more and more training tasks are accumulated, while the actual cluster resources are not fully utilized.
[0004] In summary, there is an urgent need for a resource scheduling technology to solve the problems existing in the above-mentioned existing technologies. Summary of the invention
[0005] Since the existing methods have the above problems, the present invention proposes a method, device, electronic device and storage medium for resource scheduling.
[0006] In a first aspect, the present invention provides a method for resource scheduling, comprising:
[0007] Obtaining a task and a current environment state; the task includes resource requirements; the current environment state includes the current load state of multiple physical nodes;
[0008] Inputting the task and the current environment state into a trained resource scheduling model to obtain action description information for deploying the task;
[0009] Deploy the task to the physical node corresponding to the action description information according to the action description information;
[0010] The trained resource scheduling model is obtained by training using different tasks and different environmental states.
[0011] Furthermore, the resource scheduling model includes a value network and a target network, and before the task and the current environment state are input into the trained resource scheduling model to obtain the action description information for deploying the task, it also includes:
[0012] Obtain a preset number of training sample sets; each set of training samples includes a first environment state, action description information, a second environment state, and an action reward; the first environment state is the load state of each physical node before executing the action description information; the second environment state is the load state of each physical node after executing the action description information; the action reward is the reward value corresponding to executing the action description information;
[0013] Inputting the first environmental state and the action description information into the value network to obtain a first function value;
[0014] Inputting the second environment state into the target network to obtain a second function value;
[0015] Determine a loss function according to the first function value and the second function value;
[0016] The parameters of the resource scheduling model are updated according to the loss function to obtain a trained resource scheduling model.
[0017] Further, updating the parameters of the resource scheduling model according to the loss function includes:
[0018] Updating parameters of the value network according to the loss function;
[0019] It is determined whether the update times of the value network reaches a preset threshold. If so, the parameters of the value network are assigned to the parameters of the target network.
[0020] Further, the value network includes a first network and a second network; the step of inputting the first environment state and the action description information into the value network to obtain a first function value includes:
[0021] Inputting the first environment state and the task into the first network to obtain the action description information;
[0022] The first environmental state and the action description information are input into the second network to obtain the first function value.
[0023] Furthermore, the obtaining of a preset number of training sample sets includes:
[0024] Obtain resource uniformity and resource saturation corresponding to the action description information;
[0025] An action reward corresponding to the action description information is determined according to the resource uniformity and the resource saturation.
[0026] Further, the determining the action reward corresponding to the action description information according to the resource uniformity and the resource saturation includes:
[0027] The action reward corresponding to the action description information is obtained by performing weighted summation on the resource uniformity and the resource saturation.
[0028] Furthermore, the task includes memory requirement, disk requirement and video memory requirement.
[0029] In a second aspect, the present invention provides a resource scheduling device, comprising:
[0030] An acquisition module is used to acquire tasks and current environment status; the tasks include resource requirements; the current environment status includes current load status of multiple physical nodes;
[0031] A processing module is used to input the task and the current environment state into a trained resource scheduling model to obtain action description information for deploying the task; and deploy the task to a physical node corresponding to the action description information according to the action description information; wherein the trained resource scheduling model is obtained after training using different tasks and different environment states.
[0032] Furthermore, the resource scheduling model includes a value network and a target network, and the processing module is further used for:
[0033] Before inputting the task and the current environment state into the trained resource scheduling model to obtain the action description information for deploying the task, a preset number of training sample sets are obtained; each set of training samples includes a first environment state, action description information, a second environment state, and an action reward; the first environment state is the load state of each physical node before executing the action description information; the second environment state is the load state of each physical node after executing the action description information; the action reward is the reward value corresponding to executing the action description information;
[0034] Inputting the first environmental state and the action description information into the value network to obtain a first function value;
[0035] Inputting the second environment state into the target network to obtain a second function value;
[0036] Determine a loss function according to the first function value and the second function value;
[0037] The parameters of the resource scheduling model are updated according to the loss function to obtain a trained resource scheduling model.
[0038] Furthermore, the processing module is specifically used for:
[0039] Updating parameters of the value network according to the loss function;
[0040] It is determined whether the update times of the value network reaches a preset threshold. If so, the parameters of the value network are assigned to the parameters of the target network.
[0041] Furthermore, the value network includes a first network and a second network; and the processing module is specifically used for:
[0042] Inputting the first environment state and the task into the first network to obtain the action description information;
[0043] The first environmental state and the action description information are input into the second network to obtain the first function value.
[0044] Furthermore, the processing module is specifically used for:
[0045] Obtain resource uniformity and resource saturation corresponding to the action description information;
[0046] An action reward corresponding to the action description information is determined according to the resource uniformity and the resource saturation.
[0047] Furthermore, the processing module is specifically used for:
[0048] The action reward corresponding to the action description information is obtained by performing weighted summation on the resource uniformity and the resource saturation.
[0049] Furthermore, the processing module is specifically used for: the task includes memory requirement, disk requirement and video memory requirement.
[0050] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the resource scheduling method as described in the first aspect when executing the computer program.
[0051] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the resource scheduling method as described in the first aspect.
[0052] It can be seen from the above technical solutions that the resource scheduling method, device, electronic device and storage medium provided by the present invention, by setting action rewards based on resource uniformity and resource saturation, take into account the uniformity of resource allocation and the resource saturation of a single node, so that more tasks can be undertaken under the same hardware resources, thereby realizing the reasonable allocation of submitted tasks and achieving effective utilization of cluster resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0054] Figure 1 A system framework for the resource scheduling method provided by the present invention;
[0055] Figure 2 A schematic diagram of a flow chart of a resource scheduling method provided by the present invention;
[0056] Figure 3 A flow chart of the training resource scheduling model provided by the present invention;
[0057] Figure 4 A schematic diagram of a value network in the resource scheduling method provided by the present invention;
[0058] Figure 5 A schematic diagram of the structure of a resource scheduling device provided by the present invention;
[0059] Figure 6 This is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION
[0060] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.
[0061] The resource scheduling method provided in the embodiment of the present invention can be applied to Figure 1 In the system architecture shown, the system architecture includes a resource scheduling model 100 and a server 200 .
[0062] Specifically, the resource scheduling model 100 is used to obtain action description information of the deployment task after inputting the task and the current environment status;
[0063] Further, the task is deployed to the server 200 corresponding to the action description information according to the action description information;
[0064] It should be noted that the trained resource scheduling model is obtained after training using different tasks and different environmental states.
[0065] For example, there are three servers in the current environment. After inputting the task and the current environment status, the resource scheduling model 100 obtains the action description information of the deployment task as (1, 0, 0), indicating that the task is deployed to the first server.
[0066] It should be noted that Figure 1 This is only an example of the system architecture of the embodiment of the present invention, and the present invention does not make any specific limitation to this.
[0067] Based on the system architecture shown above, Figure 2 A schematic diagram of a flow chart corresponding to a resource scheduling method provided in an embodiment of the present invention, such as Figure 2 As shown, the method includes:
[0068] Step 201, obtain the task and current environment status.
[0069] It should be noted that the task includes resource requirements; the current environment status includes the current load status of multiple physical nodes.
[0070] In a possible implementation, the task includes memory requirement, disk requirement, and video memory requirement.
[0071] In the embodiment of the present invention, the task may also include network bandwidth demand, disk I / O throughput, etc., which is not specifically limited in the embodiment of the present invention.
[0072] Taking memory requirement, disk requirement and video memory requirement as an example, for a task χ, χ∈Γ, where Γ is the combination of all tasks.
[0073] In the embodiment of the present invention, for the memory space b(0 <b≤B,b∈N + )、Disk space c(0 <c≤C,c∈N + ) and video memory size d(0≤d≤D,d∈N + ), Γ has a total of B*C*(D+1) combinations.
[0074] It should be noted that, among them, N + is a positive integer. The units of the above memory space, disk space, and video memory size are all megabytes.
[0075] In one possible implementation, the current environment state Indicates the state of the i-th physical node at time t, that is, its remaining memory resources, disk resources, and video memory resources, as follows:
[0076]
[0077] It should be noted that, They respectively represent the remaining memory, disk and video memory of the i-th physical node at time t.
[0078] In the embodiment of the present invention, three main resources, namely, memory space, disk space, and video memory size, are considered, which simplifies the problem and improves the efficiency of resource scheduling.
[0079] Step 202, input the task and the current environment status into the trained resource scheduling model to obtain the action description information of the deployment task.
[0080] Step 203: deploy the task to the corresponding physical node according to the action description information.
[0081] For example, there are four physical nodes in the current environment, and the action description information output by the resource scheduling model is (0, 0, 0, 1), indicating that the task is deployed to the fourth physical node.
[0082] It should be noted that the trained resource scheduling model is obtained after training using different tasks and different environmental states.
[0083] The above scheme obtains the action description information of the deployed task by inputting the task and the current environment status into the trained resource scheduling model, thereby realizing the reasonable allocation of the submitted tasks and achieving the effective utilization of cluster resources.
[0084] In the embodiment of the present invention, before step 202, the steps are as follows: Figure 3 As shown, the details are as follows:
[0085] Step 301: Obtain a preset number of training sample sets.
[0086] It should be noted that each set of training samples includes a first environment state, action description information, a second environment state, and an action reward.
[0087] Specifically, the first environment state is the load state of each physical node before executing the action description information; the second environment state is the load state of each physical node after executing the action description information; and the action reward is the reward value corresponding to the execution of the action description information.
[0088] In the embodiment of the present invention, the process of generating the training sample set is as follows:
[0089] S 0 →A 0 →R 1 →S 1 →…→S t-1 →A t-1 →R t →S t →…
[0090] It should be noted that, among them, S t-1 represents the environmental state at time t-1, A t-1Represents the action description information taken at time t-1, R t Represents the action reward obtained at time t.
[0091] Specifically, obtaining resource uniformity and resource saturation corresponding to the action description information;
[0092] The action reward corresponding to the action description information is determined based on resource uniformity and resource saturation.
[0093] In a possible implementation, the action reward corresponding to the action description information is obtained by performing weighted summation on the resource uniformity and the resource saturation.
[0094] In the embodiment of the present invention, the action reward is mainly to ensure two indicators, one is a higher resource uniformity, and the other is a lower resource saturation.
[0095] Furthermore, higher resource uniformity can ensure that tasks are evenly distributed to each physical node, and lower resource saturation can avoid excessive load on the physical nodes.
[0096] In the embodiment of the present invention, the specific formula for obtaining the action reward corresponding to the action description information by weighted summing the resource uniformity φ(z) and the resource saturation ψ(z) is as follows:
[0097] Υ(Γ,S,A)=αφ(Γ,S,A)+βψ(Γ,S,A),0<α,β≤1
[0098] Among them, α and β are the weighting coefficients of resource uniformity φ(z) and resource saturation ψ(z), respectively.
[0099] Furthermore, the specific calculation formula of resource uniformity is as follows:
[0100]
[0101] Among them, avg represents the average value of the vector, ⊙ is the Hadamard product, μ, σ are the average and variance of the memory, disk and video memory usage of each physical node, and ξ is a minimum value.
[0102] Specifically, the specific calculation formulas for μ and σ are as follows:
[0103]
[0104] It should be noted that is the total amount of resource M for the ith physical node, and k is the total number of physical nodes.
[0105] Furthermore, the specific calculation formula of resource saturation in the embodiment of the present invention is as follows:
[0106]
[0107] in, The specific calculation is as follows:
[0108]
[0109] Among them, τ M Indicates the remaining amount of physical nodes tolerated by M resources. Clip(value,min) means that if the value is less than the min value, the value of the function is the min value.
[0110] Step 302: Input the first environment state and action description information into the value network to obtain the first function value.
[0111] In the embodiment of the present invention, a mapping such as Formula 1 is established, specifically as follows:
[0112]
[0113] Among them, a is the action description information, a∈{V 1 …V k},V k represents the kth physical node, a * Describes the optimal action to be selected.
[0114] For example, there are 3 servers in the current environment, a * (0, 0, 1), indicating that the task is deployed on the third physical node.
[0115] It should be noted that the first function value is the output corresponding to the optimal action description information selected by the value network after the first environmental state is input, and the optimal action description information is the action description information that maximizes the first function value.
[0116] Furthermore, in an embodiment of the present invention, the value network includes a first network and a second network.
[0117] Specifically, in step 302, the first environment state and the task are input into the first network to obtain action description information;
[0118] The first environment state and action description information are input into the second network to obtain a first function value.
[0119] Furthermore, if Figure 4 As shown, state vector is the current environment state vector, H is the hidden layer, O is the output layer, and in the embodiment of the present invention, the environment state at time t is expressed as follows:
[0120]
[0121] It should be noted that, Represents the state of the kth physical node at time t.
[0122] In one possible implementation, Indicates the status of the i-th physical node at time t, namely its remaining memory resources, disk resources, and video memory resources, as follows:
[0123]
[0124] It should be noted that, They respectively represent the remaining memory, disk and video memory of the i-th physical node at time t.
[0125] Further, Figure 4 The task vector in is sampled from task Γ, as follows:
[0126]
[0127] It should be noted that, They respectively represent the memory requirement, disk requirement and video memory requirement corresponding to the task at time t.
[0128] Furthermore, actor vector is the action description information probability vector output by the first network, which is as follows:
[0129]
[0130] It should be noted that, among them, p i represents the probability of selecting the i-th physical node.
[0131] In the above scheme, since the action description information is for selecting a server, the action description information is obtained by inputting the first environment state and the task into the first network, and the action description information is associated with the task, thereby realizing reasonable allocation of submitted tasks and achieving effective utilization of cluster resources.
[0132] Step 303: input the second environment state into the target network to obtain a second function value.
[0133] It should be noted that the second function value is the output corresponding to the optimal action description information selected by the target network after the second environment state is input, and the optimal action description information is the action description information that maximizes the second function value.
[0134] Step 304: determine a loss function according to the first function value and the second function value.
[0135] Step 305: Update the parameters of the resource scheduling model according to the loss function to obtain a trained resource scheduling model.
[0136] Specifically, update the parameters of the value network according to the loss function;
[0137] It is determined whether the number of updates of the value network reaches a preset threshold. If so, the parameters of the value network are assigned to the parameters of the target network.
[0138] In the embodiment of the present application, whenever the number of value network updates reaches a preset threshold, the parameters of the value network are assigned to the parameters of the target network.
[0139] In the embodiment of the present invention, the value network and the target network have the same network structure, and a delayed update method is adopted to update the weights of the network structure of the value network to the target network after multiple batches of training.
[0140] The above scheme, by setting action rewards based on resource uniformity and resource saturation, takes into account the uniformity of resource allocation and the resource saturation of a single node, so that more tasks can be undertaken under the same hardware resources, thereby achieving a reasonable allocation of submitted tasks and achieving effective utilization of cluster resources.
[0141] Based on the same inventive concept, Figure 5 A resource scheduling device provided by an embodiment of the present invention is exemplarily shown, and the device may be a process of a resource scheduling method.
[0142] The device comprises:
[0143] The acquisition module 501 is used to acquire tasks and current environment status; the tasks include resource requirements; the current environment status includes current load status of multiple physical nodes;
[0144] Processing module 502 is used to input the task and the current environment state into a trained resource scheduling model to obtain action description information for deploying the task; and deploy the task to a physical node corresponding to the action description information according to the action description information; wherein the trained resource scheduling model is obtained after training using different tasks and different environment states.
[0145] Furthermore, the resource scheduling model includes a value network and a target network, and the processing module 502 is further used for:
[0146] Before inputting the task and the current environment state into the trained resource scheduling model to obtain the action description information for deploying the task, a preset number of training sample sets are obtained; each set of training samples includes a first environment state, action description information, a second environment state, and an action reward; the first environment state is the load state of each physical node before executing the action description information; the second environment state is the load state of each physical node after executing the action description information; the action reward is the reward value corresponding to executing the action description information;
[0147] Inputting the first environmental state and the action description information into the value network to obtain a first function value;
[0148] Inputting the second environment state into the target network to obtain a second function value;
[0149] Determine a loss function according to the first function value and the second function value;
[0150] The parameters of the resource scheduling model are updated according to the loss function to obtain a trained resource scheduling model.
[0151] Furthermore, the processing module 502 is specifically used for:
[0152] Updating parameters of the value network according to the loss function;
[0153] It is determined whether the update times of the value network reaches a preset threshold. If so, the parameters of the value network are assigned to the parameters of the target network.
[0154] Furthermore, the value network includes a first network and a second network; and the processing module is specifically used for:
[0155] Inputting the first environment state and the task into the first network to obtain the action description information;
[0156] The first environmental state and the action description information are input into the second network to obtain the first function value.
[0157] Furthermore, the processing module 502 is specifically used for:
[0158] Obtain resource uniformity and resource saturation corresponding to the action description information;
[0159] An action reward corresponding to the action description information is determined according to the resource uniformity and the resource saturation.
[0160] Furthermore, the processing module 502 is specifically used for:
[0161] The action reward corresponding to the action description information is obtained by performing weighted summation on the resource uniformity and the resource saturation.
[0162] Furthermore, the processing module 502 is specifically used for: the task includes memory requirement, disk requirement and video memory requirement.
[0163] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, see Figure 6 , the electronic device specifically includes the following contents: a processor 601, a memory 602, a communication interface 603 and a communication bus 604;
[0164] The processor 601, the memory 602, and the communication interface 603 communicate with each other through the communication bus 604; the communication interface 603 is used to realize information transmission between various devices;
[0165] The processor 601 is used to call the computer program in the memory 602. When the processor executes the computer program, all the steps of the above-mentioned resource scheduling method are implemented. For example, when the processor executes the computer program, the following steps are implemented: obtaining a task and a current environmental state; the task includes a resource demand; the current environmental state includes the current load state of multiple physical nodes; inputting the task and the current environmental state into a trained resource scheduling model to obtain action description information for deploying the task; and deploying the task to the physical node corresponding to the action description information according to the action description information; wherein the trained resource scheduling model is obtained after training using different tasks and different environmental states.
[0166] Based on the same inventive concept, another embodiment of the present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, all the steps of the above-mentioned resource scheduling method are implemented. For example, when the processor executes the computer program, the following steps are implemented: obtaining a task and a current environmental state; the task includes a resource demand; the current environmental state includes the current load state of multiple physical nodes; the task and the current environmental state are input into a trained resource scheduling model to obtain action description information for deploying the task; according to the action description information, the task is deployed to the physical node corresponding to the action description information; wherein the trained resource scheduling model is obtained after training using different tasks and different environmental states.
[0167] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a user life pattern prediction device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0168] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of the present invention. Those of ordinary skill in the art may understand and implement them without creative effort.
[0169] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a user life pattern prediction device, or a network device, etc.) to execute the user life pattern prediction method described in each embodiment or some parts of the embodiment.
[0170] In addition, in the present invention, terms such as "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0171] In addition, in the present invention, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0172] In addition, in the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A resource scheduling method, characterized in that: include: Get the task and current environment status; The task includes resource requirements, memory requirements, disk requirements and video memory requirements; the current environment status includes the current load status of multiple physical nodes; Inputting the task and the current environment state into a trained resource scheduling model to obtain action description information for deploying the task; Deploy the task to the corresponding physical node according to the action description information; The trained resource scheduling model is obtained by training using different tasks and different environmental states, and the resource scheduling model includes a value network and a target network; Before inputting the task and the current environment state into the trained resource scheduling model to obtain the action description information for deploying the task, the method further includes: Obtain a preset number of training sample sets; each set of training samples includes a first environment state, action description information, a second environment state, and an action reward; the first environment state is the load state of each physical node before executing the action description information; the second environment state is the load state of each physical node after executing the action description information; the action reward is the reward value corresponding to executing the action description information, which is determined based on the resource uniformity and resource saturation corresponding to the action description information; Inputting the first environmental state and the action description information into the value network to obtain a first function value; The value network includes a first network and a second network; the step of inputting the first environment state and the action description information into the value network to obtain a first function value includes: Inputting the first environment state and the task into the first network to obtain the action description information; Inputting the first environmental state and the action description information into the second network to obtain the first function value; Inputting the second environment state into the target network to obtain a second function value; Determine a loss function according to the first function value and the second function value; The parameters of the resource scheduling model are updated according to the loss function to obtain a trained resource scheduling model.
2. The resource scheduling method according to claim 1, characterized in that: The updating of the parameters of the resource scheduling model according to the loss function includes: Updating parameters of the value network according to the loss function; It is determined whether the update times of the value network reaches a preset threshold. If so, the parameters of the value network are assigned to the parameters of the target network.
3. The resource scheduling method according to claim 1, characterized in that: The step of obtaining a preset number of training sample sets includes: Obtain resource uniformity and resource saturation corresponding to the action description information; An action reward corresponding to the action description information is determined according to the resource uniformity and the resource saturation.
4. The resource scheduling method according to claim 3, characterized in that: The determining the action reward corresponding to the action description information according to the resource uniformity and the resource saturation includes: The action reward corresponding to the action description information is obtained by performing weighted summation on the resource uniformity and the resource saturation.
5. A resource scheduling device, characterized in that: include: The acquisition module is used to obtain tasks and the current environment status; The task includes resource requirements, memory requirements, disk requirements and video memory requirements; the current environment status includes the current load status of multiple physical nodes; A processing module, used to input the task and the current environment state into a trained resource scheduling model to obtain action description information for deploying the task; and deploy the task to a physical node corresponding to the action description information according to the action description information; wherein the trained resource scheduling model is obtained by training with different tasks and different environment states, and the resource scheduling model includes a value network and a target network; The processing module is also used to obtain a preset number of training sample sets; each set of training samples includes a first environment state, action description information, a second environment state, and an action reward; the first environment state is the load state of each physical node before executing the action description information; the second environment state is the load state of each physical node after executing the action description information; the action reward is the reward value corresponding to the execution of the action description information, which is determined based on the resource uniformity and resource saturation corresponding to the action description information; the first environment state and the action description information are input into the value network to obtain a first function value; the value network includes a first network and a second network; the inputting of the first environment state and the action description information into the value network to obtain the first function value includes: inputting the first environment state and the task into the first network to obtain the action description information; inputting the first environment state and the action description information into the second network to obtain the first function value; inputting the second environment state into the target network to obtain the second function value; determining a loss function according to the first function value and the second function value; updating the parameters of the resource scheduling model according to the loss function to obtain a trained resource scheduling model.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 4 are implemented.
7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.
