A resource allocation method for parallel training of task planning models

Abstract

The embodiment of the present application is to provide a resource allocation method for parallel training of task planning models, which belongs to the field of computers. The method includes: obtaining the environmental resource occupancy parameters of the hardware resources occupied when the simulation training environment is run once and the model update occupancy parameters of the hardware resources occupied when the task planning model is updated once; based on the environmental resource occupancy parameters and the model update occupancy parameters, a main node is generated on the current hardware and part of the resources configured on the current hardware are allocated to the main node; control parameters are obtained; based on the control parameters and the remaining hardware resources on the current hardware, multiple sub-nodes are generated, and the remaining hardware resources on the current hardware include the number of remaining CPUs, the number of remaining GPUs, the remaining resources of each CPU, and the remaining resources of each GPU; the remaining hardware resources are allocated to multiple sub-nodes. The present application aims to solve the problem of resource allocation in the task planning model training process.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and in particular, to a resource allocation method for parallel training of a task planning model. Background Art

[0002] In recent years, driven by both data and computing power, artificial intelligence technology has made outstanding progress in solving application problems related to feature extraction and expression, and has played an important role in environmental pollution monitoring and emergency rescue. At present, drones are generally used for environmental pollution monitoring and emergency rescue, because the specific situation of environmental pollution and emergency rescue is generally unknown in advance. Therefore, in the process of using drones to monitor the environment, it is necessary to train a mission planning model to plan the flight path and destination of the drone.

[0003] For the problem of UAV mission planning, the training environment can usually only be constructed through simulation. The accuracy of the simulation model and data is difficult to define. Even if a simulation model with extremely high fidelity is established, the training efficiency is low in the model training because the fine model is usually computationally complex and has little use value. Sometimes, in order to meet the training efficiency requirements, the fidelity of the model must be reduced, resulting in a decrease in the effectiveness of the model. In addition, in order to make the model more robust, it is necessary to add various possible random disturbance factors to the simulation training environment. The more complete and larger the range of random disturbance considerations, the better the model training effect will be in theory. However, even so, in actual applications, it is possible to encounter situations that exceed the training environment settings, and it is necessary to quickly adjust the training environment and retrain the model on the spot.

[0004] The deep reinforcement learning technology is used to construct a task planning model under complex multi-constraints. The training convergence itself is very time-consuming. If we consider adding a certain amount of sophisticated models and adding more random disturbance factors to the simulation training environment mentioned above, the training time will increase significantly, and it will be even more unable to support the on-the-spot retraining of the model in actual application. In summary, the serious training time consumption is one of the important factors restricting the practical application of task planning models based on deep reinforcement learning technology.

[0005] To this end, researchers proposed a parallel training architecture to solve the above problems. The more typical methods are A3C and A2C. The A3C method constructs a main node and multiple child nodes. The child nodes independently train the model and synchronize the model parameters trained by the child nodes to the main node. The main node independently receives parameters sent by different child nodes, and continuously updates the parameters of the main node model according to the received parameters, and synchronizes the updated model to the corresponding child nodes. The A2C method also constructs a main node and multiple child nodes. The difference is that the child nodes only collect sample data and synchronize them to the main node. The main node updates the model according to the received samples and synchronizes the updated model to the child nodes.

[0006] However, different deep reinforcement learning algorithms are suitable for different task planning problems, and the simulation training environments used are also different. Under certain hardware conditions, the most effective parallel strategies for different algorithm architectures are also different. For example, for tasks using an algorithm architecture where the sub-node is only responsible for delivering samples to the main node, the sub-node usually has low requirements for the graphics processor, i.e., the GPU. For tasks where the sub-node needs to independently calculate the model parameters and share the model parameters with the main node, the sub-node has higher requirements for both the GPU and the CPU.

[0007] For different UAV mission planning problems, under different hardware resource conditions, when task planning models for different specific mission planning problems are trained in parallel, the hardware resources cannot be flexibly configured, resulting in low efficiency in task planning model training. Summary of the invention

[0008] An embodiment of the present application provides a resource allocation method for parallel training of a task planning model, aiming to solve the resource allocation problem in the task planning model training process to improve the training efficiency of the task planning model.

[0009] In a first aspect, an embodiment of the present application provides a resource allocation method for parallel training of a task planning model, wherein the task planning model is a model for environmental problem monitoring, and the method comprises:

[0010] Obtaining environment resource occupation parameters of hardware resources occupied during a single operation of a simulation training environment and model update occupation parameters of hardware resources occupied during a single update of a task planning model; the simulation training environment is a hardware and software environment for training the task planning model;

[0011] Based on the environmental resource occupation parameters and the model update occupation parameters, a master node is generated on the current hardware and part of the resources configured on the current hardware are allocated to the master node; the master node is used to update the task planning model;

[0012] Acquiring control parameters, wherein the control parameters are used to generate subnodes for training the task planning model;

[0013] Generate multiple child nodes based on the control parameters and the remaining hardware resources on the current hardware, wherein the remaining hardware resources on the current hardware include the number of remaining CPUs, the number of remaining GPUs, the remaining resources of each CPU, and the remaining resources of each GPU;

[0014] Allocate the remaining hardware resources to the multiple sub-nodes; wherein the sub-nodes are used to train the task planning model.

[0015] Optionally, the method further comprises:

[0016] Based on the control parameters, a first program package corresponding to the master node and second program packages corresponding to the plurality of sub-nodes are respectively generated; the first program package is an update step for the master node to update the task planning model, and the second program package is training data for the sub-nodes to train the task planning model;

[0017] Sending the first program package to a master node so that the master node trains the task planning model based on the first program package;

[0018] The second program package is sent to each child node, so that the child node trains the task planning model based on the second program package.

[0019] Optionally, the step of obtaining environment resource occupation parameters of hardware resources occupied when the simulation training environment is run once and model update occupation parameters of hardware resources occupied when the task planning model is updated once includes:

[0020] Running the simulation training environment on the current hardware resources, and recording a first hardware resource occupancy parameter when the simulation training environment is running;

[0021] Running the simulation training environment multiple times, calculating a first average value of the first hardware resource occupancy parameter recorded multiple times, the first average value being the environment resource occupancy parameter;

[0022] Updating the task planning model on the simulation training environment, and recording the second hardware resource occupancy parameter when the task planning model is updated;

[0023] The task planning model is updated multiple times, and a second average value of the second hardware resource occupancy parameter recorded multiple times is calculated, where the second average value is the model update occupancy parameter.

[0024] Optionally, the control parameters include parallel strategy parameters and resource allocation strategy parameters;

[0025] The parallel strategy parameters include parameters of a gradient sharing strategy and parameters of a sample sharing strategy;

[0026] The resource allocation strategy parameters include a strategy for the maximum number of sub-nodes and a strategy for optimal sub-node resources;

[0027] Based on the control parameter and the remaining hardware resources on the current hardware, a plurality of child nodes are generated, including:

[0028] Based on the resource allocation strategy parameters and parallel training strategy parameters corresponding to the control parameters, and the remaining hardware resources on the current hardware, multiple sub-nodes are generated, and the number of the multiple sub-nodes and the hardware resources of each sub-node are determined.

[0029] Optionally, the model update occupation parameter includes a GPU resource occupation parameter and a CPU resource occupation parameter; the hardware resources include CPU resources and GPU resources;

[0030] The step of generating a master node on the current hardware based on the environmental resource occupation parameter and the model update occupation parameter and allocating part of the resources configured on the current hardware to the master node includes:

[0031] Allocate a CPU resource greater than the CPU resource occupancy parameter to the master node;

[0032] Allocate a GPU resource greater than the GPU resource occupancy parameter to the master node.

[0033] Optionally, generating a plurality of sub-nodes based on the resource allocation strategy parameters and the parallel training strategy parameters corresponding to the control parameters and the remaining hardware resources on the current hardware, and determining the number of the plurality of sub-nodes and the hardware resources of each sub-node includes:

[0034] If the parallel strategy parameter is a sample sharing parameter and the resource allocation strategy parameter is a strategy with the largest number of child nodes, a first operation is performed on the CPU resource occupancy parameter and the remaining resources of each CPU, and the first operation is performed on the GPU resource occupancy parameter and the remaining resources of each GPU, and the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node are determined according to the result of the first operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU;

[0035] If the parallel strategy parameter is a sample sharing parameter and the resource allocation strategy parameter is an optimal strategy for the number of sub-nodes, the second operation and the third operation are performed on the CPU resource occupancy parameter and the remaining resources of each CPU, and the second operation and the third operation are performed on the GPU resource occupancy parameter and the remaining resources of each GPU, and the number of sub-nodes, the CPU resources allocated to each sub-node, and the GPU resources allocated to the sub-node are determined according to the results of the second operation and the third operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU;

[0036] If the parallel strategy parameter is a gradient sharing parameter and the resource allocation strategy parameter is a strategy with the largest number of child nodes, the first operation and the fourth operation are performed on the CPU resource occupancy parameter and the remaining resources of each CPU, and the first operation and the fourth operation are performed on the GPU resource occupancy parameter and the remaining resources of each GPU, and the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node are determined according to the result of the first operation, the result of the fourth operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU;

[0037] If the parallel strategy parameter is a gradient sharing parameter and the resource allocation strategy parameter is a strategy with the largest number of child nodes, the second operation, the third operation and the fourth operation are performed on the CPU resource occupancy parameter and the remaining resources of each CPU, and the second operation, the third operation and the fourth operation are performed on the GPU resource occupancy parameter and the remaining resources of each GPU, and the number of child nodes, the CPU resources allocated to each child node and the GPU resources allocated to the child node are determined based on a result of the second operation, a result of the second operation, a result of the fourth operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU.

[0038] Optionally, the first operation is a rounding operation; performing the first operation on the CPU resource occupancy parameter and the remaining resources of each CPU, and performing the first operation on the GPU resource occupancy parameter and the remaining resources of each GPU, and determining the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node according to the result of the first operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU, including:

[0039] When the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU, the following formula is used to determine

[0040] The number of subnodes:

[0041]

[0042] Where, Q1 is the number of child nodes; N is the number of remaining CPUs; int is the rounding operation; CPU_C0 is the CPU resource occupation parameter in the environmental resource occupation parameter; CPU_C is the remaining resources of each CPU;

[0043] The CPU resources allocated to each subnode are determined by the following formula:

[0044]

[0045] Among them, T1 is the CPU resource allocated to each child node;

[0046] The GPU resources allocated to each child node are determined by the following formula:

[0047]

[0048] Among them, F1 is the GPU resource allocated to each child node; M is the number of remaining GPUs; GPU_C is the GPU resource occupancy parameter in the environment resource occupancy parameter;

[0049] When the CPU resource occupation parameter in the environmental resource occupation parameter is less than the remaining resource of each CPU, the number of the child nodes is determined by the following formula:

[0050]

[0051] Among them, Q2 is the number of child nodes;

[0052] The CPU resources allocated to each subnode are determined by the following formula:

[0053] ;

[0054] Among them, T2 is the CPU resource allocated to each child node;

[0055] The GPU resources allocated to each child node are determined by the following formula:

[0056] ;

[0057] Among them, F2 is the GPU resource allocated to each child node.

[0058] Optionally, the second operation is a round-up operation, and the third operation is a round-down operation; performing the second operation and the third operation on the CPU resource occupancy parameter and the remaining resources of each CPU, and performing the second operation and the third operation on the GPU resource occupancy parameter and the remaining resources of each GPU, and determining the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node according to the results of the second operation and the third operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU, including:

[0059] When the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU, it is determined by the following formula:

[0060] The number of subnodes:

[0061] ;

[0062] Where, Q3 is the number of child nodes; upper is the rounding operation;

[0063] The CPU resources allocated to each subnode are determined by the following formula:

[0064] ;

[0065] Among them, T3 is the CPU resource allocated to each child node;

[0066] The GPU resources allocated to each child node are determined by the following formula:

[0067] ;

[0068] Among them, F3 is the GPU resource allocated to each child node;

[0069] When the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU, it is determined by the following formula:

[0070] The number of subnodes:

[0071] ;

[0072] Where Q4 is the number of child nodes, and floor is the rounding down operation;

[0073] The CPU resources allocated to each subnode are determined by the following formula:

[0074] ;

[0075] Among them, T4 is the CPU resource allocated to each child node;

[0076] The GPU resources allocated to each child node are determined by the following formula:

[0077] ;

[0078] Among them, F4 is the GPU resource allocated to each child node.

[0079] Optionally, the fourth operation is an operation that returns the smaller of the two, performs the first operation and the fourth operation on the CPU resource occupancy parameter and the remaining resources of each CPU, and performs the first operation and the fourth operation on the GPU resource occupancy parameter and the remaining resources of each GPU, and determines the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node according to the result of the first operation, the result of the fourth operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU, including:

[0080] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each GPU,

[0081] The number of subnodes:

[0082] ;

[0083] Among them, Q5 is the number of child nodes, and min(,) is an operation that returns the smaller of the two;

[0084] The CPU resources allocated to each subnode are determined by the following formula:

[0085] ;

[0086] Among them, T5 is the CPU resource allocated to each child node;

[0087] The GPU resources allocated to each child node are determined by the following formula:

[0088] ;

[0089] Among them, F5 is the GPU resource allocated to each child node;

[0090] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each GPU;

[0091] The number of subnodes:

[0092] ;

[0093] Among them, Q6 is the number of child nodes;

[0094] The CPU resources allocated to each subnode are determined by the following formula:

[0095] ;

[0096] Among them, T6 is the CPU resource allocated to each child node;

[0097] The GPU resources allocated to each child node are determined by the following formula:

[0098] ;

[0099] Among them, F6 is the GPU resource allocated to each child node;

[0100] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each GPU;

[0101] Then the number of child nodes is:

[0102] ;

[0103] Among them, Q7 is the number of child nodes;

[0104] The CPU resources allocated to each subnode are determined by the following formula:

[0105] ;

[0106] Among them, T7 is the CPU resource allocated to each child node;

[0107] The GPU resources allocated to each child node are determined by the following formula:

[0108] ;

[0109] Among them, F7 is the GPU resource allocated to each child node;

[0110] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each GPU:

[0111] The number of subnodes:

[0112] ;

[0113] Among them, Q8 is the number of child nodes;

[0114] The CPU resources allocated to each subnode are determined by the following formula:

[0115] ;

[0116] Among them, T8 is the CPU resource allocated to each child node;

[0117] The GPU resources allocated to each child node are calculated using the following formula:

[0118] ;

[0119] Among them, F8 is the GPU resource allocated to each child node.

[0120] Optionally, performing the second operation, the third operation, and the fourth operation on the CPU resource occupancy parameter and the remaining resources of each CPU, performing the second operation, the third operation, and the fourth operation on the GPU resource occupancy parameter and the remaining resources of each GPU, and determining the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node according to a result of the second operation, a result of the second operation, a result of the fourth operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU, including:

[0121] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each GPU,

[0122] The number of subnodes:

[0123] ;

[0124] Among them, Q9 is the number of child nodes;

[0125] The CPU resources allocated to each subnode are determined by the following formula:

[0126] ;

[0127] Among them, T9 is the CPU resource allocated to each child node;

[0128] The GPU resources allocated to each child node are determined by the following formula:

[0129] ;

[0130] Among them, F9 is the GPU resource allocated to each child node;

[0131] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each GPU;

[0132] The number of subnodes:

[0133] ;

[0134] Among them, Q 10 is the number of child nodes;

[0135] The CPU resources allocated to each subnode are determined by the following formula:

[0136] ;

[0137] Among them, T 10 The CPU resources allocated to each child node;

[0138] The GPU resources allocated to each child node are determined by the following formula:

[0139] ;

[0140] Among them, F 10 GPU resources allocated to each child node;

[0141] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each GPU;

[0142] The number of subnodes:

[0143] ;

[0144] Among them, Q 11 is the number of child nodes;

[0145] The CPU resources allocated to each subnode are determined by the following formula:

[0146] ;

[0147] Among them, T 11 The CPU resources allocated to each child node;

[0148] The GPU resources allocated to each child node are determined by the following formula:

[0149] ;

[0150] Among them, F 11 The GPU resources allocated to each child node;

[0151] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each GPU;

[0152] The number of subnodes:

[0153] ;

[0154] Among them, Q 12 is the number of child nodes;

[0155] The CPU resources allocated to each subnode are determined by the following formula:

[0156] ;

[0157] Among them, T 12 The CPU resources allocated to each child node;

[0158] The GPU resources allocated to each child node are determined by the following formula:

[0159] ;

[0160] Among them, F 12 The GPU resources allocated to each child node.

[0161] In a second aspect, an embodiment of the present application provides a resource allocation device for parallel training of a task planning model, wherein the task planning model is a model for environmental problem monitoring, and the device comprises: the device comprises: a parameter acquisition module, a master node generation module, a control parameter acquisition module, a child node generation module, a remaining hardware resource allocation module, a program package generation module, and a program package sending module; wherein,

[0162] A parameter acquisition module, which acquires the environment resource occupation parameters of the hardware resources occupied when the simulation training environment is run once and the model update occupation parameters of the hardware resources occupied when the task planning model is updated once; the simulation training environment is the software and hardware environment for training the task planning model;

[0163] A master node generation module generates a master node on the current hardware and allocates part of the resources configured on the current hardware to the master node based on the environmental resource occupation parameters and the model update occupation parameters; the master node is used to update the task planning model;

[0164] A control parameter acquisition module, which acquires control parameters, wherein the control parameters are used to generate sub-nodes for training the task planning model;

[0165] A subnode generation module, used to generate multiple subnodes based on the control parameters and the remaining hardware resources on the current hardware, wherein the remaining hardware resources on the current hardware include the number of remaining CPUs, the number of remaining GPUs, the remaining resources of each CPU, and the remaining resources of each GPU;

[0166] A remaining hardware resource allocation module allocates the remaining hardware resources to the plurality of sub-nodes; wherein the sub-nodes are used to train the task planning model;

[0167] A program package generation module, used to generate a first program package corresponding to the master node and second program packages corresponding to the plurality of sub-nodes based on the control parameters; the first program package is an update step for the master node to update the task planning model, and the second program package is training data for the sub-nodes to train the task planning model;

[0168] A program package sending module is used to send the first program package to the main node so that the main node trains the task planning model based on the first program package; and is used to send the second program package to each child node so that the child node trains the task planning model based on the second program package.

[0169] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executed, implements the resource allocation method as described in any one of the first aspects.

[0170] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program that enables a processor to execute any resource allocation method as described.

[0171] Beneficial effects: When training a task planning model, firstly, the environment resource occupation parameters of the hardware resources occupied when the simulation training environment is running and the model update occupation parameters of the hardware resources occupied when the task planning model is updated are calculated, a main node is generated and the hardware resources required for these two parameters are configured; then, multiple sub-nodes are generated according to the remaining hardware resources and control parameters and the remaining hardware resources are allocated to the multiple sub-nodes; by first configuring the hardware resources occupied by the main node and then generating multiple sub-nodes according to the remaining hardware resources and control parameters, because the number of generated sub-nodes is determined by the control parameters and the remaining hardware resources on the current hardware, the hardware resources can be maximized in the parallel training of the task planning model; a main node and multiple sub-nodes are generated according to the hardware resources, and hardware resources are flexibly configured for each sub-node, so that all hardware resources can be fully utilized under the same current hardware resources, so as to achieve the effect of flexible resource configuration and improve the training efficiency of the task planning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0172] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0173] Figure 1 is a flowchart of the steps of the resource configuration method proposed in one embodiment of the present application;

[0174] Figure 2 It is a flowchart of the steps of the resource configuration method proposed in one embodiment of the present application. DETAILED DESCRIPTION

[0175] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0176] Embodiment 1

[0177] Reference Figure 1 , shows a flow chart of the steps of a method for allocating resources for parallel training of a task planning model in an embodiment of the present invention, such as Figure 1 As shown, the task planning model is a model for environmental problem monitoring. The resource allocation method may specifically include the following steps:

[0178] S101, obtaining environment resource occupation parameters of hardware resources occupied when a simulation training environment is run once and model update occupation parameters of hardware resources occupied when a task planning model is updated once; the simulation training environment is a hardware and software environment for training the task planning model;

[0179] Parallel training is an architecture proposed to reduce the time consumption of task planning model training. It is implemented by building a main node and multiple sub-nodes, including two typical methods, A3C and A2C. Among them, A3C is that the sub-nodes independently train the task planning model, and synchronize the data of the task planning model trained by the sub-nodes to the main node. The main node independently receives the data of the task planning model sent by different sub-nodes, and continuously updates the data of the task planning model according to the received data of the task planning model, and synchronizes the updated task planning model to the corresponding sub-nodes. The difference of the A2C method is that the sub-node only synchronizes the sample data obtained by the sub-node training to the main node. The main node updates the model according to the received sample data and synchronizes the updated task planning model to the sub-node.

[0180] When the task planning model is trained in parallel, the required hardware resources are mainly allocated to the master node and sub-nodes; among them, the master node needs to realize the construction of the parallel training environment and the update of the task planning model; by obtaining the environment resource occupancy parameters and the model update occupancy parameters, the amount of hardware resources that the master node needs to allocate when training the task planning model can be obtained.

[0181] S102, based on the environmental resource occupation parameter and the model update occupation parameter, generating a master node on the current hardware and allocating part of the resources configured on the current hardware to the master node; the master node is used to update the task planning model;

[0182] In this embodiment, the master node and the sub-node are both virtual nodes used in the process of training the task planning model, and the master node and the sub-node are just virtual concepts, which are more for distinguishing different hardware resources to achieve the effect of parallel training; their functions are different, the master node is mainly used to update the task planning model, and the sub-node is mainly used to train the task planning model.

[0183] The current hardware is the hardware resource used to train the task planning model. The current hardware can be considered as a computer that has all the resources needed to train the task planning model. By generating a master node and allocating some of the resources configured on the current hardware to the master node, the remaining hardware resources on the current hardware can be allocated to the child nodes for use; after the hardware resources of the master node are configured, the task planning model can be updated during the parallel training process on the master node.

[0184] S103, obtaining control parameters, where the control parameters are used to generate sub-nodes for training the task planning model;

[0185] In the parallel training process of the task planning model, the subnode is a single parallel training node from which the hardware resources for executing the task planning model training are divided. Its main function is to train the task planning model. During training, the number of training rounds of the subnode training the task planning model, the training duration of the subnode training the task planning model and the number of updates of the subnode updating the task planning model are all determined by the control parameters. By obtaining the control parameters and the data contained in the control parameters, the subnode can be generated.

[0186] S104, generating a plurality of child nodes based on the control parameter and the remaining hardware resources on the current hardware, wherein the remaining hardware resources on the current hardware include the number of remaining CPUs, the number of remaining GPUs, the remaining resources of each CPU, and the remaining resources of each GPU;

[0187] In the current hardware, the key to how many child nodes can be generated to perform parallel training of the task planning model is the GPU resources and CPU resources in the current hardware. After allocating part of the hardware resources to the main node, the four parameters of the number of remaining CPUs, the number of remaining GPUs, the remaining resources of each CPU, and the remaining resources of each GPU can quantify the remaining hardware resources on the current hardware. When generating child nodes, all the remaining hardware resources on the current hardware need to be allocated to the child nodes.

[0188] S105, allocating the remaining hardware resources to the multiple sub-nodes; wherein the sub-nodes are used to train the task planning model.

[0189] After generating subnodes, the remaining hardware resources are allocated to multiple subnodes, which can maximize the use of hardware resources on the current hardware and achieve the effect of flexible configuration of hardware resources. Under the same current hardware resources, all hardware resources can be fully utilized to improve the training efficiency of the task planning model.

[0190] In this embodiment, when training the task planning model, first, the environment resource occupation parameters of the hardware resources occupied when the simulation training environment is running and the model update occupation parameters of the hardware resources occupied when the task planning model is updated are calculated, a main node is generated and the hardware resources required for these two parameters are configured; then, multiple sub-nodes are generated according to the remaining hardware resources and control parameters and the remaining hardware resources are allocated to the multiple sub-nodes; by first configuring the hardware resources occupied by the main node, and then generating multiple sub-nodes according to the remaining hardware resources and control parameters, because the number of generated sub-nodes is determined by the control parameters and the remaining hardware resources on the current hardware, the hardware resources can be maximized in the parallel training of the task planning model; a main node and multiple sub-nodes are generated according to the hardware resources, and hardware resources are flexibly configured for each sub-node. Under the same current hardware resources, all hardware resources are fully utilized, so as to achieve the effect of flexible resource configuration and improve the training efficiency of the task planning model.

[0191] Embodiment 2

[0192] Reference Figure 2 As shown, a flowchart of the steps of a resource allocation method for parallel training of a task planning model in an embodiment of the present application is shown, wherein the task planning model is a model for environmental problem monitoring, such as Figure 2 As shown, the resource configuration method may specifically include the following steps:

[0193] S101, obtaining environment resource occupation parameters of hardware resources occupied when a simulation training environment is run once and model update occupation parameters of hardware resources occupied when a task planning model is updated once; the simulation training environment is a hardware and software environment for training the task planning model;

[0194] In this embodiment, the task planning model is used to monitor environmental problems. During the monitoring process, the task planning model can perform corresponding training for the problems encountered in environmental monitoring to obtain a trained task planning model; and the task planning model is used to solve the problems encountered in environmental monitoring in a targeted manner; in the process of training the task planning model, by adopting the resource allocation method in this method, the flexibility of software and hardware resource allocation during the task planning model training process can be improved, thereby improving the training efficiency of the task planning model.

[0195] When the task planning model is trained in parallel, the allocation of current hardware resources includes three directions. One is the environment resource occupancy parameters of the hardware resources occupied when the simulation training environment is run once, the second is to obtain the model update occupancy parameters of the hardware resources occupied when the task planning model is updated once, and the third is the hardware resources occupied when the task planning model is trained once.

[0196] The step of obtaining the environment resource occupation parameters of the hardware resources occupied when the simulation training environment is run once and the model update occupation parameters of the hardware resources occupied when the task planning model is updated once includes:

[0197] Running the simulation training environment on the current hardware resources, and recording a first hardware resource occupancy parameter when the simulation training environment is running;

[0198] Running the simulation training environment multiple times, calculating a first average value of the first hardware resource occupancy parameter recorded multiple times, the first average value being the environment resource occupancy parameter;

[0199] By running the simulation training environment on the current hardware resources, the first hardware resource occupancy parameters required for the simulation training environment during one operation can be obtained. After running the simulation training environment multiple times, multiple first hardware resource occupancy parameters can be obtained. By recording the first hardware resource occupancy parameters obtained multiple times as the first average value, the accuracy of the obtained environmental resource occupancy parameters can be improved.

[0200] Updating the task planning model on the simulation training environment, and recording the second hardware resource occupancy parameter when the task planning model is updated;

[0201] The task planning model is updated multiple times, and a second average value of the second hardware resource occupancy parameter recorded multiple times is calculated, where the second average value is the model update occupancy parameter.

[0202] By updating the task planning model on the current hardware resources, the second hardware resource occupancy parameters required for the task planning model during one update can be obtained, and the second average value of the second hardware resource occupancy parameters recorded multiple times can be calculated. By averaging the second hardware resource occupancy parameters obtained multiple times and obtaining the second average value as the model update occupancy parameter, the accuracy of the obtained model update occupancy parameter can be improved.

[0203] S102, based on the environmental resource occupation parameters and the model update occupation parameters, generate a master node on the current hardware and allocate part of the resources configured on the current hardware to the master node; the master node is used to update the task planning model; wherein,

[0204] The model update occupation parameters include GPU resource occupation parameters and CPU resource occupation parameters; the hardware resources include CPU resources and GPU resources;

[0205] Different deep reinforcement learning algorithms are suitable for different task planning problems, and the simulation training environments used are also different. Under certain hardware conditions, the most effective parallel training strategies for different algorithm architectures are also different. For example, for an algorithm architecture in which the sub-node is only responsible for delivering samples to the main node, the sub-node usually has low requirements for the graphics processing unit, i.e., GPU. For the sub-node to independently calculate the model parameters and share the model parameters with the main node, the sub-node has higher requirements for the GPU and the central processing unit, i.e., CPU.

[0206] The step of generating a master node on the current hardware based on the environmental resource occupation parameter and the model update occupation parameter and allocating part of the resources configured on the current hardware to the master node includes:

[0207] Allocate a CPU resource greater than the CPU resource occupancy parameter to the master node;

[0208] Allocate a GPU resource greater than the GPU resource occupancy parameter to the master node.

[0209] When allocating hardware resources to the master node, because it is necessary to ensure that the master node can operate normally, the GPU resources and CPU resources allocated to the master node must be greater than the CPU resource occupancy parameters and GPU resource occupancy parameters; and it can be ensured that during the process of updating the task planning model, the master node has a certain system redundancy and can provide additional hardware resources when needed.

[0210] S103, obtaining control parameters, wherein the control parameters are used to generate sub-nodes for training the task planning model; wherein,

[0211] The control parameters include parallel strategy parameters and resource allocation strategy parameters;

[0212] The parallel strategy parameters include parameters of a gradient sharing strategy and parameters of a sample sharing strategy;

[0213] The resource allocation strategy parameters include a strategy for the maximum number of sub-nodes and a strategy for optimal sub-node resources;

[0214] The control parameters are preset by the staff, where the parallel strategy parameters refer to the parallel training architecture. In the parallel training of the task planning model, the two parallel training architectures A3C and A2C are generally used. The architecture of A3C is to build a main node and multiple child nodes. The child nodes independently train the model and synchronize the model data trained by the child nodes to the main node. The main node independently receives the model data sent by different child nodes, and continuously updates the model data of the main node according to the received model data, and synchronizes the updated model to the corresponding child nodes; the A2C method also builds a main node and multiple child nodes. The difference is that the child nodes only collect sample data and synchronize them to the main node. The main node updates the model according to the received samples and synchronizes the updated model to the child nodes; the two parallel training frames A3C and A2C correspond to the parameters of the gradient sharing strategy and the parameters of the sample sharing strategy, respectively.

[0215] The resource allocation strategy parameters include the maximum number of sub-nodes strategy and the optimal sub-node resources strategy; when the hardware resource requirements of a single sub-node of the task planning model to be trained are not high, the maximum number of sub-nodes strategy is adopted, which can train more sub-nodes in a shorter time, thereby improving the training efficiency of the task planning model; when the hardware resource requirements of a single sub-node of the task planning model to be trained are high, the optimal sub-node strategy is adopted to improve the training efficiency of a single sub-node, thereby completing the training of the task planning model on a single sub-node in a shorter time.

[0216] S104, generating a plurality of child nodes based on the control parameter and the remaining hardware resources on the current hardware, wherein the remaining hardware resources on the current hardware include the number of remaining CPUs, the number of remaining GPUs, the remaining resources of each CPU, and the remaining resources of each GPU;

[0217] When generating child nodes, it is necessary to evenly distribute all remaining hardware resources on the current hardware to each child node, so as to maximize the use of hardware resources on the current hardware and ensure that each child node has the same progress in training the task planning model.

[0218] Wherein, based on the control parameter and the remaining hardware resources on the current hardware, a plurality of sub-nodes are generated, including:

[0219] Based on the resource allocation strategy parameters and parallel training strategy parameters corresponding to the control parameters and the remaining hardware resources on the current hardware, multiple sub-nodes are generated, and the number of the multiple sub-nodes and the hardware resources of each sub-node are determined. Including:

[0220] If the parallel strategy parameter is a sample sharing parameter and the resource allocation strategy parameter is a strategy with the largest number of child nodes, a first operation is performed on the CPU resource occupancy parameter and the remaining resources of each CPU, and the first operation is performed on the GPU resource occupancy parameter and the remaining resources of each GPU, and the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node are determined according to the result of the first operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU;

[0221] The first operation is a rounding operation; performing the first operation on the CPU resource occupancy parameter and the remaining resources of each CPU, and performing the first operation on the GPU resource occupancy parameter and the remaining resources of each GPU, and determining the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node according to the result of the first operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU, including:

[0222] When the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU, the following formula is used to determine

[0223] The number of subnodes:

[0224]

[0225] Where, Q1 is the number of child nodes; N is the number of remaining CPUs; int is the rounding operation; CPU_C0 is the CPU resource occupation parameter in the environmental resource occupation parameter; CPU_C is the remaining resources of each CPU;

[0226] The CPU resources allocated to each subnode are determined by the following formula:

[0227]

[0228] Among them, T1 is the CPU resource allocated to each child node;

[0229] The GPU resources allocated to each child node are determined by the following formula:

[0230]

[0231] Among them, F1 is the GPU resource allocated to each child node; M is the number of remaining GPUs; GPU_C is the GPU resource occupancy parameter in the environment resource occupancy parameter;

[0232] When the CPU resource occupation parameter in the environmental resource occupation parameter is less than the remaining resource of each CPU, the number of the child nodes is determined by the following formula:

[0233]

[0234] Among them, Q2 is the number of child nodes;

[0235] The CPU resources allocated to each subnode are determined by the following formula:

[0236] ;

[0237] Among them, T2 is the CPU resource allocated to each child node;

[0238] The GPU resources allocated to each child node are determined by the following formula:

[0239] ;

[0240] Among them, F2 is the GPU resource allocated to each child node.

[0241] In this embodiment, the rounding operation is to round off the result after the result is obtained; through the rounding operation, the number of child nodes obtained is an integer, which is convenient for subsequent operations.

[0242] If the parallel strategy parameter is a sample sharing parameter and the resource allocation strategy parameter is an optimal strategy for the number of sub-nodes, the second operation and the third operation are performed on the CPU resource occupancy parameter and the remaining resources of each CPU, and the second operation and the third operation are performed on the GPU resource occupancy parameter and the remaining resources of each GPU, and the number of sub-nodes, the CPU resources allocated to each sub-node, and the GPU resources allocated to the sub-node are determined according to the results of the second operation and the third operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU;

[0243] The second operation is an upward rounding operation, and the third operation is a downward rounding operation; performing the second operation and the third operation on the CPU resource occupancy parameter and the remaining resources of each CPU, and performing the second operation and the third operation on the GPU resource occupancy parameter and the remaining resources of each GPU, and determining the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node according to the results of the second operation and the third operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU, including:

[0244] When the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU, it is determined by the following formula:

[0245] The number of subnodes:

[0246] ;

[0247] Where Q3 is the number of child nodes; upper is the rounding operation;

[0248] The CPU resources allocated to each subnode are determined by the following formula:

[0249] ;

[0250] Among them, T3 is the CPU resource allocated to each child node;

[0251] The GPU resources allocated to each child node are determined by the following formula:

[0252] ;

[0253] Among them, F3 is the GPU resource allocated to each child node;

[0254] When the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU, it is determined by the following formula:

[0255] The number of subnodes:

[0256] ;

[0257] Where Q4 is the number of child nodes, and floor is the rounding down operation;

[0258] The CPU resources allocated to each subnode are determined by the following formula:

[0259] ;

[0260] Among them, T4 is the CPU resource allocated to each child node;

[0261] The GPU resources allocated to each child node are determined by the following formula:

[0262] ;

[0263] Among them, F4 is the GPU resource allocated to each child node.

[0264] If the parallel strategy parameter is a gradient sharing parameter and the resource allocation strategy parameter is a strategy with the largest number of child nodes, the first operation and the fourth operation are performed on the CPU resource occupancy parameter and the remaining resources of each CPU, and the first operation and the fourth operation are performed on the GPU resource occupancy parameter and the remaining resources of each GPU, and the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node are determined according to the result of the first operation, the result of the fourth operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU;

[0265] The fourth operation is an operation that returns the smaller of the two, performing the first operation and the fourth operation on the CPU resource occupancy parameter and the remaining resources of each CPU, and performing the first operation and the fourth operation on the GPU resource occupancy parameter and the remaining resources of each GPU, and determining the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node according to the result of the first operation, the result of the fourth operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU, including:

[0266] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each GPU,

[0267] The number of subnodes:

[0268] ;

[0269] Among them, Q5 is the number of child nodes, and min(,) is an operation that returns the smaller of the two;

[0270] The CPU resources allocated to each subnode are determined by the following formula:

[0271] ;

[0272] Among them, T5 is the CPU resource allocated to each child node;

[0273] The GPU resources allocated to each child node are determined by the following formula:

[0274] ;

[0275] Among them, F5 is the GPU resource allocated to each child node;

[0276] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each GPU;

[0277] The number of subnodes:

[0278] ;

[0279] Among them, Q6 is the number of child nodes;

[0280] The CPU resources allocated to each subnode are determined by the following formula:

[0281] ;

[0282] Among them, T6 is the CPU resource allocated to each child node;

[0283] The GPU resources allocated to each child node are determined by the following formula:

[0284] ;

[0285] Among them, F6 is the GPU resource allocated to each child node;

[0286] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each GPU;

[0287] Then the number of child nodes is:

[0288] ;

[0289] Among them, Q7 is the number of child nodes;

[0290] The CPU resources allocated to each subnode are determined by the following formula:

[0291] ;

[0292] Among them, T7 is the CPU resource allocated to each child node;

[0293] The GPU resources allocated to each child node are determined by the following formula:

[0294] ;

[0295] Among them, F7 is the GPU resource allocated to each child node;

[0296] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each GPU:

[0297] The number of subnodes:

[0298] ;

[0299] Among them, Q8 is the number of child nodes;

[0300] The CPU resources allocated to each subnode are determined by the following formula:

[0301] ;

[0302] Among them, T8 is the CPU resource allocated to each child node;

[0303] The GPU resources allocated to each child node are calculated using the following formula:

[0304] ;

[0305] Among them, F8 is the GPU resource allocated to each child node.

[0306] If the parallel strategy parameter is a gradient sharing parameter and the resource allocation strategy parameter is a strategy with the largest number of child nodes, the second operation, the third operation and the fourth operation are performed on the CPU resource occupancy parameter and the remaining resources of each CPU, and the second operation, the third operation and the fourth operation are performed on the GPU resource occupancy parameter and the remaining resources of each GPU, and the number of child nodes, the CPU resources allocated to each child node and the GPU resources allocated to the child node are determined based on a result of the second operation, a result of the second operation, a result of the fourth operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU.

[0307] The second operation, the third operation, and the fourth operation are performed on the CPU resource occupancy parameter and the remaining resources of each CPU, and the second operation, the third operation, and the fourth operation are performed on the GPU resource occupancy parameter and the remaining resources of each GPU, and the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node are determined according to the result of the second operation, the result of the second operation, the result of the fourth operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU, including:

[0308] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each GPU,

[0309] The number of subnodes:

[0310] ;

[0311] Among them, Q9 is the number of child nodes;

[0312] The CPU resources allocated to each subnode are determined by the following formula:

[0313] ;

[0314] Among them, T9 is the CPU resource allocated to each child node;

[0315] The GPU resources allocated to each child node are determined by the following formula:

[0316] ;

[0317] Among them, F9 is the GPU resource allocated to each child node;

[0318] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each GPU;

[0319] The number of subnodes:

[0320] ;

[0321] Among them, Q 10 is the number of child nodes;

[0322] The CPU resources allocated to each subnode are determined by the following formula:

[0323] ;

[0324] Among them, T 10 The CPU resources allocated to each child node;

[0325] The GPU resources allocated to each child node are determined by the following formula:

[0326] ;

[0327] Among them, F 10 The GPU resources allocated to each child node;

[0328] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each GPU;

[0329] The number of subnodes:

[0330] ;

[0331] Among them, Q 11 is the number of child nodes;

[0332] The CPU resources allocated to each subnode are determined by the following formula:

[0333] ;

[0334] Among them, T 11 The CPU resources allocated to each child node;

[0335] The GPU resources allocated to each child node are determined by the following formula:

[0336] ;

[0337] Among them, F 11 The GPU resources allocated to each child node;

[0338] In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each GPU;

[0339] The number of subnodes:

[0340] ;

[0341] Among them, Q 12 is the number of child nodes;

[0342] The CPU resources allocated to each subnode are determined by the following formula:

[0343] ;

[0344] Among them, T 12 The CPU resources allocated to each child node;

[0345] The GPU resources allocated to each child node are determined by the following formula:

[0346] ;

[0347] Among them, F12 The GPU resources allocated to each child node.

[0348] S105, allocating the remaining hardware resources to the multiple sub-nodes; wherein the sub-nodes are used to train the task planning model.

[0349] After calculating the number of child nodes by controlling the parameters and the remaining hardware resources on the current hardware, the remaining hardware resources on the current hardware are allocated to multiple generated child nodes. When allocating, because the number of child nodes is generated according to the remaining hardware resources on the current hardware, all the remaining hardware resources can be allocated, so that all the hardware resources on the current hardware are allocated to the main node and multiple child nodes, which will not cause waste of hardware resources; the effect of flexible allocation of hardware resources is achieved.

[0350] The method further comprises:

[0351] S106, based on the control parameters, respectively generate a first program package corresponding to the main node and a second program package corresponding to the plurality of sub-nodes; the first program package is an update step for the main node to update the task planning model, and the second program package is training data for the sub-nodes to train the task planning model.

[0352] S107, sending the first program package to the main node so that the main node trains the task planning model based on the first program package; sending the second program package to each child node so that the child node trains the task planning model based on the second program package.

[0353] Through the first program package and the second program package, the main node and the sub-node can be controlled to carry out the training process of the task planning model. During the training, the first program package is used on the main node, and the main node is used to update the task planning model. During the updating process, the main node controls the updating steps of the task planning model through the content of the first program package; the sub-node is used to train the task planning model. During the training process, the sub-node trains the task planning model through the training data contained in the second program package.

[0354] During the training of the task planning model, the master node also needs to record the training data and result data during the training process, where the training data and result data include: the training time of the master node and the child node, the number of task planning model updates on the master node, the number of task planning model updates on the child node, and the task planning model finally trained on the master node during the training process.

[0355] In this embodiment, the hardware resource occupancy parameters and the model update occupancy parameters of the hardware resources occupied when the task planning model is updated are calculated, a main node is generated and the hardware resources required for these two parameters are configured; then multiple sub-nodes are generated according to the remaining hardware resources and control parameters and the remaining hardware resources are allocated to the multiple sub-nodes; by first configuring the hardware resources occupied by the main node, and then generating multiple sub-nodes according to the remaining hardware resources and control parameters, because the number of generated sub-nodes is determined by the control parameters and the remaining hardware resources on the current hardware, the hardware resources can be maximized in the parallel training of the task planning model; a main node and multiple sub-nodes are generated according to the hardware resources, and hardware resources are flexibly configured for each sub-node. Under the same current hardware resources, all hardware resources are fully utilized to achieve the effect of flexible resource configuration and improve the training efficiency of the task planning model.

[0356] Embodiment 3

[0357] The embodiment of the present application provides a resource allocation device for parallel training of a task planning model, wherein the task planning model is a model for environmental problem monitoring, and the device comprises: a parameter acquisition module, a master node generation module, a control parameter acquisition module, a child node generation module, a remaining hardware resource allocation module, a program package generation module, and a program package sending module; wherein,

[0358] A parameter acquisition module, which acquires the environment resource occupation parameters of the hardware resources occupied when the simulation training environment is run once and the model update occupation parameters of the hardware resources occupied when the task planning model is updated once; the simulation training environment is the software and hardware environment for training the task planning model;

[0359] A master node generation module generates a master node on the current hardware and allocates part of the resources configured on the current hardware to the master node based on the environmental resource occupation parameters and the model update occupation parameters; the master node is used to update the task planning model;

[0360] A control parameter acquisition module, which acquires control parameters, wherein the control parameters are used to generate sub-nodes for training the task planning model;

[0361] A subnode generation module, used to generate multiple subnodes based on the control parameters and the remaining hardware resources on the current hardware, wherein the remaining hardware resources on the current hardware include the number of remaining CPUs, the number of remaining GPUs, the remaining resources of each CPU, and the remaining resources of each GPU;

[0362] A remaining hardware resource allocation module allocates the remaining hardware resources to the plurality of sub-nodes; wherein the sub-nodes are used to train the task planning model;

[0363] A program package generation module, used to generate a first program package corresponding to the master node and second program packages corresponding to the plurality of sub-nodes based on the control parameters; the first program package is an update step for the master node to update the task planning model, and the second program package is training data for the sub-nodes to train the task planning model;

[0364] A program package sending module is used to send the first program package to the main node so that the main node trains the task planning model based on the first program package; and is used to send the second program package to each child node so that the child node trains the task planning model based on the second program package.

[0365] Embodiment 4

[0366] Based on the same inventive concept, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes, the resource allocation method described in one of Embodiment 1 or Embodiment 2 is implemented.

[0367] Embodiment 5

[0368] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program that enables a processor to execute the resource allocation method as described in one of Embodiment 1 or Embodiment 2.

[0369] Embodiment 6

[0370] Based on the same inventive concept, this embodiment provides a method for resource allocation for parallel training of a task planning model for environmental problem monitoring. The resource allocation method may include the following steps:

[0371] STEP 1: Obtain hardware resources for the task planning model for training environment problem monitoring;

[0372] STEP 2: Use hardware resources to perform multiple training runs on the simulation training environment of the task planning model and take the average value of the hardware resources occupied during the run as the environment resource occupation parameter;

[0373] STEP3: Use hardware resources to update the task planning model once, and take the average value after multiple updates as the model update occupancy parameter;

[0374] STEP4: The staff inputs the control parameters according to the needs. The control parameters include parallel strategy parameters and resource allocation strategy parameters. When monitoring environmental problems, there are several task planning models required to control the drone, one of which is a task planning model for controlling the flight path of the drone. When training this task planning model, the parallel strategy parameters and resource allocation strategy parameters that need to be selected first are as follows: the parallel strategy parameters are sample sharing parameters and the resource allocation strategy parameters are the strategy with the largest number of sub-nodes.

[0375] STEP6: After selecting the parallel strategy parameters and resource allocation strategy parameters, calculate the number of child nodes based on the control parameters.

[0376] STEP7: Generate a first program package corresponding to the main node and a second program package corresponding to multiple sub-nodes according to the number of sub-nodes; wherein the first program package is used to control the main node to update the task planning model, and the second program package is used to control the sub-node to train the task planning model.

[0377] STEP 8: Send the first program package corresponding to the master node to the master node, and send the second program package corresponding to the child node to multiple child nodes.

[0378] STEP 9: The plurality of sub-nodes train the task planning model according to the second program package, and send the trained task planning model to the master node.

[0379] STEP 10: The master node updates the task planning model according to the data sent by multiple child nodes, and sends the updated task planning model to multiple child nodes.

[0380] STEP 11: After the master node has the task planning model for a predetermined number of times, the training of the task planning model is completed.

[0381] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0382] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0383] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0384] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0385] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0386] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0387] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0388] Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A resource allocation method for parallel training of a task planning model, characterized in that: The task planning model is a model for environmental problem monitoring, and the method comprises: Obtaining environment resource occupation parameters of hardware resources occupied during a single operation of a simulation training environment and model update occupation parameters of hardware resources occupied during a single update of a task planning model; the simulation training environment is a hardware and software environment for training the task planning model; Based on the environmental resource occupation parameters and the model update occupation parameters, a master node is generated on the current hardware and part of the resources configured on the current hardware are allocated to the master node; the master node is used to update the task planning model; Acquiring control parameters, wherein the control parameters are used to generate subnodes for training the task planning model; Generate multiple child nodes based on the control parameters and the remaining hardware resources on the current hardware, wherein the remaining hardware resources on the current hardware include the number of remaining CPUs, the number of remaining GPUs, the remaining resources of each CPU, and the remaining resources of each GPU; Allocating the remaining hardware resources to the plurality of sub-nodes; wherein the sub-nodes are used to train the task planning model; The control parameters include parallel strategy parameters and resource allocation strategy parameters; The parallel strategy parameters include parameters of a gradient sharing strategy and parameters of a sample sharing strategy; The resource allocation strategy parameters include a strategy for the maximum number of sub-nodes and a strategy for optimal sub-node resources; Based on the control parameter and the remaining hardware resources on the current hardware, a plurality of child nodes are generated, including: Based on the resource allocation strategy parameters and the parallel training strategy parameters corresponding to the control parameters, and the remaining hardware resources on the current hardware, generate multiple sub-nodes, and determine the number of the multiple sub-nodes and the hardware resources of each sub-node; The model update occupation parameters include GPU resource occupation parameters and CPU resource occupation parameters; the hardware resources include CPU resources and GPU resources; The step of generating a master node on the current hardware based on the environmental resource occupation parameter and the model update occupation parameter and allocating part of the resources configured on the current hardware to the master node includes: Allocate a CPU resource greater than the CPU resource occupancy parameter to the master node; Allocating GPU resources greater than the GPU resource occupancy parameter to the master node; generating multiple child nodes based on the resource allocation strategy parameters and parallel training strategy parameters corresponding to the control parameters and the remaining hardware resources on the current hardware, and determining the number of the multiple child nodes and the hardware resources of each child node, including: If the parallel strategy parameter is a sample sharing parameter and the resource allocation strategy parameter is a strategy with the largest number of child nodes, a first operation is performed on the CPU resource occupancy parameter and the remaining resources of each CPU, and the first operation is performed on the GPU resource occupancy parameter and the remaining resources of each GPU, and the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node are determined according to the result of the first operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU; If the parallel strategy parameter is a sample sharing parameter and the resource allocation strategy parameter is an optimal strategy for the number of sub-nodes, the second operation and the third operation are performed on the CPU resource occupancy parameter and the remaining resources of each CPU, and the second operation and the third operation are performed on the GPU resource occupancy parameter and the remaining resources of each GPU, and the number of sub-nodes, the CPU resources allocated to each sub-node, and the GPU resources allocated to the sub-node are determined according to the results of the second operation and the third operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU; If the parallel strategy parameter is a gradient sharing parameter and the resource allocation strategy parameter is a strategy with the largest number of child nodes, the first operation and the fourth operation are performed on the CPU resource occupancy parameter and the remaining resources of each CPU, and the first operation and the fourth operation are performed on the GPU resource occupancy parameter and the remaining resources of each GPU, and the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node are determined according to the result of the first operation, the result of the fourth operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU; If the parallel strategy parameter is a gradient sharing parameter and the resource allocation strategy parameter is a strategy with the largest number of child nodes, the second operation, the third operation and the fourth operation are performed on the CPU resource occupancy parameter and the remaining resources of each CPU, and the second operation, the third operation and the fourth operation are performed on the GPU resource occupancy parameter and the remaining resources of each GPU, and the number of child nodes, the CPU resources allocated to each child node and the GPU resources allocated to the child node are determined based on a result of the second operation, a result of the second operation, a result of the fourth operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU.

2. The method according to claim 1, characterized in that The method further comprises: Based on the control parameters, a first program package corresponding to the master node and second program packages corresponding to the plurality of sub-nodes are respectively generated; the first program package is an update step for the master node to update the task planning model, and the second program package is training data for the sub-nodes to train the task planning model; Sending the first program package to a master node so that the master node trains the task planning model based on the first program package; The second program package is sent to each child node, so that the child node trains the task planning model based on the second program package.

3. A method according to claim 1, characterized in that The method of obtaining the environment resource occupation parameters of the hardware resources occupied when the simulation training environment is run once and the model update occupation parameters of the hardware resources occupied when the task planning model is updated once includes: Running the simulation training environment on the current hardware resources, and recording a first hardware resource occupancy parameter when the simulation training environment is running; Running the simulation training environment multiple times, calculating a first average value of the first hardware resource occupancy parameter recorded multiple times, the first average value being the environment resource occupancy parameter; Updating the task planning model on the simulation training environment, and recording the second hardware resource occupancy parameter when the task planning model is updated; The task planning model is updated multiple times, and a second average value of the second hardware resource occupancy parameter recorded multiple times is calculated, where the second average value is the model update occupancy parameter.

4. A method according to claim 1, characterized in that The first operation is a rounding operation; performing the first operation on the CPU resource occupancy parameter and the remaining resources of each CPU, and performing the first operation on the GPU resource occupancy parameter and the remaining resources of each GPU, and determining the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node according to the result of the first operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU, including: When the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU, the following formula is used to determine The number of subnodes: Where, Q1 is the number of child nodes; N is the number of remaining CPUs; int is the rounding operation; CPU_C0 is the CPU resource occupation parameter in the environmental resource occupation parameter; CPU_C is the remaining resources of each CPU; The CPU resources allocated to each subnode are determined by the following formula: Among them, T1 is the CPU resource allocated to each child node; The GPU resources allocated to each child node are determined by the following formula: Among them, F1 is the GPU resource allocated to each child node; M is the number of remaining GPUs; GPU_C is the GPU resource occupancy parameter in the environment resource occupancy parameter; When the CPU resource occupation parameter in the environmental resource occupation parameter is less than the remaining resource of each CPU, the number of the child nodes is determined by the following formula: Among them, Q2 is the number of child nodes; The CPU resources allocated to each subnode are determined by the following formula: ； Among them, T2 is the CPU resource allocated to each child node; The GPU resources allocated to each child node are determined by the following formula: ； Among them, F2 is the GPU resource allocated to each child node.

5. A method according to claim 4, characterized in that The second operation is an upward rounding operation, and the third operation is a downward rounding operation; performing the second operation and the third operation on the CPU resource occupancy parameter and the remaining resources of each CPU, and performing the second operation and the third operation on the GPU resource occupancy parameter and the remaining resources of each GPU, and determining the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node according to the results of the second operation and the third operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU, including: When the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU, it is determined by the following formula: The number of subnodes: ； Where Q3 is the number of child nodes; upper is the rounding operation; The CPU resources allocated to each subnode are determined by the following formula: ； Among them, T3 is the CPU resource allocated to each child node; The GPU resources allocated to each child node are determined by the following formula: ； Among them, F3 is the GPU resource allocated to each child node; When the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU, it is determined by the following formula: The number of subnodes: ； Where Q4 is the number of child nodes, and floor is the rounding down operation; The CPU resources allocated to each subnode are determined by the following formula: ； Among them, T4 is the CPU resource allocated to each child node; The GPU resources allocated to each child node are determined by the following formula: ； Among them, F4 is the GPU resource allocated to each child node.

6. A method according to claim 5, characterized in that The fourth operation is an operation that returns the smaller of the two, performing the first operation and the fourth operation on the CPU resource occupancy parameter and the remaining resources of each CPU, and performing the first operation and the fourth operation on the GPU resource occupancy parameter and the remaining resources of each GPU, and determining the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node according to the result of the first operation, the result of the fourth operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU, including: In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each GPU, The number of subnodes: ； Among them, Q5 is the number of child nodes, and min(,) is an operation that returns the smaller of the two; The CPU resources allocated to each subnode are determined by the following formula: ； Among them, T5 is the CPU resource allocated to each child node; The GPU resources allocated to each child node are determined by the following formula: ； Among them, F5 is the GPU resource allocated to each child node; In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each GPU; The number of subnodes: ； Among them, Q6 is the number of child nodes; The CPU resources allocated to each subnode are determined by the following formula: ； Among them, T6 is the CPU resource allocated to each child node; The GPU resources allocated to each child node are determined by the following formula: ； Among them, F6 is the GPU resource allocated to each child node; In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each GPU; Then the number of child nodes is: ； Among them, Q7 is the number of child nodes; The CPU resources allocated to each subnode are determined by the following formula: ； Among them, T7 is the CPU resource allocated to each child node; The GPU resources allocated to each child node are determined by the following formula: ； Among them, F7 is the GPU resource allocated to each child node; In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each GPU: The number of subnodes: ； Among them, Q8 is the number of child nodes; The CPU resources allocated to each subnode are determined by the following formula: ； Among them, T8 is the CPU resource allocated to each child node; The GPU resources allocated to each child node are calculated using the following formula: ； Among them, F8 is the GPU resource allocated to each child node.

7. A method according to claim 6, characterized in that The second operation, the third operation, and the fourth operation are performed on the CPU resource occupancy parameter and the remaining resources of each CPU, and the second operation, the third operation, and the fourth operation are performed on the GPU resource occupancy parameter and the remaining resources of each GPU, and the number of child nodes, the CPU resources allocated to each child node, and the GPU resources allocated to the child node are determined according to the result of the second operation, the result of the second operation, the result of the fourth operation, the number of remaining CPUs on the current hardware, and the remaining resources of each CPU, including: In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each GPU, The number of subnodes: ； Among them, Q9 is the number of child nodes; The CPU resources allocated to each subnode are determined by the following formula: ； Among them, T9 is the CPU resource allocated to each child node; The GPU resources allocated to each child node are determined by the following formula: ； Among them, F9 is the GPU resource allocated to each child node; In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each GPU; The number of subnodes: ； Among them, Q 10 is the number of child nodes; The CPU resources allocated to each subnode are determined by the following formula: ； Among them, T 10 The CPU resources allocated to each child node; The GPU resources allocated to each child node are determined by the following formula: ； Among them, F 10 The GPU resources allocated to each child node; In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is greater than or equal to the remaining resources of each GPU; The number of subnodes: ； Among them, Q 11 is the number of child nodes; The CPU resources allocated to each subnode are determined by the following formula: ； Among them, T 11 The CPU resources allocated to each child node; The GPU resources allocated to each child node are determined by the following formula: ； Among them, F 11 The GPU resources allocated to each child node; In the case where the CPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each CPU and the GPU resource occupancy parameter in the environmental resource occupancy parameter is less than the remaining resources of each GPU; The number of subnodes: ； Among them, Q 12 is the number of child nodes; The CPU resources allocated to each subnode are determined by the following formula: ； Among them, T 12 The CPU resources allocated to each child node; The GPU resources allocated to each child node are determined by the following formula: ； Among them, F 12 The GPU resources allocated to each child node.

Images