Task scheduling method and device and cloud platform

Abstract

The embodiment of the invention discloses a task scheduling method and device and a cloud platform. The method comprises the steps that in response to received downstream tasks, state information of multiple to-be-allocated vehicles and task information of the downstream tasks are acquired, the state information comprises residual computing power, residual electric quantity and distance information, and the task information comprises computing power and task amount required by the downstream tasks; and on the basis of the state information, the task information and a pre-trained task scheduling agent, performing downstream task allocation on the plurality of to-be-allocated vehicles. Through the above mode, downstream task allocation can be performed on the plurality of to-be-allocated vehicles based on the state information, the task information and the pre-trained task scheduling agent, so that different downstream tasks can be allocated based on the computing power, the remaining power and the like of the plurality of to-be-allocated vehicles, and the utilization rate of computing power resources is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a task scheduling method, device and cloud platform. Background Art

[0002] The intelligentization of automobiles is an important trend in the development of the current automobile industry. It integrates high-computing chips, advanced sensors, communication technology, artificial intelligence algorithms and big data analysis to provide drivers with a more intelligent, safe, convenient and personalized driving experience and enhance user comfort. However, driving scenarios only account for a small part of daily behavior, resulting in high-computing chips being idle for a long time, and the utilization rate of computing resources is greatly reduced. Summary of the invention

[0003] In view of the above problems, the embodiments of the present application propose a task scheduling method, device and cloud platform to improve the above problems.

[0004] In a first aspect, an embodiment of the present application provides a task scheduling method, the method comprising: in response to receiving a downstream task, obtaining status information of multiple vehicles to be assigned and task information of the downstream task, the status information comprising remaining computing power, remaining power, and distance information, and the task information comprising computing power and task volume required for the downstream task; based on the status information, the task information, and a pre-trained task scheduling agent, performing downstream tasks on the multiple vehicles to be assigned.

[0005] In the second aspect, an embodiment of the present application provides a task scheduling device, which includes: an information acquisition unit, which, in response to receiving a downstream task, acquires status information of multiple vehicles to be assigned and task information of the downstream task, the status information including remaining computing power, remaining power, and distance information, and the task information includes the computing power and task amount required for the downstream task; a task scheduling unit, which is used to perform downstream task allocation to the multiple vehicles to be assigned based on the status information, the task information and a pre-trained task scheduling agent.

[0006] In a third aspect, an embodiment of the present application provides a cloud platform, comprising a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor, and the one or more programs are configured to execute the above method.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored, wherein the above method is executed when the program code is run.

[0008] The embodiment of the present application provides a task scheduling method, device, cloud platform and storage medium. In response to receiving a downstream task, after obtaining the status information of multiple vehicles to be assigned and the task information of the downstream task, the downstream task is assigned to the multiple vehicles to be assigned based on the status information, the task information and the pre-trained task scheduling agent. Through the above-mentioned method, in response to receiving a downstream task, the status information of multiple vehicles to be assigned and the task information of the downstream task are obtained, and the downstream task is assigned to the multiple vehicles to be assigned based on the status information, the task information and the pre-trained task scheduling agent, so that different downstream tasks can be assigned based on the computing power, remaining power, etc. of the multiple vehicles to be assigned, thereby improving the utilization rate of computing power resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0010] Figure 1 A flowchart of a task scheduling method proposed in an embodiment of the present application is shown;

[0011] Figure 2 A schematic diagram of a task scheduling proposed in this application is shown;

[0012] Figure 3 A flowchart of a task scheduling method proposed in another embodiment of the present application is shown;

[0013] Figure 4 A schematic diagram of an initial task scheduling agent proposed in this application is shown;

[0014] Figure 5 Shows this application Figure 3 A flowchart of an embodiment of S240;

[0015] Figure 6 Shows this application Figure 5 A flowchart of an embodiment of S241;

[0016] Figure 7 A structural block diagram of a task scheduling device proposed in an embodiment of the present application is shown;

[0017] Figure 8 A structural block diagram of a cloud platform proposed in this application is shown. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] In an embodiment of the present application, the inventor proposes a task scheduling method, device, and cloud platform, which can respond to receiving a downstream task, obtain the status information of multiple vehicles to be assigned and the task information of the downstream task, and then perform downstream task assignments on the multiple vehicles to be assigned based on the status information, the task information, and a pre-trained task scheduling agent. Through the above-mentioned method, in response to receiving a downstream task, obtain the status information of multiple vehicles to be assigned and the task information of the downstream task, and perform downstream task assignments on multiple vehicles to be assigned based on the status information, the task information, and a pre-trained task scheduling agent, so that different downstream tasks can be assigned based on the computing power, remaining power, etc. of the multiple vehicles to be assigned, thereby improving the utilization rate of computing power resources.

[0020] The embodiments of the present application will be described below with reference to the accompanying drawings.

[0021] See also Figure 1 , the present application provides a task scheduling method, the method comprising:

[0022] S110: In response to receiving a downstream task, status information of multiple vehicles to be assigned and task information of the downstream task are obtained, the status information includes remaining computing power, remaining power, and distance information, and the task information includes computing power and task amount required for the downstream task.

[0023] Among them, downstream tasks can refer to tasks that need to be completed by high-computing chips uploaded by vehicles to the cloud platform, such as: cloud platform reasoning, training tasks, vehicle-side reasoning, training tasks, etc. Among them, high-computing chips can refer to chips with computing power exceeding the preset computing power. The preset computing power can refer to the intelligent computing power of the computing power value determined according to industry specifications. High-computing chips can include GPU, FPGA, ASIC, etc. There can be multiple downstream tasks. When there are multiple downstream tasks, the task information can refer to the computing power required for each downstream task and the task volume of each downstream task. The task volume can refer to the scale that characterizes the downstream task.

[0024] For example, when the downstream task is a model training task and the model training task requires a card with 100Tops computing power to implement and the training set size is 50G, the computing power required for the downstream task can be 100Tops and the task volume can be 50G; when the downstream task is a model inference task and the size of the model is 7B, the task volume of the downstream task can be 7B.

[0025] In the embodiment of the present application, the cloud platform may refer to a cloud server. The vehicle to be assigned may refer to a vehicle available for task assignment by the cloud platform. The distance information may refer to the distance between the cloud platform and the vehicle.

[0026] As a method, in response to receiving a downstream task, the cloud platform can send a status information acquisition request to all vehicles that have established a communication connection with the cloud platform, so that all vehicles that have established a communication connection with the cloud platform upload their own status information together with the vehicle identification to the cloud platform after receiving the status information acquisition request, so that the cloud platform can obtain the status information of multiple vehicles to be assigned. At the same time, when the cloud platform receives a downstream task, it can parse the received downstream task to obtain the task information of the downstream task.

[0027] Optionally, when a vehicle has a downstream task, the vehicle may send the task identifier and task information of the downstream task to the cloud platform.

[0028] Optionally, after receiving the status information and task information, the cloud platform can generate the following representation:

[0029] s={(tops 1 , e 1 , d 1 ), (tops 2 , e 2 , d 2 ), ..., (q 1 , m 1 ), (q 2 , m 2 ), ...}

[0030] Among them, (tops 1 , e 1 , d 1 ) can represent the remaining computing power, remaining power and distance information of vehicle 1, (q 1 , m 1 ) can represent the required computing power and task volume of downstream task 1.

[0031] Optionally, the cloud platform can obtain status information of multiple candidate vehicles and task information of downstream tasks in response to receiving downstream tasks; based on the status information of each candidate vehicle, determine whether the corresponding candidate vehicle meets the preset computing power conditions; if the corresponding candidate vehicle meets the preset computing power conditions, use the corresponding candidate vehicle as a vehicle to be assigned to obtain status information of multiple vehicles to be assigned.

[0032] The preset computing power conditions may include that the remaining computing power is greater than the computing power required by the downstream task, and the remaining power is greater than the power required to execute the downstream task. When there are multiple downstream tasks, the computing power required by the downstream task may refer to the computing power corresponding to the task with the least computing power required among the multiple downstream tasks, and the power required by the downstream task may refer to the power corresponding to the task with the least power required among the multiple downstream tasks. The power required by the downstream task can be estimated by the task volume.

[0033] In some embodiments, the candidate vehicles may refer to all vehicles that have established a communication connection with the cloud platform. In other embodiments, the candidate vehicles may refer to all vehicles that have established a communication connection with the cloud platform and whose distance from the cloud platform is less than a preset distance, wherein by limiting the preset distance, the communication time between the vehicle and the cloud platform can be reduced as much as possible, thereby improving the execution efficiency of downstream tasks.

[0034] Optionally, the preset distance may be obtained based on multiple experiments. The preset distance may also be obtained based on the real-time requirements of downstream tasks. The higher the real-time requirements, the smaller the preset distance.

[0035] Optionally, the real-time requirements of downstream tasks can be classified according to preset rules. The real-time requirements can be divided into high, medium, and low.

[0036] S120: Based on the state information, the task information and a pre-trained task scheduling agent, perform downstream task allocation for the multiple vehicles to be allocated.

[0037] The pre-trained task scheduling agent may refer to an agent trained based on reinforcement learning and used for downstream task allocation.

[0038] As a way, Figure 2 As shown, the state information and task information can be input into a pre-trained task scheduling agent to obtain an allocation result, and downstream tasks can be allocated to multiple vehicles to be allocated based on the allocation result.

[0039] The allocation result can be expressed as: a={(car 1 , task 1 ), (car 2 , task 2 ),……,(car n , task n )},(car i , task j ) can indicate that vehicle i performs downstream task j.

[0040] It should be noted that the downstream tasks received by the cloud platform can be completed in multiple scheduling. For example, when there are 50 vehicles to be assigned and 100 downstream tasks, a task can be assigned to each vehicle to be assigned in the first scheduling, so that there are 50 downstream tasks left. After the state information and task information are updated, the downstream task allocation continues based on the updated state information, task information and pre-trained task scheduling agents until all downstream tasks are assigned.

[0041] A task scheduling method provided in this embodiment can respond to receiving a downstream task, obtain the status information of multiple vehicles to be assigned and the task information of the downstream task, and then assign downstream tasks to the multiple vehicles to be assigned based on the status information, the task information and a pre-trained task scheduling agent. Through the above method, in response to receiving a downstream task, the status information of multiple vehicles to be assigned and the task information of the downstream task are obtained, and the downstream tasks are assigned to multiple vehicles to be assigned based on the status information, the task information and a pre-trained task scheduling agent, so that different downstream tasks can be assigned based on the computing power, remaining power, etc. of the multiple vehicles to be assigned, thereby improving the utilization rate of computing power resources.

[0042] See also Figure 3 , the present application provides a task scheduling method, the method comprising:

[0043] S210: Acquire an initial training state, where the initial training state includes state information of each of a plurality of training vehicles and task information of each of a plurality of training tasks.

[0044] The initial training state may refer to the initial environment information during the task scheduling agent training process. The training vehicle may refer to a physical vehicle or a simulated vehicle participating in the task scheduling agent training process, and the training task may refer to a downstream task participating in the task scheduling agent training process. The initial training state may be expressed as: t ={(t_tops 1 , t_e 1 , t_d 1 ), (t_tops 2 , t_e 2 , t_d 2 ), ..., (t_q 1 , t_m 1 ), (t_q 2 , t_m 2 ),……}.

[0045] As a method, R&D personnel can input the status information of multiple training vehicles and the task information of multiple training tasks into the cloud platform so that the cloud platform can obtain the initial training status.

[0046] S220: Acquire a pre-constructed action space, where the action space includes the multiple training vehicles and the multiple training tasks.

[0047] The action space can be expressed as A = {(t_car 1 ,......,t_car n ), (t_task 1 , ..., t_task m )}

[0048] As a way, developers can input the pre-built action space into the cloud platform so that the cloud platform can obtain the pre-built action space.

[0049] Optionally, action space construction can be performed based on the initial training state.

[0050] S230: Obtain a pre-designed reward function, where the reward function represents the benefits that can be generated after the initial task scheduling agent performs a task assignment.

[0051] Among them, the reward function may include task completion rate, computing power utilization rate and policy entropy. The completion rate may refer to the ratio of the number of assigned training tasks to the total number of training tasks. The computing power utilization rate may refer to the ratio of the computing power required for the assigned training tasks to the total remaining computing power of the training vehicle. Policy entropy can be used as a measure of the randomness (or uncertainty) of policy distribution. Setting policy entropy in the reward function helps increase exploratory behavior and may find a better strategy. The calculation formula of the reward function can be:

[0052] r t =α+β+H

[0053] Among them, α can represent the task completion rate, β can represent the computing power utilization rate, and H can represent the strategy entropy.

[0054] As a way, R&D personnel can input the pre-designed reward function into the cloud platform so that the cloud platform can obtain the reward function.

[0055] S240: Based on the initial training state, the action space and the reward function, the initial task scheduling agent is trained to obtain the pre-trained task scheduling agent.

[0056] Among them, Figure 4As shown, the initial task scheduling agent may include an initial action (Actor) network and an initial evaluation (Critic) network, and the network structures of the initial action network and the initial evaluation network may be the same.

[0057] As a way, Figure 5 As shown, based on the initial training state, action space and reward function, the initial task scheduling agent is trained to obtain a pre-trained task scheduling agent, including:

[0058] S241: During the current iteration process, based on the initial training state, the action space, the reward function and the initial task scheduling agent, an experience pool corresponding to the current iteration process is obtained, and the experience pool includes multiple experiences, each of which includes the current training state, scheduling action, reward value, and training state at the next moment.

[0059] As a way, Figure 6 As shown in the figure, in the current iteration process, based on the initial training state, action space, reward function and initial task scheduling agent, the experience pool corresponding to the current iteration process is obtained, including:

[0060] S2411: In the current time step of the current iteration process, the current training state corresponding to the current time step is input into the initial action network to obtain the scheduling action of the current time step from the action space, wherein the scheduling action includes multiple vehicle-task pairs, each of which represents a training task assigned to a training vehicle, wherein the current training state corresponding to the first time step is the initial training state.

[0061] As a method, in the current time step of the iteration process, the current training state corresponding to the current time step can be input into the initial action network, and the initial action network can obtain the scheduling action of the current time step from the action space according to the parameters of the initial action network (ie, the strategy).

[0062] S2412: Execute the scheduling action of the current time step to obtain the training state at the next moment corresponding to the current time step.

[0063] As a method, in the current time step of the iteration process, the scheduling action of the current time step can be executed through the initial evaluation network to obtain the training state of the next moment corresponding to the current time step.

[0064] S2413: Obtain a reward value for the current time step through the initial evaluation network and the reward function.

[0065] As a way, in the current time step of the iteration process, the reward value of the current time step can be obtained through the initial evaluation network and the reward function.

[0066] S2414: The current training state, the scheduling action, the next training state, and the reward value of the current time step are stored as an experience in the experience pool corresponding to the current iteration process.

[0067] Among them, one experience can be expressed as: (s t ,a t ,s t+1 ,r t ), where s t It can represent the training state at the current time step of the iteration process, a t It can represent the scheduling action at the current time step of the iteration process, s t+1 It can represent the training state at the next moment of the current time step of the iteration process, r t It can represent the reward value at the current time step of the iteration process.

[0068] As a method, the current training state, scheduling action, next training state, and reward value of the current time step can be stored as an experience in the experience pool corresponding to the current iteration process.

[0069] S2415: Continue to use the next moment training state of the current time step as the current moment training state of the next time step to obtain the scheduling action, next moment training state, and reward value of the next time step until the experience pool corresponding to the current iteration process is generated.

[0070] As a method, the next moment training state of the current time step can be continued as the current moment training state of the next time step to obtain the scheduling action, next moment training state, and reward value of the next time step until the experience pool corresponding to the current iteration process is generated.

[0071] Optionally, before executing step S241, the parameters of the initial action (Actor) network and the initial evaluation (Critic) network can be randomly initialized, and the initialization parameters of the initial action (Actor) network are assigned to the target action network, the initialization parameters of the initial evaluation (Actor) network are assigned to the target evaluation network, and the experience pool is cleared.

[0072] S242: In the current iteration process, part of the experience is collected from the experience pool to obtain a plurality of training samples.

[0073] As a method, during the current iteration, some experience can be randomly collected from the experience pool to obtain multiple training samples.

[0074] S243: Based on the multiple training samples, update the network parameters of the initial evaluation network to obtain an updated evaluation network corresponding to the current iteration process.

[0075] As a method, the network parameters of the initial evaluation network can be updated based on multiple training samples and minimizing the loss function to obtain an updated evaluation network corresponding to the current iteration process.

[0076] S244: Based on the multiple training samples, update the network parameters of the initial action network to obtain an updated action network corresponding to the current iteration process.

[0077] As a method, the network parameters of the initial action network can be updated based on multiple training samples and the gradient descent method to obtain the updated action network corresponding to the current iteration process.

[0078] Optionally, after executing step S244, the network parameters of the target action network and the target evaluation network may be updated based on multiple training samples and a soft update method to obtain an updated target action network and an updated target evaluation network corresponding to the current iteration process.

[0079] S245: If the current iteration process meets the target condition, the updated action network is used as the pre-trained task scheduling agent; if the current iteration process does not meet the target condition, the next iteration process is entered, and the updated action network and the updated evaluation network are used as the initial action network and initial evaluation network of the next iteration process respectively.

[0080] The target condition may be that the difference between the network parameters of the initial evaluation network updated in two adjacent iterative processes is within a preset range, or the target condition may be that the number of iterations reaches a specified number, etc.

[0081] As a method, if the current iteration process meets the target conditions, the updated action network is used as the pre-trained task scheduling agent; if the current iteration process does not meet the target conditions, the next iteration process is entered, and the updated action network and the updated evaluation network are used as the initial action network and initial evaluation network of the next iteration process respectively.

[0082] S250: In response to receiving a downstream task, status information of multiple vehicles to be assigned and task information of the downstream task are obtained, the status information includes remaining computing power, remaining power, and distance information, and the task information includes computing power and task amount required for the downstream task.

[0083] S260: Based on the state information, the task information and a pre-trained task scheduling agent, perform downstream task allocation for the multiple vehicles to be allocated.

[0084] A task scheduling method provided in this embodiment, through the above-mentioned method, in response to receiving downstream tasks, obtains the state information of multiple vehicles to be assigned and the task information of downstream tasks, and assigns downstream tasks to multiple vehicles to be assigned based on the state information, task information and pre-trained task scheduling agents, so that different downstream tasks can be assigned based on the computing power, remaining power, etc. of multiple vehicles to be assigned, thereby improving the utilization rate of computing power resources. In addition, in this embodiment, the pre-trained task scheduling intelligence can reasonably allocate vehicle-side computing power resources and downstream tasks, select appropriate vehicles to be assigned according to influencing factors such as computing power, distance, and duration, and realize reasonable call of vehicle-side computing power resources. In addition, during the training process, the input of the initial task scheduling agent is the experience stored in the experience pool, and a part of the data is randomly extracted each time as a training sample to update the network parameters, so that the initial task scheduling agent gradually fits the real environment and outputs a reasonable cloud platform strategy for allocating vehicle-side computing power.

[0085] See also Figure 7 The present application provides a task scheduling device 400, the device comprising:

[0086] The information acquisition unit 410 acquires, in response to receiving a downstream task, state information of a plurality of vehicles to be assigned and task information of the downstream task, wherein the state information includes remaining computing power, remaining power, and distance information, and the task information includes computing power and task amount required by the downstream task;

[0087] The task scheduling unit 420 is used to perform downstream task allocation to the multiple vehicles to be allocated based on the state information, the task information and a pre-trained task scheduling agent.

[0088] As a method, the information acquisition unit 410 is specifically used to obtain the status information of multiple candidate vehicles and the task information of the downstream task in response to receiving a downstream task; based on the status information of each candidate vehicle, determine whether the corresponding candidate vehicle meets the preset computing power condition; if the corresponding candidate vehicle meets the preset computing power condition, use the corresponding candidate vehicle as the vehicle to be assigned to obtain the status information of the multiple vehicles to be assigned.

[0089] The device 400 further includes:

[0090] The agent training unit 430 is used to obtain an initial training state, wherein the initial training state includes state information of each of a plurality of training vehicles and task information of each of a plurality of training tasks; obtain a pre-constructed action space, wherein the action space includes the plurality of training vehicles and the plurality of training tasks; obtain a pre-designed reward function, wherein the reward function represents the benefit that can be generated after the initial task scheduling agent performs a task assignment; based on the initial training state, the action space and the reward function, the initial task scheduling agent is trained to obtain the pre-trained task scheduling agent.

[0091] As a method, the initial task scheduling agent includes an initial action network and an initial evaluation network. The agent training unit 430 is specifically used to obtain an experience pool corresponding to the current iteration process based on the initial training state, the action space, the reward function and the initial task scheduling agent during the current iteration process, wherein the experience pool includes multiple experiences, each of which includes a current moment training state, a scheduling action, a reward value, and a next moment training state; during the current iteration process, part of the experience is collected from the experience pool to obtain multiple training samples; based on the multiple training samples, the network parameters of the initial evaluation network are updated to obtain an updated evaluation network corresponding to the current iteration process; based on the multiple training samples, the network parameters of the initial action network are updated to obtain an updated action network corresponding to the current iteration process; if the current iteration process meets the target condition, the updated action network is used as the pre-trained task scheduling agent; if the current iteration process does not meet the target condition, the next iteration process is entered, and the updated action network and the updated evaluation network are used as the initial action network and the initial evaluation network of the next iteration process respectively.

[0092] Optionally, the intelligent agent training unit 430 is specifically used to input the current training state corresponding to the current time step into the initial action network in the current time step of the current iteration process, so as to obtain the scheduling action of the current time step from the action space, the scheduling action including multiple vehicle-task pairs, each of the vehicle-task pairs representing a training task assigned to a training vehicle, wherein the current training state corresponding to the first time step is the initial training state; execute the scheduling action of the current time step to obtain the next training state corresponding to the current time step; obtain the reward value of the current time step through the initial evaluation network and the reward function; store the current training state, scheduling action, next training state and reward value of the current time step as an experience in the experience pool corresponding to the current iteration process; continue to use the next training state of the current time step as the current training state of the next time step to obtain the scheduling action, next training state and reward value of the next time step until the experience pool corresponding to the current iteration process is generated.

[0093] Optionally, the reward function includes task completion rate, computing power utilization rate and strategy entropy.

[0094] Optionally, the preset computing power condition includes that the remaining computing power is greater than the computing power required by the downstream task, and the remaining power is greater than the power required to execute the downstream task.

[0095] The following will be combined Figure 8 A cloud platform provided by this application is described.

[0096] See also Figure 8 Based on the above-mentioned task scheduling method and device, the embodiment of the present application also provides another cloud platform 100 that can execute the above-mentioned task scheduling method. The cloud platform 100 includes one or more (only one is shown in the figure) processors 102, a memory 104, and a communication module 106 that are coupled to each other. Among them, the memory 104 stores a program that can execute the content of the above-mentioned embodiment, and the processor 102 can execute the program stored in the memory 104.

[0097] Among them, the processor 102 may include one or more processing cores. The processor 102 uses various interfaces and lines to connect various parts within the entire cloud platform 100, and executes various functions and processes data of the cloud platform 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 104, and calling data stored in the memory 104. Optionally, the processor 102 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 102 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 102, and it can be implemented separately through a communication chip.

[0098] The memory 104 may include a random access memory (RAM) or a read-only memory (ROM). The memory 104 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 104 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the cloud platform 100 during use.

[0099] The communication module 106 is used to implement information exchange between the cloud platform 100 and other devices, for example, transmission of device control instructions, operation request instructions, and status information acquisition instructions, etc. When the cloud platform 100 is specifically different devices, the corresponding communication modules 106 may be different.

[0100] The embodiment of the present application also provides a structural block diagram of a computer-readable storage medium. The computer-readable storage medium stores program codes, and the program codes can be called by a processor to execute the method described in the above method embodiment.

[0101] The computer readable storage medium may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer readable storage medium has storage space for program codes that execute any of the method steps in the above method. These program codes may be read from or written to one or more computer program products. The program code may be compressed, for example, in an appropriate form.

[0102] In summary, the present application provides a task scheduling method, device, and cloud platform, which, in response to receiving a downstream task, obtains the status information of multiple vehicles to be assigned and the task information of the downstream task, and then, based on the status information, the task information, and a pre-trained task scheduling agent, performs downstream task assignments on the multiple vehicles to be assigned. Through the above-mentioned method, in response to receiving a downstream task, obtains the status information of multiple vehicles to be assigned and the task information of the downstream task, and, based on the status information, the task information, and a pre-trained task scheduling agent, performs downstream task assignments on multiple vehicles to be assigned, thereby making it possible to assign different downstream tasks based on the computing power, remaining power, etc. of the multiple vehicles to be assigned, thereby improving the utilization rate of computing power resources.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A task scheduling method, characterized in that: The method comprises: In response to receiving a downstream task, obtaining status information of a plurality of vehicles to be assigned and task information of the downstream task, wherein the status information includes remaining computing power, remaining power, and distance information, and the task information includes computing power and task amount required by the downstream task; Based on the state information, the task information and a pre-trained task scheduling agent, downstream tasks are assigned to the multiple vehicles to be assigned.

2. The method according to claim 1, characterized in that: In response to receiving a downstream task, obtaining status information of a plurality of vehicles to be assigned and task information of the downstream task includes: In response to receiving a downstream task, obtaining status information of a plurality of candidate vehicles and task information of the downstream task; Based on the status information of each candidate vehicle, determining whether the corresponding candidate vehicle meets the preset computing power condition; If the corresponding candidate vehicle meets the preset computing power condition, the corresponding candidate vehicle is used as the vehicle to be assigned to obtain the status information of the multiple vehicles to be assigned.

3. The method according to claim 2, characterized in that The preset computing power conditions include that the remaining computing power is greater than the computing power required by the downstream task, and the remaining power is greater than the power required to execute the downstream task.

4. The method according to claim 1, characterized in that: Before the step of obtaining the status information of the plurality of vehicles to be assigned and the task information of the downstream task in response to receiving the downstream task, the method further comprises: Acquire an initial training state, wherein the initial training state includes state information of each of a plurality of training vehicles and task information of each of a plurality of training tasks; Acquire a pre-constructed action space, wherein the action space includes the plurality of training vehicles and the plurality of training tasks; Obtaining a pre-designed reward function, wherein the reward function represents the benefits that can be generated after the initial task scheduling agent performs a task assignment; Based on the initial training state, the action space and the reward function, the initial task scheduling agent is trained to obtain the pre-trained task scheduling agent.

5. The method according to claim 4, characterized in that The reward function includes task completion rate, computing power utilization rate and strategy entropy.

6. The method according to claim 4, characterized in that The initial task scheduling agent includes an initial action network and an initial evaluation network. The initial task scheduling agent is trained based on the initial training state, the action space and the reward function to obtain the pre-trained task scheduling agent, including: In the current iteration process, based on the initial training state, the action space, the reward function and the initial task scheduling agent, an experience pool corresponding to the current iteration process is obtained, wherein the experience pool includes multiple experiences, each of which includes the current training state, the scheduling action, the reward value, and the next training state; In the current iteration process, part of the experience is collected from the experience pool to obtain a plurality of training samples; Based on the multiple training samples, updating the network parameters of the initial evaluation network to obtain an updated evaluation network corresponding to the current iteration process; Based on the multiple training samples, updating the network parameters of the initial action network to obtain an updated action network corresponding to the current iteration process; If the current iteration process meets the target conditions, the updated action network is used as the pre-trained task scheduling agent; if the current iteration process does not meet the target conditions, the next iteration process is entered, and the updated action network and the updated evaluation network are used as the initial action network and initial evaluation network of the next iteration process respectively.

7. The method according to claim 6, characterized in that In the current iteration process, based on the initial training state, the action space, the reward function and the initial task scheduling agent, an experience pool corresponding to the current iteration process is obtained, including: In the current time step of the current iteration process, the current training state corresponding to the current time step is input into the initial action network to obtain the scheduling action of the current time step from the action space, wherein the scheduling action includes a plurality of vehicle-task pairs, each of which represents a training task assigned to a training vehicle, wherein the current training state corresponding to the first time step is the initial training state; Execute the scheduling action of the current time step to obtain the next moment training state corresponding to the current time step; Obtaining a reward value for the current time step through the initial evaluation network and the reward function; The current training state, scheduling action, next training state, and reward value of the current time step are stored as an experience in the experience pool corresponding to the current iteration process; Continue to use the next moment training state of the current time step as the current moment training state of the next time step to obtain the scheduling action, next moment training state, and reward value of the next time step until the experience pool corresponding to the current iteration process is generated.

8. A task scheduling device, characterized in that: The device comprises: an information acquisition unit, in response to receiving a downstream task, acquiring status information of a plurality of vehicles to be assigned and task information of the downstream task, wherein the status information includes remaining computing power, remaining power, and distance information, and the task information includes computing power and task amount required by the downstream task; The task scheduling unit is used to perform downstream task allocation to the multiple vehicles to be allocated based on the state information, the task information and a pre-trained task scheduling agent.

9. A cloud platform, characterized in that: including one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 10.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, wherein when the program code is run, the method according to any one of claims 1 to 10 is executed.

Images