An emergency task scheduling method and device in a limited space

Abstract

The application relates to an emergency task scheduling method and device in a limited space. In the scene of multiple emergency tasks and autonomous selection of emergency tasks by a task executor, the method considers the feature that movement of the task executor will affect the task execution duration, takes the task execution duration of N emergency tasks as a state in a reinforcement learning model, takes a settable stage-type incentive value as an action in the reinforcement learning model, and under the influence of a reward, adjusts the action at different time steps through the reinforcement learning model, guides task executors with a short distance to select corresponding emergency tasks, realizes dynamic optimization scheduling of the task executors performing the emergency tasks, and minimizes the task execution duration of the multiple emergency tasks.

Description

Technical Field

[0001] This application relates to the field of task scheduling technology, and in particular to an emergency task scheduling method and apparatus within a limited space. Background Technology

[0002] Confined spaces typically refer to special working environments that are enclosed or partially enclosed and relatively isolated from the outside world, such as underground mines and tunnels. Traditional confined space operations mainly rely on manual labor, but due to the limitations of the space, workers cannot stay for extended periods, severely impacting work efficiency. With the rapid development of 5G communication technology, mobile work equipment is gradually replacing manual labor, enabling the execution of various tasks within confined spaces, ensuring both operational safety and improved work efficiency.

[0003] When performing multiple urgent tasks within a confined space, collaborative processing based on multiple mobile work devices can significantly improve task processing quality. However, existing collaborative processing schemes often overlook the negative impact of device mobility: as the distance between the mobile work device and the task issuing device (such as a base station) increases, channel quality deteriorates significantly due to communication distance and environmental interference (such as dust). This deterioration manifests as fluctuations in signal-to-noise ratio and a decrease in communication rate, leading to a substantial increase in the upload time of task execution results, thereby prolonging the overall task completion time.

[0004] Of particular note is that, in scenarios where mobile work equipment can autonomously select urgent tasks, how to intelligently schedule suitable mobile work equipment to execute urgent tasks by dynamically adjusting the parameter settings of the task issuing equipment, and ultimately reduce the overall execution time of multiple urgent tasks, has become a key technical challenge that urgently needs to be solved. Summary of the Invention

[0005] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0006] The main objective of this disclosure is to propose an emergency task scheduling method and apparatus within a limited space, which can guide a task executor at a suitable distance to select and execute the corresponding emergency task, thereby achieving dynamic optimization scheduling of the task executor executing the emergency task and minimizing the execution time of multiple emergency tasks.

[0007] The first aspect of this application proposes a method for scheduling N emergency tasks in an emergency task management system within a confined space, wherein the emergency task management system includes:

[0008] Multiple communication devices, each having at least one emergency task, are configured to issue emergency tasks to mobile task executors within their coverage area, issue multiple incentive values, and receive task execution results uploaded by the task executors; an emergency task is executed by at least one of the task executors; the multiple incentive values ​​are set based on the communication distance between the task executor and the communication device;

[0009] The system includes multiple task executors, each of which can choose to execute or exit an emergency task and upload the task execution results. When executing an emergency task, a task incentive is generated. The task incentive is determined based on an incentive value corresponding to the communication distance between the task executor and the communication device.

[0010] The method includes:

[0011] Obtain the execution time of each emergency task at the current time step; the execution time includes the task execution time of the task executor and the upload time of the task executor uploading the task execution result, and the upload time is related to the communication distance between the task executor and the communication device;

[0012] Using the execution time of N urgent tasks as the state in the reinforcement learning model, multiple incentive values ​​as actions in the reinforcement learning model, setting the reward of the reinforcement learning model, and learning to set the action for the next time step based on the reinforcement learning model with the objective function of minimizing the execution time of the N urgent tasks; the reward is associated with the objective function.

[0013] Scheduling is completed based on the decisions made at each time step.

[0014] The emergency task scheduling method within a limited space provided in this embodiment has at least the following advantages:

[0015] To address the aforementioned issues, in scenarios where task executors in a confined space can autonomously select urgent tasks for multi-urgent task processing, a reinforcement learning model is introduced, considering the characteristic that task executor movement affects the task's execution time. The model uses the execution time of N urgent tasks as a state factor, multiple configurable incentive values ​​as action factors, and a reward factor. Under the influence of the reward factor, the reinforcement learning model adjusts the action factors at different time steps, guiding the task executors to maximize their respective task incentives and select an urgent task issued by a nearby communication device. This minimizes the impact of task executor movement on the task's execution time, achieving dynamic optimization scheduling of task executors executing urgent tasks, ultimately minimizing the total execution time of multiple urgent tasks.

[0016] In some implementations, the function representing the reward is:

[0017]

[0018] Where r is the reward mapping function, α1 and α2 are weight values, and T n,sum Let T be the execution time of the nth urgent task. n,sum ′ is the nth emergency task in action a t The adjusted execution time of the task in the next time step, T n,expect Let T be the expected duration of the nth emergency task, min(T) n,sum ,T n,expect ) is from T n,sum and T n,expect The mapping function that takes the smaller value from T, min(T) n,sum ′,T n,expect ) is from T n,sum ′ and T n,expect The mapping function takes the smaller value from the two, where N is the number of multiple urgent tasks, Δt is the time taken in a time step, and (s) t ,a t ) represents the state s at time step t. t and action a t Mapping pairs.

[0019] In some implementations, the upload duration for obtaining the execution result of each task includes:

[0020]

[0021] v = w log2[1 + SNR];

[0022]

[0023] Where T2 is the upload duration, D is the data volume of the task execution result, v is the communication rate between the communication device and the task executor in the limited space, w is the communication bandwidth, SNR is the signal-to-noise ratio in the limited space, p1 is the channel fading factor, p2 is the path loss factor, l is the communication distance between the task executor and the communication device, P is the transmit power during upload, and ρ0 is the Gaussian noise density in the limited space.

[0024] In some implementations, the process of determining the task incentive includes:

[0025] Determine the current communication distance between the task executor and the communication device at the current time step;

[0026] Based on the current communication distance, determine the current excitation value;

[0027] Based on the task execution duration and the current incentive value, calculate the positive incentive for the task executor;

[0028] Determine the task execution cost and upload cost of the task executor, and calculate the total cost based on the task execution cost and the upload cost;

[0029] The difference between the positive incentive and the total cost is calculated to obtain the task incentive.

[0030] In some implementations, before determining the current excitation value based on the current communication distance, the method further includes:

[0031] The communication range of the communication device is divided into multiple communication range segments;

[0032] Different excitation values ​​are assigned to different communication range segments;

[0033] Determining the current excitation value based on the current communication distance includes:

[0034] Determine the communication range segment to which the current communication distance belongs;

[0035] The current excitation value for the current communication distance is based on the excitation value corresponding to the communication range segment to which it belongs.

[0036] In some implementations, before learning to set the decision for the action at the next time step based on the reinforcement learning model with the objective function of minimizing the execution time of the N urgent tasks, the method further includes:

[0037] Determine the number of task executors within the coverage area of ​​each of the communication devices within the current time step;

[0038] Based on the quantity, multiple performance standard values ​​are determined; the performance standard values ​​are used to determine the task executors participating in the emergency task, and only task executors with remaining performance higher than the performance standard values ​​are allowed to participate in the emergency task;

[0039] The step of using configurable incentive values ​​as actions in the reinforcement learning model includes:

[0040] The configurable incentive value and the configurable performance standard value are used as actions in the reinforcement learning model.

[0041] In some implementations, the reinforcement learning model is a DQN model.

[0042] A second aspect of this application provides an emergency task scheduling device within a confined space, applied to the scheduling of N emergency tasks in an emergency task management system within a confined space, wherein the emergency task management system includes:

[0043] Multiple communication devices, each having at least one emergency task, are configured to issue emergency tasks to mobile task executors within their coverage area, issue multiple incentive values, and receive task execution results uploaded by the task executors; an emergency task is executed by at least one of the task executors; the multiple incentive values ​​are set based on the communication distance between the task executor and the communication device;

[0044] The system includes multiple task executors, each of which can choose to execute or exit an emergency task and upload the task execution results. When executing an emergency task, a task incentive is generated. The task incentive is determined based on an incentive value corresponding to the communication distance between the task executor and the communication device.

[0045] The device includes:

[0046] The duration acquisition module is used to acquire the execution duration of each emergency task at the current time step; the execution duration includes the task execution duration of the task executor and the upload duration of the task executor uploading the task execution result, and the upload duration is related to the communication distance between the task executor and the communication device;

[0047] An action decision module is used to take the execution time of N urgent tasks as the state in a reinforcement learning model, take multiple incentive values ​​as actions in the reinforcement learning model, set the reward of the reinforcement learning model, and learn to set the action for the next time step based on the reinforcement learning model with the objective function of minimizing the execution time of the N urgent tasks; the reward is associated with the objective function.

[0048] The task scheduling module is used to complete the scheduling based on the decision at each time step.

[0049] A third aspect of this application provides an electronic device including at least one controller and a memory for communicatively connecting to the controller; the memory stores instructions executable by the at least one controller to cause the at least one controller to perform an emergency task scheduling method within a limited space as described above.

[0050] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed, implements an emergency task scheduling method within a limited space as described above.

[0051] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart illustrating an embodiment of an emergency task scheduling method within a limited space provided in this application;

[0054] Figure 2 This is a flowchart of the training process of DQN provided in this application;

[0055] Figure 3 This is a schematic diagram of an embodiment of an emergency task dispatching device in a confined space provided in this application;

[0056] Figure 4 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0059] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed or function in a specific orientation, and therefore should not be construed as a limitation of this application.

[0060] Confined spaces typically refer to special working environments that are enclosed or partially enclosed and relatively isolated from the outside world, such as underground mines and tunnels. Traditional confined space operations mainly rely on manual labor, but due to the limitations of the space, workers cannot stay for extended periods, severely impacting work efficiency. With the rapid development of 5G communication technology, mobile work equipment is gradually replacing manual labor, enabling the execution of various tasks within confined spaces, ensuring both operational safety and improved work efficiency.

[0061] When performing multiple urgent tasks within a confined space, collaborative processing based on multiple mobile work devices can significantly improve task processing quality. However, existing collaborative processing schemes often overlook the negative impact of device mobility: as the distance between the mobile work device and the task issuing device (such as a base station) increases, channel quality deteriorates significantly due to communication distance and environmental interference (such as dust). This deterioration manifests as fluctuations in signal-to-noise ratio and a decrease in communication rate, leading to a substantial increase in the upload time of task execution results, thereby prolonging the overall task completion time.

[0062] Of particular note is that, in scenarios where mobile work equipment can autonomously select urgent tasks, how to intelligently schedule suitable mobile work equipment to execute urgent tasks by dynamically adjusting the parameter settings of the task issuing equipment, and ultimately reduce the overall execution time of multiple urgent tasks, has become a key technical challenge that urgently needs to be solved.

[0063] like Figure 1 and Figure 2 In one embodiment of this application, to address the aforementioned technical deficiencies, an emergency task scheduling method within a limited space is provided for use in an emergency task scheduling system, comprising the following steps S100 to S300:

[0064] Step S100: Obtain the execution duration of each emergency task at the current time step; the execution duration includes the task execution duration of the task executor and the upload duration of the task executor uploading the task execution result. The upload duration is related to the communication distance between the task executor and the communication device.

[0065] Step S200: Using the execution time of N urgent tasks as the state in the reinforcement learning model, multiple incentive values ​​as actions in the reinforcement learning model, setting the reward of the reinforcement learning model, and learning the decision to set the action for the next time step based on the reinforcement learning model with the objective function of minimizing the execution time of N urgent tasks; the reward is associated with the objective function.

[0066] Step S300: Complete the scheduling based on the decision made at each time step.

[0067] First, let's introduce the emergency task dispatch system:

[0068] Communication equipment can be any device capable of enabling data communication and a certain level of data processing within a confined space, such as a base station. Of course, different communication devices can be deployed in different confined spaces. It's also important to note that multiple communication devices can achieve full communication coverage within a confined space.

[0069] A task executor, capable of moving within a confined space, is equipped with a microcontroller unit (MCU). When within the communication range of a communication device, the executor can interconnect with the device via the MCU. For example, in an underground mine scenario where the task is to collect the composition of harmful gases from various areas, the task executor could be a robot or a drone. Similarly, in an underground tunnel scenario where the task is to explore geological components, the task executor could be a tunneling machine or a trackless rubber-wheeled vehicle.

[0070] A high-performance communication device can be selected from the communication devices as the decision-maker for steps S100 to S300.

[0071] The main functions of the above components are described below in chronological order:

[0072] The decision-maker counts N emergency tasks and sends them to multiple communication devices, ensuring that each communication device has at least one emergency task. For example, there are N communication devices and N emergency tasks, and each communication device assists in handling one emergency task.

[0073] The aforementioned communication equipment issues emergency tasks to all task executors within its coverage area and sets multiple incentive values ​​distributed in stages. These incentive values ​​are reward parameters that can be dynamically adjusted according to the task's progress. Throughout the execution of the emergency task, the task executor, influenced by the task incentives, selects to undertake the emergency task. Therefore, in scenarios where the task executor has autonomous decision-making capabilities, choosing an appropriate incentive value is key to reducing task duration.

[0074] In this embodiment, the incentive value is related to the communication distance between the task executor and the communication device. Due to the influence of limited space (such as signal-to-noise ratio), the longer the communication distance between the task executor and the communication device, the longer it takes for the task execution result to be uploaded (it should be noted that the movement of the executor does not affect the task execution time of the executor). Therefore, a smaller incentive value can be set for the executor with a longer communication distance from the communication device, and conversely, a larger incentive value can be set for the executor with a shorter communication distance from the communication device. This guides the executor to select the emergency task issued by the nearest communication device for task incentive (the larger the incentive value, the greater the task incentive).

[0075] For example, the incentive value is inversely proportional to the communication distance between the task executor and the communication device. The greater the distance, the longer the transmission of the task execution result between the task executor and the communication device takes. To minimize the task executor's participation in urgent tasks, a lower incentive value can be set to guide it away from such tasks. Conversely, to encourage task executors closer to the communication device to participate in urgent tasks, a higher incentive value can be set. Since different task executors are at different distances from the communication device, influenced by the incentive value, task executors closer to the communication device receive higher rewards and are more willing to execute the task. Conversely, those farther away receive lower rewards, and may consider participating in other urgent tasks with higher incentive values.

[0076] It is worth noting that multiple incentive values ​​can be sent as key-value pairs, and at different time steps, the values ​​of multiple incentive values ​​can be adjusted based on subsequent model-derived decisions.

[0077] The task executor can choose whether to execute the corresponding emergency task based on the incentive value published by the associated communication device (within its coverage area, where it can receive published content) and the distance. Furthermore, the task executor can choose to exit midway. For example, within a confined space, the task executor might move from a position closer to the communication device to a position farther away. In this case, the task executor's incentive value gradually decreases, and it might choose to exit the current emergency task and select a new one if it receives another emergency task with a higher incentive value.

[0078] Under the influence of incentive values, the task executor accepts and executes urgent tasks, generating rewards (i.e., task incentives). At each time step, the task executor uploads the task execution result (i.e., the result of executing the task between two time steps) to the publisher (communication device). Additionally, the task executor may choose to exit midway through its journey.

[0079] After receiving the task execution results, the communication equipment reports them to the decision-maker;

[0080] Based on the aggregated information, the decision-maker uses a reinforcement learning model to make the next scheduling decision. It is important to note that the magnitudes of multiple incentive values ​​are adjusted to guide the task executor to make relevant decisions.

[0081] The following section focuses on this method:

[0082] In step S100, the decision-maker obtains the execution duration of each urgent task at the current time step. This is specifically based on the uploaded task execution results. For example, the task execution results record timestamps, execution processes, and execution status; however, specific limitations are not specified here due to varying circumstances. The task's execution duration includes two parts:

[0083] The first part is the task execution time of the task executor, that is, the time it takes for the task executor to execute the task.

[0084] The second part is the upload time of the task executor, that is, the time it takes for the task executor to upload the task execution results to the publisher. Due to the limited space environment, the upload time varies for task executors at different distances.

[0085] In some embodiments, the upload duration for obtaining the execution result of each task includes:

[0086]

[0087] v = w log2[1+SNR] (2);

[0088]

[0089] Formula (2) is derived based on the Shannon function. Where T2 is the upload time, D is the data volume of the task execution result, v is the communication rate between the communication device and the task executor in the finite space, w is the communication bandwidth, SNR is the signal-to-noise ratio in the finite space, p1 is the channel fading factor, p2 is the path loss factor, l is the communication distance between the task executor and the communication device, P is the transmit power during upload, and ρ0 is the Gaussian noise density in the finite space.

[0090] As can be seen from the above formulas (1) to (3), different distances will have a significant impact on the upload time due to the influence of distance l. Therefore, the decision executor needs to consider the distance factor and adjust the appropriate phased incentive value to prompt the task executor to execute the corresponding emergency task issued by the communication device that is closer in distance.

[0091] It should be noted that in this application, the task executor actively accepts tasks, while the decision-maker does not actively screen task executors. Moreover, the decision-maker adjusts the incentive value to encourage the task executor to actively accept urgent tasks issued by nearby communication devices, thereby reducing the total task execution time.

[0092] In some embodiments, the process of determining the task stimulus of the task executor includes the following steps S110 to S150:

[0093] Step S110: Determine the current communication distance between the task executor and the communication device at the current time step; this can be determined based on the message transmission time or on the map coordinates based on the positioning system, and is not limited here.

[0094] Step S120: Divide the communication range of the communication device into multiple communication range segments; for example, divide it into 5 segments, and give each communication range segment a different excitation value. This can be set based on experience and is not limited here.

[0095] Step S130: Assign different excitation values ​​to different communication range segments;

[0096] Step S140: Determine the current excitation value based on the current communication distance;

[0097] Threshold judgment can be used here. For example, in step S140, the communication range segment to which the current communication distance belongs is determined; the excitation value corresponding to the communication range segment to which it belongs is used as the current excitation value of the current communication distance.

[0098] Step S150: Calculate the positive incentive for the task executor based on the task execution duration and the current incentive value; this part belongs to the positive income of the executor.

[0099] Step S160: Determine the task execution cost and upload cost of the task executor, and calculate the total cost based on the task execution cost and upload cost. This part belongs to the cost of the executor. It should be noted that the cost calculation formulas for task execution cost and upload cost are different depending on the task executor. For example, the energy consumption costs of drones and robots are different, but this is not limited here.

[0100] Step S170: Calculate the difference between the positive incentive and the total cost to obtain the task incentive. Subtract the expenditure from the positive income to obtain the total income.

[0101] In this embodiment, the above steps can prompt the task executor to select to execute or exit an emergency task based on the communication distance, thereby reducing the execution time of multiple emergency tasks.

[0102] Furthermore, step S200 introduces the selection of the reinforcement learning model for decision-making (i.e., how to appropriately set the values ​​corresponding to multiple incentives), for example, the DQN model, such as... Figure 2 As shown.

[0103] The following section introduces the model, which is based on Markov chains. The training process is illustrated with pseudocode:

[0104] Table 1

[0105]

[0106]

[0107] This section introduces DQN, using the execution time of N urgent tasks as the state in the reinforcement learning model, and setting the configurable incentive values ​​as the actions in the reinforcement learning model, setting rewards; based on the reinforcement learning model, using the state, actions, and rewards, and with the objective function of minimizing the execution time of the tasks, it learns the optimal decision for choosing the action at the next time step.

[0108] For example:

[0109] Status: s t =(T 1,sum ,T 2,sum ,...,T n,sum ,...T N,sum ), where s represents the execution time of the N urgent tasks. t The subscript t in the text represents a time step (stage).

[0110] Action: a t =D, where D is the stimulus value corresponding to the communication range segment. The variable part is the adjustment of the stimulus value. For example, the decision maker can set a stimulus value for each communication range segment, and the stimulus value has 5 numerical items. The purpose of the decision is to determine the value of the stimulus value corresponding to each communication range segment in the next time step.

[0111] After training, DQN uses the evaluation network to determine the action with the maximum cumulative reward (selecting the value of the incentive) to regulate the selection of task executors, thereby minimizing the execution time of N urgent tasks.

[0112] The state-action value function of DQN is:

[0113]

[0114] Where E is the expected value, γ is the discount factor, and r(s) t ,a t ) is a mapping pair of states and actions (s) t ,a t When the time step is T, the model outputs a reward, which is a reward function set according to the desired objective. T is the total number of time steps.

[0115] A greedy algorithm is used to select actions at the first time step, and subsequent actions are based on argmax. a Q(s t ,a t Use θ) to select the action.

[0116] This embodiment has at least the following beneficial effects:

[0117] To address the aforementioned issues, in scenarios where task executors in a confined space can autonomously select urgent tasks for multi-urgent task processing, a reinforcement learning model is introduced, considering the characteristic that task executor movement affects the task's execution time. The model uses the execution time of N urgent tasks as a state factor, multiple configurable incentive values ​​as action factors, and a reward factor. Under the influence of the reward factor, the reinforcement learning model adjusts the action factors at different time steps, guiding the task executors to maximize their respective task incentives and select an urgent task issued by a nearby communication device. This minimizes the impact of task executor movement on the task's execution time, achieving dynamic optimization scheduling of task executors executing urgent tasks, ultimately minimizing the total execution time of multiple urgent tasks.

[0118] Furthermore, the reward function r is represented as:

[0119]

[0120] Where r is the reward mapping function, α1 and α2 are weight values, and T n,sum Let T be the execution time of the nth urgent task. n,sum ′ is the nth emergency task in action a t The adjusted execution time of the task in the next time step, T n,expect Let T be the expected duration of the nth emergency task, min(T) n,sum ,T n,expect ) is from T n,sum and T n,expect The mapping function that takes the smaller value from T, min(T) n,sum ′,T n,expect ) is from T n,sum ′ and T n,expect The mapping function takes the smaller value from the two, where N is the number of multiple urgent tasks, Δt is the time taken in a time step, and (s) t ,a t ) represents the state s at time step t. t and action a t Mapping pairs.

[0121] The reward function r here includes: determining T n,sum and T n,expect The minimum value between, and based on (s) t ,a t T under the action of ) n,sum ′ and T n,expectThe minimum value between two minimum values ​​is determined, and the difference between these two minimum values ​​is used to preserve the contribution of the best policies selected in the reinforcement learning model as much as possible. This allows urgent tasks with long execution times to be reduced through decision-making. Furthermore, a time step duration is set, and a fixed penalty is applied at each step based on Δt to encourage faster completion of urgent tasks.

[0122] Furthermore, before learning the optimal decision for selecting the action at the next time step based on a reinforcement learning model that utilizes states, actions, and rewards, and with the objective function of minimizing the time elapsed since the task was completed, the method also includes:

[0123] Step S210: Determine the number of task executors within the coverage area of ​​each communication device in the current time step;

[0124] Step S220: Based on the quantity, set multiple performance standard values; the performance standard values ​​are used to determine the task executors participating in the emergency task, and only the remaining task executors with performance higher than the performance standard value are allowed to participate in the emergency task.

[0125] Step S200, which involves using the optional activation value as an action in the reinforcement learning model, includes:

[0126] Step S230: Selectable activation values ​​and selectable performance criterion values ​​are used as actions in the reinforcement learning model.

[0127] This method dynamically adjusts performance criterion values, adaptively relaxing or tightening conditions based on the number of real-time task executors to ensure efficient allocation of urgent tasks. By dynamically setting performance criterion values ​​and incorporating them into the decision-making process of a reinforcement learning model, the system can automatically adjust the selection strategy of intervention task executors under different device counts, prioritizing high-performance devices for urgent tasks, thereby reducing duplicate executions and communication delays caused by device failures or inefficiencies. Simultaneously, combined with incentive value adjustment strategies, it further guides task executors to optimize urgent task selection while ensuring performance, ultimately minimizing the execution time of multiple urgent tasks.

[0128] like Figure 3 According to one embodiment of this application, an emergency task scheduling device for a confined space is provided, which is applied to the scheduling of N emergency tasks in an emergency task management system within a confined space. The emergency task management system includes:

[0129] Multiple communication devices, each with at least one emergency task, are used to issue emergency tasks to mobile task executors within their coverage area, issue multiple incentive values, and receive task execution results uploaded by the task executors; an emergency task is executed by at least one task executor; multiple incentive values ​​are set according to the communication distance between the task executor and the communication device;

[0130] Multiple task executors are available. Each task executor can choose to execute or exit an emergency task and upload the task execution results. When executing an emergency task, a task incentive is generated. The task incentive is determined based on an incentive value corresponding to the communication distance between the task executor and the communication device.

[0131] The device includes:

[0132] The duration acquisition module is used to obtain the execution duration of each emergency task at the current time step; the execution duration includes the task execution duration of the task executor and the upload duration of the task execution result uploaded by the task executor. The upload duration is related to the communication distance between the task executor and the communication device.

[0133] The action decision module is used to take the execution time of N urgent tasks as the state in the reinforcement learning model, take multiple incentive values ​​as the actions in the reinforcement learning model, set the reward of the reinforcement learning model, and learn to set the action for the next time step based on the reinforcement learning model with the objective function of minimizing the execution time of N urgent tasks; the reward is associated with the objective function.

[0134] The task scheduling module is used to complete scheduling based on the decisions made at each time step.

[0135] It is worth noting that the embodiments of the emergency task scheduling device in this confined space and the embodiments of the emergency task scheduling method in the confined space described above are based on the same inventive concept. Therefore, the relevant content of the embodiments of the emergency task scheduling method in the confined space described above also applies to the embodiments of the emergency task scheduling device in this confined space, and will not be repeated here.

[0136] Reference Figure 4 This application also provides an electronic device, which includes:

[0137] At least one memory;

[0138] At least one processor;

[0139] At least one program;

[0140] The program is stored in memory, and the processor executes at least one program to implement the emergency task scheduling method within a limited space described above in this disclosure.

[0141] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0142] The electronic devices according to embodiments of this application will now be described in detail.

[0143] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0144] The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store function devices and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the program code is stored in the memory 1700 and is invoked by the processor 1600 to execute the emergency task scheduling method within the limited space of the embodiments of this application.

[0145] The input / output interface 1800 is used to implement information input and output.

[0146] The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0147] Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900);

[0148] The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0149] This application also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described emergency task scheduling method within a limited space.

[0150] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0151] The embodiments described in this application are intended to more clearly illustrate the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0152] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0153] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0154] Those skilled in the art will understand that all or some of the steps, apparatuses, or functional modules / units in the methods disclosed above can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0155] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0156] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0157] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0158] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0159] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0160] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0161] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

Claims

1. A method for scheduling emergency tasks within a limited space, characterized in that, In the scheduling of N emergency tasks within an emergency task management system in a confined space, the emergency task management system includes: Multiple communication devices, each having at least one emergency task, are configured to issue emergency tasks to mobile task executors within their coverage area, issue multiple incentive values, and receive task execution results uploaded by the task executors; an emergency task is executed by at least one of the task executors; the multiple incentive values ​​are set based on the communication distance between the task executor and the communication device; The system includes multiple task executors, each of which can choose to execute or exit an emergency task and upload the task execution results. When executing an emergency task, a task incentive is generated. The task incentive is determined based on an incentive value corresponding to the communication distance between the task executor and the communication device. The method includes: Obtain the execution time of each emergency task at the current time step; the execution time includes the task execution time of the task executor and the upload time of the task executor uploading the task execution result, and the upload time is related to the communication distance between the task executor and the communication device; Using the execution time of N urgent tasks as the state in the reinforcement learning model, multiple incentive values ​​as actions in the reinforcement learning model, setting the reward of the reinforcement learning model, and learning to set the action for the next time step based on the reinforcement learning model with the objective function of minimizing the execution time of the N urgent tasks; the reward is associated with the objective function. Scheduling is completed based on the decisions made at each time step.

2. The emergency task scheduling method within a confined space according to claim 1, characterized in that, The function representing the reward is: ； in, For the reward mapping function, and For weight values, For the first The duration of the urgent task has been completed. For the first An urgent task in action The adjusted execution time of the task in the next time step. For the first The expected duration of an emergency mission From and The mapping function that takes the smaller value from the middle. From and The mapping function that takes the smaller value from the middle. For the number of multiple emergency tasks, The time consumed by the time step. For time step status and actions Mapping pairs.

3. The emergency task scheduling method within a confined space according to claim 1, characterized in that, The upload time for obtaining the execution result of each task includes: ； ； ； in, For upload duration, The amount of data resulting from the task execution. The communication rate between the communication device and the task executor within a limited space. It is communication bandwidth. For the signal-to-noise ratio in a finite space, Channel fading factor This is the path loss factor. The communication distance between the task executor and the communication device. This refers to the transmit power during uploading. This represents the Gaussian noise density within a finite space.

4. The emergency task scheduling method in a confined space according to claim 3, characterized in that, The process of determining the task incentive includes: Determine the current communication distance between the task executor and the communication device at the current time step; Based on the current communication distance, determine the current excitation value; Based on the task execution duration and the current incentive value, calculate the positive incentive for the task executor; Determine the task execution cost and upload cost of the task executor, and calculate the total cost based on the task execution cost and the upload cost; The difference between the positive incentive and the total cost is calculated to obtain the task incentive.

5. The emergency task scheduling method in a confined space according to claim 4, characterized in that, Before determining the current excitation value based on the current communication distance, the method further includes: The communication range of the communication device is divided into multiple communication range segments; Different excitation values ​​are assigned to different communication range segments; Determining the current excitation value based on the current communication distance includes: Determine the communication range segment to which the current communication distance belongs; The current excitation value for the current communication distance is based on the excitation value corresponding to the communication range segment to which it belongs.

6. The emergency task scheduling method in a confined space according to claim 5, characterized in that, Before learning the decision to set the action for the next time step based on the reinforcement learning model with the objective function of minimizing the execution time of the N urgent tasks, the method further includes: Determine the number of task executors within the coverage area of ​​each of the communication devices within the current time step; Based on the quantity, multiple performance standard values ​​are determined; the performance standard values ​​are used to determine the task executors participating in the emergency task, and only task executors with remaining performance higher than the performance standard values ​​are allowed to participate in the emergency task; Using configurable incentive values ​​as actions in the reinforcement learning model includes: The configurable incentive value and the configurable performance standard value are used as actions in the reinforcement learning model.

7. The emergency task scheduling method in a confined space according to claim 1, characterized in that, The reinforcement learning model is the DQN model.

8. An emergency task dispatching device for a confined space, characterized in that, In the scheduling of N emergency tasks within an emergency task management system in a confined space, the emergency task management system includes: Multiple communication devices, each having at least one emergency task, are configured to issue emergency tasks to mobile task executors within their coverage area, issue multiple incentive values, and receive task execution results uploaded by the task executors; an emergency task is executed by at least one of the task executors; the multiple incentive values ​​are set based on the communication distance between the task executor and the communication device; The system includes multiple task executors, each of which can choose to execute or exit an emergency task and upload the task execution results. When executing an emergency task, a task incentive is generated. The task incentive is determined based on an incentive value corresponding to the communication distance between the task executor and the communication device. The device includes: The duration acquisition module is used to acquire the execution duration of each emergency task at the current time step; the execution duration includes the task execution duration of the task executor and the upload duration of the task executor uploading the task execution result, and the upload duration is related to the communication distance between the task executor and the communication device; An action decision module is used to take the execution time of N urgent tasks as the state in a reinforcement learning model, take multiple incentive values ​​as actions in the reinforcement learning model, set the reward of the reinforcement learning model, and learn to set the action for the next time step based on the reinforcement learning model with the objective function of minimizing the execution time of the N urgent tasks; the reward is associated with the objective function. The task scheduling module is used to complete the scheduling based on the decision at each time step.

9. An electronic device, characterized in that, It includes at least one controller and a memory for communicatively connecting with the controller; the memory stores instructions executable by the at least one controller, which, when executed by the at least one controller, causes the at least one controller to perform an emergency task scheduling method within a confined space as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the emergency task scheduling method within a limited space as described in any one of claims 1 to 7.

Images