Unmanned device cluster management system, method, device, medium and program product

Abstract

One or more embodiments of the present specification provide an unmanned device cluster management system, method, device, medium and program product. The system comprises an information acquisition module used for acquiring a to-be-executed task, unmanned equipment cluster information, unmanned equipment available resource information and initial environment information; the intelligent decision-making module is used for splitting a task to be executed into a plurality of sub-tasks according to the information collected by the information collection module, and determining equipment information required for executing each sub-task and available resources of the sub-tasks; the intelligent planning module is used for determining task planning corresponding to the subtasks, and the task planning comprises resource allocation, an execution path and required equipment; the intelligent scheduling module is used for scheduling adaptive target unmanned equipment to execute subtasks according to the equipment state and task planning; the feedback receiving module is used for dynamically receiving execution data fed back by the target unmanned equipment; and the execution display module is used for synchronously displaying the corresponding virtual execution instance in the virtual simulation scene based on the execution data.

Description

Technical Field

[0001] One or more embodiments of this specification relate to the field of unmanned equipment technology, and in particular, to an unmanned equipment cluster management system, method, device, medium, and program product. Background Art

[0002] With the rapid development of science and technology, unmanned equipment such as drones, autonomous vehicles, and autonomous ships have been widely used in various fields. Each of these unmanned devices has its own unique characteristics, making them suitable for different scenarios. Drones, with their flexible flight capabilities and rapid response times, are suitable for tasks such as aerial reconnaissance, logistics distribution, and emergency rescue. Unmanned vehicles, with their strong ground maneuverability and load-carrying capacity, excel in logistics and transportation, security patrols, and exploration of complex terrain. Unmanned ships play an important role in areas such as water monitoring, ocean mapping, and water rescue.

[0003] However, in many complex missions, a single type of unmanned device often falls short of meeting the requirements, necessitating the coordinated operation of multiple devices. However, significant differences in performance, specifications, and equipment limitations exist among different unmanned devices, posing significant challenges to collaborative operation. For example, adverse weather conditions such as strong winds, heavy rain, and dense fog can severely impact a drone's flight stability and control capabilities, while complex terrain, such as mountains and hills, can also increase operational instability. Consequently, achieving efficient, real-time collaboration between unmanned devices and managing their clusters has become a pressing technical challenge in the field of unmanned systems. Summary of the Invention

[0004] In view of this, one or more embodiments of this specification provide the following technical solutions:

[0005] According to a first aspect of one or more embodiments of this specification, an unmanned equipment cluster management system is proposed, including:

[0006] An information collection module is used to collect information about tasks to be performed, unmanned equipment cluster information, unmanned equipment available resource information, and initial environment information of the tasks to be performed;

[0007] an intelligent decision-making module, configured to split the task to be executed into a plurality of subtasks, and determine the device information and subtask available resources required to execute each subtask, based on the task to be executed, the unmanned device cluster information, the unmanned device available resource information, and the initial environment information, with the goal of maximizing task completion efficiency, maximizing available resource utilization, minimizing energy consumption of the unmanned device cluster, and minimizing negative impact of the environment on the unmanned device cluster;

[0008] An intelligent planning module, which is used to determine the task plan corresponding to each subtask according to the required device information of the subtask and the available resources of the subtask. The task plan includes the correspondence between the available resources of the subtask, the execution path, and the required devices. Among them, the available resources of the subtask are located on the execution path corresponding to the required device or the distance between the available resources and the execution path is within a preset distance.

[0009] An intelligent scheduling module, which is used to obtain the device status of the unmanned devices in the unmanned device cluster, and schedule the appropriate target unmanned device according to the device status and the task plan to execute the subtask by using the corresponding available resources of the subtask based on the corresponding execution path.

[0010] A feedback receiving module, which is used to dynamically receive the execution data fed back by the target unmanned device during the execution of the subtask.

[0011] An execution display module, which is used to synchronously display the corresponding virtual execution instance in the virtual simulation scenario based on the execution data.

[0012] Optionally, the intelligent decision-making module splits the task to be executed and determines the device information and the available resources of the subtask in the following way:

[0013] Obtain an optimization function, where the optimization function is a weighted function of multiple sub-functions. Each sub-function corresponds to one or more subtasks, including an efficiency term corresponding to the task completion efficiency, an energy consumption term corresponding to the cluster energy consumption, an environmental impact term corresponding to the negative impact, and a resource utilization term corresponding to the available resource utilization rate.

[0014] With the goal of minimizing the optimization function, and with the constraints that the negative impact of the initial environmental information on the device cluster is less than the environmental impact threshold, the execution duration of the task to be executed is less than the duration threshold, the device information meets the device restrictions corresponding to the unmanned device cluster, and the available resources used by the task do not exceed the available resource limit, solve the optimization function to obtain the subtasks and the device information and available resources of each subtask.

[0015] Optionally, the feedback receiving module is further used to determine the running state of the corresponding target unmanned device based on the execution data.

[0016] The intelligent scheduling module is further used to determine the available unmanned devices in the case where the running state of the target unmanned device is abnormal, and schedule the subtasks not completed by the abnormal target unmanned device to the available unmanned devices, so that the available unmanned devices continue to execute the subtasks based on the corresponding execution path.

[0017] Optionally, the execution data further includes the current environmental information detected by the target unmanned device;

[0018] The feedback receiving module is further configured to determine whether there is a significant change in the task environment of the to-be-executed task based on the current environmental information and the initial environmental information;

[0019] The intelligent decision-making module is further configured to, in the case of a significant change in the task environment, determine the remaining tasks in the to-be-executed task that have not been executed, and re-split the remaining tasks based on the remaining tasks, the unmanned device cluster information, and the current environmental information, and determine the device information required for each sub-remaining task after re-splitting.

[0020] Optionally, the intelligent decision-making module is further configured to obtain expert knowledge related to the to-be-executed task, the unmanned device cluster, and the initial environment from an expert knowledge base;

[0021] The process by which the intelligent decision-making module splits the to-be-executed task and determines the device information includes:

[0022] According to the expert knowledge, split the to-be-executed task with the goals of maximizing task completion efficiency, maximizing the utilization rate of available resources, minimizing the energy consumption of the unmanned device cluster, and minimizing the negative impact of the environment on the unmanned device cluster, and determine the device information and the available resources of the sub-tasks.

[0023] Optionally, it further includes:

[0024] A knowledge maintenance module, configured to update the expert knowledge in the expert knowledge base according to the execution data.

[0025] Optionally, the expert knowledge is derived from a knowledge graph of unmanned device task implementation, and the construction of the knowledge graph is based on one or more of knowledge, experience, industry standards, environmental information (terrain, obstacles, climate, etc.), and specification parameters of unmanned devices related to unmanned devices, including unmanned device entities, task entities, and environmental entities.

[0026] According to a second aspect of one or more embodiments of the present specification, a method for managing an unmanned device cluster is provided, the method including:

[0027] Collect the to-be-executed task, unmanned device cluster information, unmanned device available resource information, and the initial environmental information of the to-be-executed task;

[0028] According to the to-be-executed task, the unmanned device cluster information, the available resources information of the unmanned devices, and the initial environment information, with the goals of maximizing the task completion efficiency, maximizing the utilization rate of available resources, minimizing the energy consumption of the unmanned device cluster, and minimizing the negative impact of the environment on the unmanned device cluster, split the to-be-executed task into multiple subtasks, and determine the device information required for each subtask and the available resources for the subtasks;

[0029] For each subtask, determine the task plan corresponding to the subtask according to the required device information and the available resources for the subtask, where the task plan includes the correspondence between the available resources for the subtask, the execution path, and the required devices, and the available resources for the subtask are located on the execution path corresponding to the required device or the distance between the available resources for the subtask and the execution path is within a preset distance;

[0030] Obtain the device status of the unmanned devices in the unmanned device cluster, and according to the device status and the task plan, schedule the adapted target unmanned devices to execute the subtask by using the corresponding available resources for the subtasks based on the corresponding execution paths;

[0031] Dynamically receive the execution data fed back by the target unmanned devices during the execution of the subtasks;

[0032] Synchronously display the corresponding virtual execution instances in the virtual simulation scenario based on the execution data.

[0033] According to the third aspect of one or more embodiments of this specification, an electronic device is provided, including: a processor; a memory for storing processor-executable instructions; wherein, the processor realizes the steps of the foregoing method by running the executable instructions.

[0034] According to the fourth aspect of one or more embodiments of this specification, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the foregoing method are realized.

[0035] According to the fifth aspect of one or more embodiments of this specification, a computer program product is provided, including computer programs / instructions, and when the computer programs / instructions are executed by a processor, the steps of the foregoing method are realized.

[0036] As can be seen from the above description, in the unmanned device cluster management system provided in this specification, the intelligent decision-making module can be based on the information collection module to collect the to-be-executed tasks, unmanned device cluster information, available resources information of unmanned devices, and initial environment information. With the goal of maximizing the task completion efficiency, maximizing the utilization rate of available resources, minimizing the energy consumption of the unmanned device cluster, and minimizing the negative impact of the environment on the unmanned device cluster, the to-be-executed task is split into multiple subtasks, and the device information and subtask available resources required for executing each subtask are determined. Furthermore, on the basis of optimizing multiple objectives simultaneously, the to-be-executed task is split, improving the quality of task splitting and reducing the difficulty of subsequent task planning. Then, the intelligent planning module can determine task plans such as the execution path and available resource allocation of each subtask, and the intelligent scheduling module can schedule the appropriate target unmanned devices according to the current device status of the unmanned devices to execute the subtask by using the corresponding subtask available resources according to the execution path, thereby realizing the flexible scheduling of various unmanned devices and also realizing efficient real-time collaborative operations. At the same time, the execution display module in the unmanned device cluster management system can also synchronously display the corresponding virtual execution instances in the virtual simulation scenario based on the execution data of the target unmanned devices dynamically received by the feedback receiving module, facilitating relevant personnel to monitor the execution status of tasks in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a schematic structural diagram of an unmanned device cluster management system provided by an exemplary embodiment.

[0038] Figure 2 is a schematic diagram of an unmanned device cluster management system provided by an exemplary embodiment.

[0039] Figure 3 is a schematic diagram of another unmanned device cluster management system provided by an exemplary embodiment.

[0040] Figure 4 is a schematic diagram of the interaction between a virtual simulation scenario and a field scenario provided by an exemplary embodiment.

[0041] Figure 5 is a flowchart of a method for splitting a to-be-executed task and determining device information provided by an exemplary embodiment.

[0042] Figure 6 is a flowchart of an unmanned device cluster management method provided by an exemplary embodiment.

[0043] Figure 7 is a schematic structural diagram of a device provided by an exemplary embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] With the rapid development of science and technology, unmanned equipment such as drones, autonomous vehicles, and autonomous ships have been widely used in various fields. Each of these unmanned devices has its own unique characteristics, making them suitable for different scenarios. Drones, with their flexible flight capabilities and rapid response times, are suitable for tasks such as aerial reconnaissance, logistics distribution, and emergency rescue. Unmanned vehicles, with their strong ground maneuverability and load-carrying capacity, excel in logistics and transportation, security patrols, and exploration of complex terrain. Unmanned ships play an important role in areas such as water monitoring, ocean mapping, and water rescue.

[0045] However, in many complex missions, a single type of unmanned device often falls short of meeting the requirements, necessitating the coordinated operation of multiple devices. However, significant differences in performance, specifications, and equipment limitations exist among different unmanned devices, posing significant challenges to collaborative operation. For example, adverse weather conditions such as strong winds, heavy rain, and dense fog can severely impact a drone's flight stability and control capabilities, while complex terrain, such as mountains and hills, can also increase operational instability. Consequently, achieving efficient, real-time collaboration between unmanned devices and managing their clusters has become a pressing technical challenge in the field of unmanned systems.

[0046] This manual provides an unmanned equipment cluster management system. Heterogeneous unmanned equipment and heterogeneous unmanned equipment clusters such as unmanned vehicles, drones, and unmanned ships can be connected to this unmanned equipment cluster management system to achieve the planning and scheduling of complex tasks.

[0047] Figure 1 This is a schematic diagram of the architecture of an unmanned equipment cluster management system provided by an exemplary embodiment. Figure 1 As shown, unmanned devices such as drones, unmanned vehicles, and unmanned ships can all be connected to the unmanned device cluster management system. The physical carrier of the unmanned device cluster management system can be a physical server of an independent host, or a virtual server hosted by a host cluster, etc. This specification does not impose any special restrictions on this.

[0048] As for the network for interaction between unmanned equipment such as drones, unmanned vehicles, and unmanned ships and unmanned equipment cluster management systems, it can be based on the communication methods supported by the corresponding unmanned equipment, such as choosing to use wireless networks to achieve communication, etc. Specifically, data interaction can be achieved through preset protocols and standardized interfaces.

[0049] It should be noted that for each heterogeneous unmanned equipment, Figure 1The figure only shows one corresponding device. In actual applications, clusters of driverless vehicles, drones, and unmanned boats can all be connected to the unmanned device management cluster provided in this specification. Taking the cluster of driverless vehicles as an example, the driverless vehicles in this cluster can include heterogeneous driverless vehicles of various different specifications / models, and the performance of driverless vehicles of different specifications / models is usually different. Similarly, the drone cluster can also include heterogeneous drones of various different specifications / models, and the unmanned boat cluster can also include heterogeneous unmanned boats of various different specifications / models.

[0050] It can be seen from this that the unmanned device cluster management system provided in this specification is adaptable to various heterogeneous unmanned device clusters, with a wider adaptability.

[0051] Please refer to Figure 2 , in an exemplary embodiment, the unmanned device cluster management system 200 may include an information collection module 201, an intelligent decision-making module 202, an intelligent planning module 203, an intelligent scheduling module 204, a feedback receiving module 205, and an execution display module 206.

[0052] The information collection module 201 can collect the tasks to be executed, unmanned device cluster information, available resources information of unmanned devices, and initial environment information of the tasks to be executed.

[0053] Among them, the unmanned device cluster information may include the device type, device model, device load, device power, device endurance, etc. of each unmanned device in the cluster. Taking the unmanned device cluster including a drone cluster, an unmanned boat cluster, and a driverless vehicle cluster as an example, the unmanned device cluster information may be: 20 drones, 10 driverless vehicles, and 8 unmanned boats. Among the 20 drones, there are 6 drones of model A, with a load of **KG and a power of **mAh, etc. The unmanned device cluster information can be uploaded by relevant personnel, or can be uploaded by the unmanned device itself after accessing the unmanned device cluster management system 200. This specification does not make special restrictions on this.

[0054] The tasks to be executed may include information required for the execution of the tasks themselves, such as task details, and may also include time requirements, quality requirements, task priorities, etc. Among them, the task details may include task type, task location, task volume, etc. The task type may include transportation tasks, search and rescue tasks, inspection tasks, etc. Taking the transportation task as an example, the task details may include the transportation starting point, transportation destination, type, specification, total weight of the goods to be transported, etc. The tasks to be executed can be uploaded by relevant personnel.

[0055] The available resources information of unmanned devices includes replacing the batteries of unmanned devices, charging piles for unmanned devices, storage areas for unmanned devices, etc. The storage areas for unmanned devices are, for example, airports for drones to take off and land, berths for unmanned boats to berth, and parking spaces for driverless vehicles.

[0056] The initial environmental information may be the current environmental information of the task location corresponding to the task to be executed, and may include topographical and geomorphic information, climate information (temperature, wind speed, precipitation), and may also include dynamic obstacle information (such as other unmanned devices), static obstacle information (such as buildings, mountains), etc. The initial environmental information may be uploaded by relevant personnel, or may be obtained by the information collection module 201 from the Internet or the Internet of Things based on the task location, etc.

[0057] The intelligent decision-making module 202 can divide the task to be executed based on the information collected by the information collection module 201, divide it into multiple subtasks, and determine the device information and available resources for each subtask. The device information may include the type of unmanned device required to execute the corresponding subtask, the unmanned device signal, the number of unmanned devices, etc. For example, the task to be executed is split into 3 subtasks. Subtask 1 requires 2 drones of model A, and the available resources for this subtask 1 are 1 unmanned device charging pile, a drone takeoff and landing airport, etc.

[0058] Among them, the intelligent decision-making module 202 can aim to maximize the task completion efficiency, maximize the utilization rate of available resources, minimize the energy consumption of the unmanned device cluster, and minimize the negative impact of the environment on the unmanned device cluster, split the task to be executed into multiple subtasks, and determine the device information required to execute each subtask and the available resources for the subtask. The specific determination method will be described in detail in the subsequent embodiments.

[0059] For each of the split subtasks, the intelligent planning module 203 can determine the task plan corresponding to the subtask according to the required device information and the available resources for the subtask. The task plan may include specific operable information such as the correspondence between the available resources for the subtask, the execution path, and the required devices. Among them, the distance between the available resources for the subtask located on the execution path corresponding to the required device and the execution path is within a preset distance (that is, the available resources for the subtask are located near the execution path, and the preset distance may be 1 kilometer, 2 kilometers, etc.). For example, algorithms such as graph search can be used for task planning, etc.

[0060] Still taking the aforementioned subtask 1 that requires 2 drones of model A as an example, the intelligent planning module 203 can respectively plan the execution paths for these 2 drones, and can also plan the flight speed, flight pose, etc. adopted for executing subtask 1. Among them, the execution paths corresponding to these 2 drones are usually different. For example, subtask 1 is divided into 2 execution paths, which are executed by different drones respectively.

[0061] The intelligent scheduling module 204 can obtain the device status of each unmanned device in the unmanned device cluster, and can schedule and adapt the target unmanned device according to the device status and the task plan to execute the subtask based on the corresponding execution path using the corresponding subtask available resources.

[0062] Among them, the device status may include the remaining power of the unmanned device, whether the unmanned device is currently available (for example, whether the unmanned device is currently executing other tasks, etc.), the device location, etc. The device status can be obtained by the intelligent scheduling module 204 after receiving the task plan determined by the intelligent planning module 203; the device status can also be actively reported by the unmanned device. For example, the unmanned device can periodically report its own device status to the system, and the intelligent scheduling module 204 can obtain the latest device status cached in the system.

[0063] Still taking the previous example that subtask 1 requires 2 drones of model A, if the intelligent scheduling module 204 obtains that all 6 drones of model A in the system are available, where drone A1 is closer to the first planned execution path and drone A2 is closer to the second planned execution path, then drone A1 can be called to execute subtask 1 according to the first execution path, and drone A2 can be called to execute subtask 1 according to the second execution path. Of course, in the case where the intelligent scheduling module 204 determines that the remaining power of drone A1 is not enough to support its task execution, drone A3 which is slightly farther away can also be called to execute subtask 1 according to the first execution path, so as to ensure that the task can be completed in one takeoff as much as possible without having to return for charging midway.

[0064] Or, allocate available resources such as a charging device for drone A1, and a path for round-trip charging to the available resources can be planned. This path also belongs to the first execution path corresponding to drone A1. Then drone A1 can execute subtask 1 according to the corresponding first execution path. During the execution process, it will reach the charging device for charging according to the execution path, and can continue to execute the corresponding subtask after charging is completed.

[0065] Or, allocate available resources such as a nearby airport for drone A1. If drone A1 encounters an emergency during the task execution, it can make a mid-air landing at the allocated nearby airport.

[0066] The feedback receiving module 205 can dynamically receive the execution data fed back by the target unmanned device during the execution of the subtask. The execution data may include: device information such as the current position, current speed, remaining power, etc., and may also include data such as images and videos captured by it.

[0067] The execution display module 206 can synchronously display the corresponding virtual execution instance in the virtual simulation scenario based on the execution data received by the feedback receiving module 205, thereby realizing real-time simulation.

[0068] As can be seen from the above description, in the unmanned device cluster management system provided in this specification, the intelligent decision-making module can be based on the to-be-executed task, unmanned device cluster information, available resources information of unmanned devices, and initial environment information collected by the information collection module, aiming to maximize the task completion efficiency, maximize the utilization rate of available resources, minimize the energy consumption of the unmanned device cluster, and minimize the negative impact of the environment on the unmanned device cluster. The to-be-executed task is split into multiple subtasks, and the device information and available resources for each subtask are determined. Furthermore, on the basis of optimizing multiple objectives simultaneously, the to-be-executed task is split, improving the quality of task splitting and reducing the difficulty of subsequent task planning. Then, the intelligent planning module can determine task plans such as the execution path and available resource allocation for each subtask, and the intelligent scheduling module can dispatch the appropriate target unmanned device according to the current device state of the unmanned device to execute the subtask by using the corresponding available resources of the subtask along the execution path, thus realizing the flexible scheduling of various unmanned devices and also realizing efficient real-time collaborative operations. At the same time, the execution display module in the unmanned device cluster management system can also synchronously display the corresponding virtual execution instances in the virtual simulation scenario based on the execution data of the target unmanned device dynamically received by the feedback receiving module, facilitating relevant personnel to monitor the execution of tasks in real time.

[0069] In another embodiment of this specification, please refer to Figure 3 , after the feedback receiving module 205 receives the execution data fed back by the target unmanned device, it can also determine the operating state of the target unmanned device according to the execution data, and the operating state can include normal and abnormal.

[0070] Specifically, the feedback receiving module 205 can judge the operating state of the target unmanned device according to the current speed and remaining power in the execution data. For example, if the remaining power of the target unmanned device is 0 or less than the threshold, it is determined that the target unmanned device is operating abnormally. For another example, if the current speed of the target unmanned device is 0, that is, the target unmanned device has stopped moving forward, and it is determined that there is an obstacle in front of it by combining the video or image it captured, it means that the target unmanned device may have collided with the obstacle and an accident has occurred, and its operating state can be determined to be abnormal.

[0071] In this embodiment, when the feedback receiving module 205 determines that the operating state of the target unmanned device is abnormal, the intelligent scheduling module 204 can dispatch available unmanned devices to take over the abnormal target unmanned device and continue to execute the corresponding subtask.

[0072] Specifically, the intelligent scheduling module 204 can first identify available unmanned devices in the unmanned device cluster, for example, identifying unmanned devices that are not currently performing tasks as available unmanned devices. Then, the subtasks that have not been completed by the abnormal target unmanned device can be scheduled to the available unmanned device. For example, the execution path that has not been executed in the abnormal target unmanned device is sent to the available unmanned device, so that the available unmanned device continues to perform the subtask based on the corresponding execution path. Of course, in actual applications, based on the task type, it may be necessary to remind other staff members to assist the available unmanned device in taking over the subtask. For example, if the task to be performed is a photography task, then no other staff members are required to assist. After arriving at the corresponding location, the available unmanned device can continue to perform the photography task according to the remaining execution path. For another example, if the task to be performed is a transportation task, the intelligent scheduling module 204 will also send a reminder message to the relevant personnel to remind them to come to the scene to assist in the transportation of the transported object, etc. This specification does not impose any special restrictions on this.

[0073] It can be seen from this that the unmanned equipment cluster management system provided in this manual can also determine the operating status of the target unmanned equipment based on the execution data feedback during the execution of the task by the target unmanned equipment, and when it is determined that the operating status of the target unmanned equipment is abnormal, it can dispatch available unmanned equipment to replace the target unmanned equipment to continue to execute the corresponding subtasks, ensuring that the tasks to be executed are completed on schedule and efficiently, and avoiding situations where the tasks cannot be completed on time due to abnormal operation of the unmanned equipment.

[0074] In another embodiment of this specification, please refer to Figure 3 The unmanned equipment cluster management system can also optimize the execution of tasks based on the execution data fed back by the target unmanned equipment to adapt to changes in environmental factors and ensure the efficient and high-quality completion of the tasks to be executed.

[0075] In this embodiment, the execution data fed back by the target unmanned device may also include the current environmental information detected by the device. This current environmental information may include the actual terrain, current wind speed, current temperature, and other information collected by the device. The feedback receiving module 205 may also compare the current environment with the initial environmental information collected by the information collection module 201 to determine whether the task environment for the task to be executed has significantly changed. The definition of a significant environmental change can be pre-set based on the degree of impact of the environmental change on the unmanned device, and this specification does not impose any specific restrictions on this.

[0076] Specifically, the initial environmental information may be obtained from the Internet or the Internet of Things, or uploaded by relevant personnel, and there may be differences from the actual environment. For example, the information collection module 201 obtains the weather forecast of the task location from the Internet or the Internet of Things as sunny, but during the execution of the subtask by the target unmanned device, the weather suddenly changes, strong winds blow, and a severe thunderstorm is about to occur. After the target unmanned device feeds back the currently collected weather to the unmanned device cluster management system, the feedback receiving module 205 can determine that a huge change has occurred in the task environment. For another example, relevant personnel know that the task location is a mountain area and then upload the initial environmental information of the mountain area. However, a tarmac road is later paved in this mountain area. After the target unmanned device feeds back the collected video or image to the unmanned device cluster management system, the feedback receiving module 205 can also determine that a huge change has occurred in the task environment.

[0077] In this embodiment, after the feedback receiving module 205 determines that a huge change has occurred in the task environment, the intelligent decision-making module 202 can determine the remaining tasks that have not been completed in the to-be-executed tasks based on the execution data fed back by each target unmanned device. For example, objects that have not been transported to the destination, areas that have not been searched thoroughly, etc. Then, the remaining tasks can be re-split based on the remaining tasks, the unmanned device cluster information, and the current environmental information to split the remaining tasks into multiple sub-remaining tasks, and the device information corresponding to each sub-remaining task can be determined. Specifically, the sub-remaining tasks can still be split with the goal of maximizing the task completion efficiency, minimizing the energy consumption of the unmanned device cluster, and minimizing the negative impact of the environment on the unmanned device cluster.

[0078] In this embodiment, after the intelligent decision-making module 202 completes the re-splitting of the remaining tasks, the intelligent planning module 203 can re-plan the tasks based on the splitting results. Then, the intelligent scheduling module 204 can schedule the appropriate unmanned devices to execute the remaining tasks based on the re-planned execution path.

[0079] For example, assume that the target unmanned device is a drone and the to-be-executed task is an inspection task. During the execution of the task, the weather suddenly changes, strong winds blow, and a severe thunderstorm is about to occur. After the drone feeds back the currently collected weather to the unmanned device cluster management system, the feedback receiving module 205 can determine that a huge change has occurred in the task environment. The intelligent decision-making module 202 re-divides the tasks, divides the remaining tasks (areas that have not been inspected) into multiple sub-remaining tasks, and determines that the device information corresponding to these sub-remaining tasks is an unmanned vehicle. The intelligent scheduling module 204 schedules the appropriate unmanned vehicle to replace the drone to continue executing the inspection task to avoid safety risks such as the drone falling due to extreme weather.

[0080] It can be seen that the unmanned device cluster management system provided in this specification can also determine whether there are significant changes in the task environment based on the execution data fed back during the task execution of the target unmanned device. When it is determined that there are significant changes in the task environment, the task can be repartitioned, and the device information corresponding to the sub-remaining tasks after repartitioning can be re-determined. Furthermore, the appropriate unmanned devices can be scheduled in a timely manner according to the changes in the environmental information to continue the task execution, which improves the task completion efficiency, reduces the loss of unmanned devices, and simultaneously improves the robustness and flexibility of the system.

[0081] In another embodiment of this specification, the unmanned device cluster management system can also maintain a corresponding expert knowledge base. When making an intelligent decision, the intelligent decision-making module 202 can obtain the expert knowledge related to the task to be executed, the unmanned device cluster, and the initial environment from the expert knowledge base, and then split the task to be executed and determine the device information corresponding to the subtasks based on the guidance of the expert knowledge.

[0082] Among them, the expert knowledge base can be a knowledge graph, which can integrate the knowledge of multiple fields, including the performance of unmanned devices, environmental information (such as climate, terrain, etc.), task characteristics, industry standards, etc. Specifically, semantic modeling can be performed on the information in these fields to extract the knowledge and experience related to the implementation of unmanned device tasks, and a knowledge graph including entities such as unmanned devices, environments, and tasks can be constructed.

[0083] For example, the knowledge graph can include an unmanned device entity, and the attributes of the unmanned device entity can include the task types it is suitable for, load capacity, speed limit, power consumption, and the relationship between the change of wind speed, etc.

[0084] In this embodiment, in the initial state, the expert knowledge base can be set based on the parameter indicators of the unmanned device and some industry experience. Subsequently, the unmanned device cluster management system can update the expert knowledge base based on the actual situation during the task execution to retain the latest knowledge.

[0085] Specifically, please refer to Figure 3 , the unmanned device cluster management system can also include a knowledge maintenance module 207, and the knowledge maintenance module 207 can update the expert knowledge in the expert knowledge base according to the execution data fed back by the target unmanned device.

[0086] For example, in the initial state, the relationship between the power consumption of the UAV of model A recorded in the expert knowledge base and the wind speed (such as when the wind speed is greater than a certain value, the power consumption increases by a certain amount) is no longer applicable as the battery performance degrades during the use of the UAV. The initial relationship between the power consumption and the wind speed cannot provide an accurate basis for change for the intelligent decision-making module. Therefore, during the process of the UAV performing tasks, the knowledge maintenance module 207 can re-determine the relationship between its power consumption and the wind speed based on the execution data fed back by the UAV, and then update the corresponding expert knowledge in the expert knowledge base to provide an accurate reference basis for the intelligent decision-making module.

[0087] Similarly, in addition to the intelligent decision-making module, when performing task planning, the intelligent planning module 203 can also refer to the relevant expert knowledge in the knowledge base, and this specification does not impose special restrictions on this.

[0088] It can be seen from this that the unmanned device cluster management system in this specification can also maintain the expert knowledge base, and thus can provide expert experience for task splitting and task planning of the system, improving the accuracy of task splitting and task planning. At the same time, the knowledge maintenance module can also update the expert knowledge base in a timely manner according to the execution data fed back by the unmanned device during the task execution process. Compared with the traditional static expert library, it can achieve dynamic intelligent reasoning, and thus can ensure that the system maintains self-adaptability and high efficiency when facing new tasks and new environments.

[0089] In another embodiment of this specification, after determining the task planning of each subtask, the unmanned device cluster management system can also send it to local or remote unmanned devices for physical simulation operation, or send it to virtual unmanned devices for virtual simulation. Specifically, digital twin technology can be used to realize the task implementation and verification of unmanned devices in virtual simulation scenarios and field scenarios, and the aforementioned task division and task planning can also be optimized according to the verification results.

[0090] In one example, before scheduling the unmanned device to execute the corresponding task, the intelligent scheduling module 204 can schedule the virtual unmanned device to perform virtual simulation in the virtual simulation scenario to verify and optimize the task specification, and after the virtual simulation is completed, schedule local or remote unmanned devices to execute the corresponding task.

[0091] In another example, it is also possible to schedule local or remote unmanned devices to execute the corresponding task in the field scenario on the basis of scheduling the virtual unmanned device to perform virtual simulation in the virtual simulation scenario. Please refer to Figure 4 , the virtual simulation scenario and the field scenario can be mapped and synchronized with each other, and can also communicate and interact with each other.

[0092] For example, in a field scenario, an unmanned device can send its execution data to a virtual simulation scenario, and the virtual simulation scenario can then update the virtual instances it displays based on the execution data. For example, after an unmanned device collects meteorological information such as the wind speed in the field and synchronizes it to the virtual simulation scenario, the corresponding meteorological information can be simulated in the virtual simulation scenario. For another example, after the task planning is optimized in the virtual simulation scenario, the optimized task planning (such as the optimized execution path, etc.) can be sent to the unmanned device in the field scenario, so that the unmanned device can execute the corresponding task according to the optimized task planning. Among them, the field scenario includes a local field scenario and a remote field scenario.

[0093] It can be seen that by using the unmanned device cluster management system provided in this specification, the virtual simulation scenario can interact with the field simulation scenario, and on the basis of dynamically monitoring the task, it can also dynamically adjust and control the unmanned devices in the field scenario.

[0094] The above virtual simulation scenario can be implemented through the aforementioned execution display module of the unmanned device cluster management system.

[0095] In another embodiment of this specification, please refer to Figure 5 , the intelligent decision-making module in the unmanned device cluster management system can split the to-be-executed task and determine the device information corresponding to the subtasks in the following manner:

[0096] Step 502, obtain an optimization function, where the optimization function is a weighted function of multiple sub-functions, and each sub-function corresponds to one or more subtasks, including an efficiency term corresponding to the task completion efficiency, an energy consumption term corresponding to the cluster energy consumption, an environmental impact term corresponding to the negative impact, and a resource utilization term corresponding to the utilization rate of the available resources.

[0097] In this embodiment, the optimization function can be pre-constructed. Since the intelligent decision-making module can split the to-be-executed task into multiple subtasks, the optimization function can be constructed as a weighted function of multiple sub-functions. Among them, each sub-function can correspond to one or more split subtasks, and the weights of each sub-function in the weighted function can also be pre-set.

[0098] In this embodiment, each sub-function can include an efficiency term representing the external task completion efficiency, an energy consumption term corresponding to the cluster energy consumption, an environmental impact term corresponding to the negative impact on the environment, and a resource utilization term corresponding to the utilization rate of the available resources, that is, the sub-function includes terms corresponding to the optimization objectives.

[0099] In this embodiment, the following optimization function can be constructed:

[0100]

[0101] Among them, is the optimization function, w i is the weight of the i-th sub-task, f i is the sub-function corresponding to the i-th sub-task, and each sub-function f i can include the aforementioned efficiency term, energy consumption term, and environmental impact term. The value of i ranges from 1 to N, where N is the number of sub-functions to be weighted.

[0102] In the optimization function, T can represent the time model, which represents the time variables and functions of the operation of the unmanned device cluster and can be used to construct the efficiency term, etc.;

[0103] In the optimization function, E can represent the environmental model, which represents the negative impacts brought by parameters such as terrain, wind speed, and climate to the unmanned device cluster and can be used to construct the environmental impact term, etc.;

[0104] In the optimization function, J can represent the task model, which represents task priorities, quantities, etc. and can be used to construct the efficiency term, energy consumption term, etc.;

[0105] In the optimization function, D can represent the unmanned device entity model, which represents device performance, specifications, parameters, quantity, unmanned device entities, etc. and can be used to construct the efficiency term, energy consumption term, and environmental impact term, etc.;

[0106] In the optimization function, S can represent the unmanned device simulation model, which is similar to D and can also represent device performance, specifications, parameters, quantity, virtual simulation bodies of unmanned devices, etc. and can be used to construct the efficiency term, energy consumption term, and environmental impact term, etc.;

[0107] In the optimization function, C can represent the control model, which represents the states, behaviors, interactions, etc. of unmanned devices and can be used to construct the efficiency term, energy consumption term, and environmental impact term, etc.;

[0108] In the optimization function, K can represent the knowledge base model, which represents the decisions and logics of cluster operations and can be used to construct the efficiency term, energy consumption term, and environmental impact term, etc.

[0109] In the optimization function, R can represent the available resource model of unmanned devices.

[0110] Of course, in other examples, other optimization terms can be set according to actual needs, and this specification does not make special restrictions on this.

[0111] Step 504: Taking the minimization of the optimization function as the goal, with the constraint conditions that the negative impact of the initial environmental information on the device cluster is less than the environmental impact threshold, the execution duration of the task to be executed is less than the duration threshold, the device information conforms to the device restrictions corresponding to the unmanned device cluster, and the available resources used by the task do not exceed the available resource restrictions, solve the optimization function to obtain the sub-tasks and the device information and sub-task available resources required for each sub-task.

[0112] In this embodiment, the negative impacts of the initial environmental information on the device cluster may include security impacts, energy consumption impacts, device wear impacts, etc. The device information conforming to the device limitation conditions corresponding to the unmanned device cluster may be the limitation conditions of the device itself (such as the endurance not exceeding ** kilometers, etc.), the industry limitation conditions (such as the maximum driving speed limited in a certain area, etc.), or the cluster limitation conditions (such as the number of drones included in the cluster), etc. The available resources used by the task not exceeding the available resource limitation may be the limitation conditions of the available resources themselves. For example, there are 3 charging piles for unmanned devices, and at most 3 unmanned vehicles can be charged, etc.

[0113] In this embodiment, the optimization function may be solved based on these constraint conditions. For example, the Lagrange multiplier method may be used to solve the optimization function, or the penalty function method may be used to solve the optimization function, etc. Through the solution, the subtasks after splitting the task to be executed and the device information corresponding to each subtask can be obtained. Specifically, the following formula can be referred to:

[0114]

[0115] subject to g(E)≤G,

[0116] h(D)≤H,

[0117] l(T)≤L,

[0118] r≤R

[0119] where, represents minimizing the optimization function, g(E)≤G represents that the negative impact of the initial environmental information on the device cluster is less than the environmental impact threshold, h(D)≤H represents that the device information conforming to the device limitation conditions corresponding to the unmanned device cluster is a constraint condition, l(T)≤L represents that the execution duration of the task to be executed is less than the duration threshold, and r≤R represents that the available resources do not exceed the available resource limitation.

[0120] Of course, in other examples, according to actual needs, other constraint conditions may also be set, and this specification does not make special restrictions on this.

[0121] It can be seen that this specification splits the task to be executed into multiple subtasks by solving the multi-objective optimization function, obtains the device information and the available resources of each subtask, can achieve a trade-off among multiple objectives, realize the splitting of complex tasks, and improve the efficiency and flexibility of subsequent task execution.

[0122] In another embodiment of this specification, a method for managing an unmanned device cluster is further provided. This method can be applied to an unmanned device cluster management system. Please refer to Figure 6 and this method may include the following steps:

[0123] Step 602, collect the to-be-executed task, the information of the unmanned device cluster, the available resources of the unmanned devices, and the initial environment information of the to-be-executed task.

[0124] Step 604, according to the to-be-executed task, the information of the unmanned device cluster, the available resources of the unmanned devices, and the initial environment information, with the goals of maximizing the task completion efficiency, maximizing the utilization rate of available resources, minimizing the energy consumption of the unmanned device cluster, and minimizing the negative impact of the environment on the unmanned device cluster, split the to-be-executed task into multiple subtasks, and determine the device information required for executing each subtask and the available resources for the subtasks.

[0125] Step 606, for each subtask, determine the task plan corresponding to the subtask according to the required device information and the available resources for the subtask. The task plan includes the correspondence between the available resources for the subtask, the execution path, and the required devices, where the available resources for the subtask are located on the execution path corresponding to the required device or the distance between the available resources for the subtask and the execution path is within a preset distance.

[0126] Step 608, obtain the device status of the unmanned devices in the unmanned device cluster, and according to the device status and the task plan, schedule the adapted target unmanned devices to execute the subtask based on the corresponding execution path and using the corresponding available resources for the subtask.

[0127] Step 610, dynamically receive the execution data fed back by the target unmanned devices during the execution of the subtasks.

[0128] Step 612, synchronously display the corresponding virtual execution instances in the virtual simulation scenario based on the execution data.

[0129] In this embodiment, the specific implementation of the above process can refer to the foregoing embodiments, and will not be elaborated herein one by one.

[0130] Exemplarily, the to-be-executed task can be split and the device information and the available resources for the subtasks can be determined in the following manner:

[0131] Obtain an optimization function, which is a weighted function of multiple sub-functions. Each sub-function corresponds to one or more subtasks and includes an efficiency term corresponding to the task completion efficiency, an energy consumption term corresponding to the cluster energy consumption, an environmental impact term corresponding to the negative impact, and a resource utilization term corresponding to the available resource utilization rate;

[0132] With the goal of minimizing the optimization function, and with the constraints that the negative impact of the initial environmental information on the device cluster is less than the environmental impact threshold, the execution duration of the task to be executed is less than the duration threshold, the device information conforms to the device restrictions corresponding to the unmanned device cluster, and the available resources used by the task do not exceed the available resource limit, solve the optimization function to obtain the subtasks and the device information and subtask available resources required for each subtask.

[0133] Exemplarily, the management method of the unmanned device cluster may further include: determining the operating state of the corresponding target unmanned device based on the execution data; in the case where the operating state of the target unmanned device is abnormal, determining available unmanned devices, and scheduling the subtasks not completed by the abnormal target unmanned device to the available unmanned devices, so that the available unmanned devices continue to execute the subtasks based on the corresponding execution paths.

[0134] Exemplarily, the execution data further includes the current environmental information detected by the target unmanned device, and the management method of the unmanned device cluster may further include: determining whether there is a huge change in the task environment of the task to be executed based on the current environmental information and the initial environmental information; in the case where the task environment has a huge change, determining the remaining tasks not executed in the task to be executed, and re-splitting the remaining tasks based on the remaining tasks, the unmanned device cluster information, and the current environmental information, and determining the device information required for each sub-remaining task after re-splitting.

[0135] Exemplarily, the management method of the unmanned device cluster may also obtain expert knowledge related to the task to be executed, the unmanned device cluster, and the initial environment from an expert knowledge base; the process of the intelligent decision module splitting the task to be executed and determining the device information includes: splitting the task to be executed and determining the device information and subtask available resources with the goals of maximizing task completion efficiency, maximizing available resource utilization rate, minimizing the energy consumption of the unmanned device cluster, and minimizing the negative impact of the environment on the unmanned device cluster according to the expert knowledge.

[0136] Exemplarily, the management method of the unmanned device cluster may further include: updating the expert knowledge in the expert knowledge base according to the execution data.

[0137] Exemplarily, the expert knowledge is derived from an unmanned device task implementation knowledge graph, and the construction of the knowledge graph is derived from knowledge, experience, industry standards, environmental information (terrain, obstacles, climate, etc.), and specification parameters of unmanned devices related to unmanned devices, including unmanned device entities, task entities, and environmental entities.

[0138] For the specific implementation process of the above method, reference can also be made to the foregoing embodiments of the unmanned device cluster management system, which will not be elaborated in this specification.

[0139] Figure 7 It is a schematic structural diagram of a device provided by an exemplary embodiment. The physical carrier of the foregoing unmanned device cluster management system in this specification may be the Figure 7 device shown. Please refer to Figure 7 . At the hardware level, the device includes a processor 702, an internal bus 704, a network interface 706, a memory 708, and a non-volatile memory 710. Of course, there may also be other hardware required for other functions. One or more embodiments of this specification can be implemented in a software manner. For example, the processor 702 reads the corresponding computer program from the non-volatile memory 710 into the memory 708 and then runs it. Of course, in addition to the software implementation manner, one or more embodiments of this specification do not exclude other implementation manners, such as logical devices or a combination of software and hardware. That is to say, the execution subject of the following processing flow is not limited to each logical unit, and can also be hardware or a logical device.

[0140] Based on the same concept as the above method, this specification also provides an electronic device, including: a processor; a memory for storing executable instructions of the processor; wherein, the processor realizes the steps of the method as described in any of the foregoing embodiments by running the executable instructions.

[0141] Based on the same concept as the above method, this specification also provides a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method as described in any of the foregoing embodiments are realized.

[0142] Based on the same concept as the above method, this specification also provides a computer program product, including a computer program / instructions, and when the computer program / instructions are executed by a processor, the steps of the method as described in any of the foregoing embodiments are realized.

Claims

1. An unmanned device cluster management system, characterized in that, The system includes: An information collection module, configured to collect the to-be-executed task, unmanned equipment cluster information, available resources information of the unmanned equipment, and initial environment information of the to-be-executed task; An intelligent decision-making module, configured to split the to-be-executed task into multiple subtasks, and determine the equipment information required for executing each subtask and the available resources of the subtask, with the goal of maximizing the task completion efficiency, maximizing the utilization rate of available resources, minimizing the energy consumption of the unmanned equipment cluster, and minimizing the negative impact of the environment on the unmanned equipment cluster, based on the to-be-executed task, the unmanned equipment cluster information, the available resources information of the unmanned equipment, and the initial environment information; An intelligent planning module, configured to, for each subtask, determine the task plan corresponding to the subtask according to the required equipment information and the available resources of the subtask, where the task plan includes the correspondence between the available resources of the subtask, the execution path, and the required equipment, and wherein the available resources of the subtask are located on the execution path corresponding to the required equipment or the distance between the available resources of the subtask and the execution path is within a preset distance; An intelligent scheduling module, configured to obtain the equipment status of the unmanned equipment in the unmanned equipment cluster, and schedule the appropriate target unmanned equipment based on the equipment status and the task plan to execute the subtask by using the corresponding available resources of the subtask along the corresponding execution path; A feedback receiving module, configured to dynamically receive the execution data fed back by the target unmanned equipment during the execution of the subtask; An execution display module, configured to synchronously display the corresponding virtual execution instance in the virtual simulation scenario based on the execution data.

2. The system according to claim 1, wherein The intelligent decision-making module splits the to-be-executed task and determines the equipment information and the available resources of the subtask in the following manner: Obtain an optimization function, where the optimization function is a weighted function of multiple sub-functions, and each sub-function corresponds to one or more subtasks, including an efficiency term corresponding to the task completion efficiency, an energy consumption term corresponding to the cluster energy consumption, an environmental impact term corresponding to the negative impact, and a resource utilization rate term corresponding to the available resource utilization rate; With the goal of minimizing the optimization function, and with the constraints that the negative impact of the initial environment information on the equipment cluster is less than an environmental impact threshold, the execution duration of the to-be-executed task is less than a duration threshold, the equipment information conforms to the equipment restrictions corresponding to the unmanned equipment cluster, and the available resources used by the task do not exceed the available resource limit, solve the optimization function to obtain the subtasks and the equipment information and the available resources of each subtask.

3. The system according to claim 1, wherein The feedback receiving module is further configured to determine the operating status of the corresponding target unmanned equipment based on the execution data; The intelligent scheduling module is further configured to, when the operating status of the target unmanned equipment is abnormal, determine the available unmanned equipment, and schedule the subtasks not completed by the abnormal target unmanned equipment to the available unmanned equipment, so that the available unmanned equipment continues to execute the subtask along the corresponding execution path.

4. The system according to claim 1, wherein The execution data further includes the current environment information detected by the target unmanned equipment. The feedback receiving module is further configured to determine whether there is a significant change in the task environment of the to-be-executed task based on the current environment information and the initial environment information; The intelligent decision-making module is further configured to, when there is a significant change in the task environment, determine the remaining tasks in the to-be-executed task that have not been executed, and re-split the remaining tasks based on the remaining tasks, the unmanned device cluster information, and the current environment information, and determine the device information required for each sub-remaining task after re-splitting.

5. The system according to claim 1, wherein The intelligent decision-making module is further configured to obtain expert knowledge related to the to-be-executed task, the unmanned device cluster, and the initial environment from the expert knowledge base; The process of the intelligent decision-making module splitting the to-be-executed task and determining the device information includes: According to the expert knowledge, splitting the to-be-executed task and determining the device information and the available resources of the sub-tasks with the goals of maximizing the task completion efficiency, maximizing the utilization rate of available resources, minimizing the energy consumption of the unmanned device cluster, and minimizing the negative impact of the environment on the unmanned device cluster.

6. The system according to claim 5, characterized in that The system further includes: A knowledge maintenance module for updating the expert knowledge in the expert knowledge base according to the execution data.

7. A method for managing a cluster of unmanned devices, characterized in that, The method includes: Collecting the to-be-executed task, the unmanned device cluster information, the available resources information of the unmanned device, and the initial environment information of the to-be-executed task; According to the to-be-executed task, the unmanned device cluster information, the available resources information of the unmanned device, and the initial environment information, with the goals of maximizing the task completion efficiency, maximizing the utilization rate of available resources, minimizing the energy consumption of the unmanned device cluster, and minimizing the negative impact of the environment on the unmanned device cluster, splitting the to-be-executed task into multiple sub-tasks, and determining the device information required for executing each sub-task and the available resources of the sub-task; For each sub-task, determining the task plan corresponding to the sub-task according to the required device information and the available resources of the sub-task, where the task plan includes the correspondence between the available resources of the sub-task, the execution path, and the required devices, and the available resources of the sub-task are located on the execution path corresponding to the required device or the distance between the available resources of the sub-task and the execution path is within a preset distance; Obtaining the device status of the unmanned devices in the unmanned device cluster, and scheduling the adapted target unmanned devices according to the device status and the task plan to execute the sub-task by using the corresponding available resources of the sub-task based on the corresponding execution path; Dynamically receiving the execution data fed back by the target unmanned device during the execution of the sub-task; Synchronously displaying the corresponding virtual execution instance in the virtual simulation scenario based on the execution data.

8. An electronic device, characterized in that, Includes: A processor; A memory for storing processor-executable instructions; wherein, the processor realizes the steps of the method as claimed in claim 7 by running the executable instructions.

9. A computer-readable storage medium, characterized in that, A computer instruction is stored thereon, and when the instruction is executed by the processor, the steps of the method as claimed in claim 7 are realized.

10. A computer program product, characterized in that, Includes a computer program / instruction, and when the computer program / instruction is executed by the processor, the steps of the method as claimed in claim 7 are realized.

Images