Unmanned device swarm management system, method, device, medium, and program product

Abstract

One or more embodiments of the present specification provide an unmanned device cluster management system, method, device, medium and program product. The system comprises: an information collection module for collecting a to-be-executed task, unmanned device cluster information, unmanned device available resource information and initial environment information; an intelligent decision module for splitting the to-be-executed task into a plurality of subtasks according to the information collected by the information collection module, determining device information required for executing each subtask and available resources of the subtask; an intelligent planning module for determining a task plan corresponding to the subtask, the task plan comprising resource allocation, an execution path and required devices; an intelligent scheduling module for scheduling an adapted target unmanned device to execute the subtask according to a device state and the task plan; a feedback receiving module for dynamically receiving execution data fed back by the target unmanned device; and an execution display module for synchronously displaying a corresponding virtual execution instance in a virtual simulation scene based on the execution data.

Description

Technical Field

[0001] This specification relates to the field of unmanned equipment technology, and more particularly to an unmanned equipment cluster management system, method, device, medium, and program product. Background Technology

[0002] With the rapid development of technology, unmanned equipment such as drones, unmanned vehicles, and unmanned ships have been widely used in various fields. These unmanned devices each have their own characteristics and are suitable for different scenarios: drones, with their flexible flight capabilities and rapid response speed, are suitable for tasks such as aerial reconnaissance, logistics delivery, and emergency rescue; unmanned vehicles, with their strong ground mobility and load-bearing capacity, perform excellently in logistics transportation, security patrols, and exploration of complex terrain; and unmanned ships play an important role in areas such as water monitoring, marine mapping, and water rescue.

[0003] However, in many complex tasks, a single type of unmanned equipment (UAV) is often insufficient to meet the requirements, necessitating the collaborative operation of multiple UAVs. But significant differences exist among different UAVs in performance, specifications, and equipment limitations, posing a considerable challenge to collaborative work. For example, adverse weather conditions such as strong winds, heavy rain, and dense fog can severely impact the flight stability and control capabilities of UAVs, while complex terrain, such as mountains and hills, can increase the stability of UAV operations. Therefore, how to achieve real-time, efficient collaborative operation among UAVs and how to manage UAV clusters in real-time have become pressing technical problems that need to be solved in the field of unmanned systems. Summary of the Invention

[0004] In view of the above, one or more embodiments of this specification provide the following technical solutions:

[0005] According to a first aspect of one or more embodiments of this specification, an unmanned equipment cluster management system is proposed, comprising:

[0006] The information acquisition module is used to collect information on the task to be executed, the unmanned equipment cluster, the available resources of the unmanned equipment, and the initial environmental information of the task to be executed.

[0007] The intelligent decision-making module is used to divide the task to be executed into multiple sub-tasks based on the task to be executed, the information of the unmanned equipment cluster, the information of available resources of the unmanned equipment, and the information of the initial environment, with the goal of maximizing task completion efficiency, maximizing the utilization rate of available resources, minimizing the energy consumption of the unmanned equipment cluster, and minimizing the negative impact of the environment on the unmanned equipment cluster, and to determine the equipment information and available resources of each sub-task to execute.

[0008] The intelligent planning module is used to determine the task plan corresponding to each subtask based on its required device information and the available resources of the subtask. The task plan includes the correspondence between the available resources of the subtask, the execution path and the required device, wherein the available resources of the subtask are located on the execution path corresponding to the required device or the distance between the subtask and the execution path is within a preset distance.

[0009] The intelligent scheduling module is used to obtain the device status of unmanned devices in the unmanned device cluster, and according to the device status and the task planning, the target unmanned device is scheduled to execute the sub-task based on the corresponding execution path and the available resources of the corresponding sub-task.

[0010] The feedback receiving module is used to dynamically receive the execution data fed back by the target unmanned device during the execution of the sub-task;

[0011] The execution display module is used to synchronously display the corresponding virtual execution instance in the virtual simulation scene based on the execution data.

[0012] Optionally, the intelligent decision-making module splits the task to be executed and determines the device information and available resources for the sub-task in the following manner:

[0013] Obtain an optimization function, which is a weighted function of multiple sub-functions, wherein each sub-function corresponds to one or more sub-tasks, including an efficiency item corresponding to the task completion efficiency, an energy consumption item corresponding to the cluster energy consumption, an environmental impact item corresponding to the negative impact, and a resource utilization item corresponding to the available resource utilization rate.

[0014] With the objective of minimizing the optimization function, and constrained by the following conditions: the negative impact of the initial environmental information on the device cluster is less than the environmental impact threshold, the execution time of the task to be executed is less than the time threshold, the device information meets the device restrictions corresponding to the unmanned device cluster, and the available resources used by the task do not exceed the available resource restrictions, the optimization function is solved to obtain the sub-task and the device information and available resources required for each sub-task.

[0015] Optionally, the feedback receiving module is further configured to determine the operating status of the corresponding target unmanned device based on the execution data;

[0016] The intelligent scheduling module is also used to determine available unmanned equipment when the operating state of the target unmanned equipment is abnormal, and to schedule the sub-tasks that the abnormal target unmanned equipment has not completed to the available unmanned equipment, so that the available unmanned equipment can continue to execute the sub-tasks based on the corresponding execution path.

[0017] Optionally, the execution data may also include current environmental information detected by the target unmanned device;

[0018] The feedback receiving module is also used to determine whether the task environment of the task to be executed has changed significantly based on the current environment information and the initial environment information;

[0019] The intelligent decision-making module is also used to determine the remaining unexecuted tasks in the tasks to be executed when the task environment undergoes a significant change, and to re-divide the remaining tasks based on the remaining tasks, the unmanned equipment cluster information and the current environment information, and to determine the equipment information required for each sub-remaining task after re-dividement.

[0020] Optionally, the intelligent decision-making module is further configured to acquire expert knowledge related to the task to be executed, the unmanned equipment cluster, and the initial environment from an expert knowledge base;

[0021] The process by which the intelligent decision-making module breaks down the task to be executed and determines the device information includes:

[0022] Based on the expert knowledge, the tasks to be executed are broken down with the objectives of maximizing task completion efficiency, maximizing the utilization rate of available resources, minimizing the energy consumption of the unmanned equipment cluster, and minimizing the negative impact of the environment on the unmanned equipment cluster, and the equipment information and available resources of the sub-tasks are determined.

[0023] Optional, also includes:

[0024] The knowledge maintenance module is used to update the expert knowledge in the expert knowledge base based on the execution data.

[0025] Optionally, the expert knowledge is derived from the unmanned equipment mission implementation knowledge graph. The knowledge graph is constructed from one or more of the following: unmanned equipment-related knowledge, experience, industry standards, environmental information (terrain, obstacles, climate, etc.), and unmanned equipment specifications, including unmanned equipment entities, mission entities, and environmental entities.

[0026] According to a second aspect of one or more embodiments of this specification, a method for managing an unmanned equipment cluster is proposed, the method comprising:

[0027] Collect information on tasks to be executed, unmanned equipment cluster information, available resources of unmanned equipment, and the initial environment information of the tasks to be executed;

[0028] Based on the task to be executed, the unmanned equipment cluster information, the unmanned equipment available resource information, and the initial environment information, with the goals of maximizing task completion efficiency, maximizing available resource utilization, minimizing unmanned equipment cluster energy consumption, and minimizing the negative impact of the environment on the unmanned equipment cluster, the task to be executed is divided into multiple sub-tasks, and the equipment information and available resources of each sub-task are determined.

[0029] For each subtask, a task plan is determined based on the required device information and the available resources of the subtask. The task plan includes the correspondence between the available resources of the subtask, the execution path, and the required device. The available resources of the subtask are located on the execution path corresponding to the required device or the distance between the available resources and the execution path is within a preset distance.

[0030] Obtain the device status of unmanned devices in the unmanned device cluster, and according to the device status and the target unmanned device adapted by the task planning and scheduling, execute the sub-task using the available resources of the corresponding sub-task based on the corresponding execution path;

[0031] Dynamically receive execution data fed back by the target unmanned device during the execution of the sub-task;

[0032] Based on the execution data, the corresponding virtual execution instance is synchronously displayed in the virtual simulation scene.

[0033] According to a third aspect of one or more embodiments of this specification, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described above by executing the executable instructions.

[0034] According to a fourth aspect of one or more embodiments of this specification, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method described above.

[0035] According to a fifth aspect of one or more embodiments of this specification, a computer program product is provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the method as described above.

[0036] As described above, the unmanned equipment cluster management system provided in this specification allows the intelligent decision-making module to collect information on the tasks to be executed, the unmanned equipment cluster, available resources of the unmanned equipment, and the initial environment from the information acquisition module. With the goals of maximizing task completion efficiency, maximizing available resource utilization, minimizing energy consumption of the unmanned equipment cluster, and minimizing the negative impact of the environment on the unmanned equipment cluster, the system breaks down the tasks to be executed into multiple sub-tasks. It also determines the equipment information and available resources required to execute each sub-task, thereby optimizing multiple objectives simultaneously, improving the quality of task breakdown, and reducing the difficulty of subsequent task planning. Next, the intelligent planning module determines the execution path and available resource allocation for each sub-task. The intelligent scheduling module then schedules suitable target unmanned equipment according to the current equipment status of the unmanned equipment, utilizing the corresponding available resources of the sub-task to execute the sub-task according to the execution path. This achieves flexible scheduling of various unmanned equipment and efficient real-time collaborative operation. Simultaneously, the execution display module in the unmanned equipment cluster management system can also synchronously display corresponding virtual execution instances in a virtual simulation scene based on the execution data of the target unmanned equipment dynamically received by the feedback receiving module, facilitating real-time monitoring of task execution by relevant personnel. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the architecture of an unmanned equipment cluster management system provided in an exemplary embodiment.

[0038] Figure 2 This is a schematic diagram of an unmanned equipment cluster management system provided in an exemplary embodiment.

[0039] Figure 3 This is a schematic diagram of another unmanned equipment cluster management system provided in an exemplary embodiment.

[0040] Figure 4 This is an exemplary embodiment of a schematic diagram illustrating the interaction between a virtual simulation scene and a real-world scene.

[0041] Figure 5 This is a flowchart illustrating a method for splitting tasks to be executed and determining device information, provided in an exemplary embodiment.

[0042] Figure 6 This is a flowchart of an exemplary embodiment of a method for managing a cluster of unmanned devices.

[0043] Figure 7 This is a schematic diagram of the structure of a device provided in an exemplary embodiment. Detailed Implementation

[0044] With the rapid development of technology, unmanned equipment such as drones, unmanned vehicles, and unmanned ships have been widely used in various fields. These unmanned devices each have their own characteristics and are suitable for different scenarios: drones, with their flexible flight capabilities and rapid response speed, are suitable for tasks such as aerial reconnaissance, logistics delivery, and emergency rescue; unmanned vehicles, with their strong ground mobility and load-bearing capacity, perform excellently in logistics transportation, security patrols, and exploration of complex terrain; and unmanned ships play an important role in areas such as water monitoring, marine mapping, and water rescue.

[0045] However, in many complex tasks, a single type of unmanned equipment (UAV) is often insufficient to meet the requirements, necessitating the collaborative operation of multiple UAVs. But significant differences exist among different UAVs in performance, specifications, and equipment limitations, posing a considerable challenge to collaborative work. For example, adverse weather conditions such as strong winds, heavy rain, and dense fog can severely impact the flight stability and control capabilities of UAVs, while complex terrain, such as mountains and hills, can increase the stability of UAV operations. Therefore, how to achieve real-time, efficient collaborative operation among UAVs and how to manage UAV clusters in real-time have become pressing technical problems that need to be solved in the field of unmanned systems.

[0046] This specification provides an unmanned equipment cluster management system. Heterogeneous unmanned equipment such as unmanned vehicles, drones, and unmanned ships, as well as heterogeneous unmanned equipment clusters, can be connected to this unmanned equipment cluster management system to realize the planning and scheduling of complex tasks.

[0047] Figure 1 This is a schematic diagram of the architecture of an unmanned equipment cluster management system provided in an exemplary embodiment. For example... Figure 1 As shown, unmanned aerial vehicles (UAVs), unmanned vehicles, unmanned boats, and other unmanned equipment can all be connected to this unmanned equipment cluster management system. The physical carrier of this unmanned equipment cluster management system can be a physical server of an independent host, or a virtual server hosted by a host cluster, etc., and this specification does not impose any special restrictions on this.

[0048] For the network that enables interaction between unmanned devices such as drones, unmanned vehicles, and unmanned ships and the unmanned device cluster management system, communication can be achieved based on the communication methods supported by the corresponding unmanned devices, such as choosing to use wireless networks. Specifically, data interaction can be achieved through preset protocols and standardized interfaces.

[0049] It should be noted that, for each type of heterogeneous unmanned equipment, Figure 1The illustration shows only one corresponding device. In practical applications, unmanned vehicle (UAV) swarms, UAV swarms, and unmanned vessel (UV) swarms can all be connected to the unmanned equipment management cluster provided in this manual. Taking an UAV swarm as an example, the UAVs in this swarm can include various heterogeneous UAVs of different specifications / models, and the performance of different specifications / models of UAVs usually varies. Similarly, a UAV swarm can also include various heterogeneous UAVs of different specifications / models, and an UV swarm can also include various heterogeneous UVs of different specifications / models.

[0050] Therefore, it can be seen that the unmanned equipment cluster management system provided in this manual is adaptable to various heterogeneous unmanned equipment clusters and has a wider range of adaptability.

[0051] Please refer to Figure 2 In an exemplary embodiment, the unmanned equipment cluster management system 200 may include an information acquisition module 201, an intelligent decision-making module 202, an intelligent planning module 203, an intelligent scheduling module 204, a feedback receiving module 205, and an execution display module 206.

[0052] The information acquisition module 201 can acquire information on tasks to be executed, unmanned equipment cluster information, unmanned equipment available resource information, and the initial environment information of the tasks to be executed.

[0053] The unmanned equipment cluster information may include the equipment type, model, load, battery level, and battery life of each unmanned equipment in the cluster. Taking an unmanned equipment cluster comprising drones, unmanned boats, and unmanned vehicles as an example, the cluster information could be: 20 drones, 10 unmanned vehicles, and 8 unmanned boats. Of the 20 drones, 6 are model A drones, with a load of **KG and a battery level of **mAh. This cluster information can be uploaded by relevant personnel or automatically uploaded by the unmanned equipment after connecting to the unmanned equipment cluster management system 200; this specification does not impose any special restrictions on this.

[0054] The tasks to be executed may include information necessary for the execution of the task itself, such as task details, and may also include time requirements, quality requirements, and task priority. The task details may include task type, task location, and task volume. The task type may include transportation tasks, search and rescue tasks, and inspection tasks. Taking a transportation task as an example, the task details may include the origin of the transportation, the destination of the transportation, the type, specifications, and total weight of the goods to be transported. The tasks to be executed can be uploaded by relevant personnel.

[0055] The available resource information for unmanned equipment includes replacement batteries, charging stations, and storage areas. Storage areas include, for example, airports for drone takeoffs and landings, docking spaces for unmanned boats, and parking spaces for unmanned vehicles.

[0056] The initial environmental information can be the current environmental information of the task location corresponding to the task to be performed, which may include terrain information, climate information (temperature, wind speed, precipitation), and may also include dynamic obstacle information (such as other unmanned equipment) and static obstacle information (such as buildings, mountains), etc. The initial environmental information can be uploaded by relevant personnel, or it can be obtained by the information acquisition module 201 from the Internet or the Internet of Things based on the task location.

[0057] The intelligent decision-making module 202 can divide the task to be executed into multiple sub-tasks based on the information collected by the information acquisition module 201, and determine the equipment information and available resources for each sub-task. The equipment information may include the type of unmanned equipment required to execute the corresponding sub-task, the unmanned equipment signal, the number of unmanned equipment, etc. For example, the task to be executed may be divided into 3 sub-tasks. Sub-task 1 requires 2 model A drones, and the available resources for sub-task 1 include 1 unmanned equipment charging station and a drone take-off and landing airport, etc.

[0058] The intelligent decision-making module 202 aims to maximize task completion efficiency, maximize available resource utilization, minimize energy consumption of the unmanned equipment cluster, and minimize the negative impact of the environment on the unmanned equipment cluster. It divides the task to be executed into multiple sub-tasks and determines the equipment information and available resources required to execute each sub-task. The specific determination method will be described in detail in subsequent embodiments.

[0059] The intelligent planning module 203, for each subtask after decomposition, can determine the task plan corresponding to the subtask based on its required device information and available resources. The task plan may include specific operable information such as the correspondence between the available resources of the subtask, the execution path, and the required devices. The distance between the available resources of the subtask and the execution path corresponding to the required devices is within a preset distance (i.e., the available resources of the subtask are located near the execution path; the preset distance can be 1 kilometer, 2 kilometers, etc.). For example, graph search algorithms can be used for task planning.

[0060] Taking the aforementioned subtask 1, which requires two Model A drones, as an example, the intelligent planning module 203 can plan execution paths for these two drones respectively, and can also plan the flight speed, flight attitude, etc., used to execute subtask 1. The execution paths for these two drones are usually different; for example, subtask 1 can be divided into two execution paths, each executed by a different drone.

[0061] The intelligent scheduling module 204 can obtain the device status of each unmanned device in the unmanned device cluster, and can schedule the target unmanned device according to the device status and task planning to execute the sub-task based on the corresponding execution path and the available resources of the corresponding sub-task.

[0062] The device status may include the remaining battery power of the unmanned device, whether the unmanned device is currently available (e.g., whether the unmanned device is currently performing other tasks), and the device location. The device status can be obtained by the intelligent scheduling module 204 after receiving the task plan determined by the intelligent planning module 203; the device status can also be actively reported by the unmanned device, for example, the unmanned device can periodically report its own device status to the system, and the intelligent scheduling module 204 can obtain the latest device status cached in the system.

[0063] Taking the aforementioned subtask 1, which requires two Model A drones, as an example, the intelligent scheduling module 204 determines that all six Model A drones in the system are available. Drone A1 is closer to the planned first execution path, and drone A2 is closer to the planned second execution path. Therefore, drone A1 can be called to execute subtask 1 according to the first execution path, and drone A2 can be called to execute subtask 1 according to the second execution path. Of course, if the intelligent scheduling module 204 determines that drone A1's remaining battery power is insufficient to support its task execution, drone A3, which is further away, can also be called to execute subtask 1 according to the first execution path, to ensure that the task can be completed in one takeoff without needing to recharge mid-flight.

[0064] Alternatively, available resources such as charging equipment can be allocated to drone A1, and a path can be planned to go to and from the available resources for charging. This path also belongs to the first execution path corresponding to drone A1. Drone A1 can then execute sub-task 1 according to the corresponding first execution path. During the execution process, it will reach the charging equipment to charge according to the execution path, and can continue to execute the corresponding sub-task after charging is completed.

[0065] Alternatively, nearby airports or other available resources can be allocated to the A1 drone. If the A1 drone encounters an emergency during its mission, it can land at one of the nearby airports it was allocated.

[0066] The feedback receiving module 205 can dynamically receive execution data fed back by the target unmanned device during the execution of the sub-task. The execution data may include device information such as current location, current speed, and remaining battery power, and may also include data such as images and videos captured by the device.

[0067] The execution display module 206 can synchronously display the corresponding virtual execution instance in the virtual simulation scene based on the execution data received by the feedback receiving module 205, thereby realizing real-time simulation.

[0068] As described above, the unmanned equipment cluster management system provided in this specification allows the intelligent decision-making module to collect information on the tasks to be executed, the unmanned equipment cluster, available resources of the unmanned equipment, and the initial environment from the information acquisition module. With the goals of maximizing task completion efficiency, maximizing available resource utilization, minimizing energy consumption of the unmanned equipment cluster, and minimizing the negative impact of the environment on the unmanned equipment cluster, the system breaks down the tasks to be executed into multiple sub-tasks. It also determines the equipment information and available resources required to execute each sub-task, thereby optimizing multiple objectives simultaneously, improving the quality of task breakdown, and reducing the difficulty of subsequent task planning. Next, the intelligent planning module determines the execution path and available resource allocation for each sub-task. The intelligent scheduling module then schedules suitable target unmanned equipment according to the current equipment status of the unmanned equipment, utilizing the corresponding available resources of the sub-task to execute the sub-task according to the execution path. This achieves flexible scheduling of various unmanned equipment and efficient real-time collaborative operation. Simultaneously, the execution display module in the unmanned equipment cluster management system can also synchronously display corresponding virtual execution instances in a virtual simulation scene based on the execution data of the target unmanned equipment dynamically received by the feedback receiving module, facilitating real-time monitoring of task execution by relevant personnel.

[0069] In another embodiment of this specification, please refer to Figure 3 After receiving the execution data fed back by the target unmanned device, the feedback receiving module 205 can also determine the operating status of the target unmanned device based on the execution data. The operating status may include normal and abnormal.

[0070] Specifically, the feedback receiving module 205 can determine the operating status of the target unmanned device based on the current speed and remaining battery power in the execution data. For example, if the remaining battery power of the target unmanned device is 0 or less than a threshold, it is determined that the target unmanned device is operating abnormally. As another example, if the current speed of the target unmanned device is 0, meaning the target unmanned device has stopped moving, and combined with the video or images it has captured, it is determined that there is an obstacle in front of it, indicating that the target unmanned device may have collided with the obstacle and caused an accident, thus confirming that its operation is abnormal.

[0071] In this embodiment, if the feedback receiving module 205 determines that the operating status of the target unmanned device is abnormal, the intelligent scheduling module 204 can schedule an available unmanned device to take over the abnormal target unmanned device and continue to execute the corresponding sub-task.

[0072] Specifically, the intelligent scheduling module 204 can first identify available unmanned devices in the unmanned device cluster, for example, identifying unmanned devices that are currently not executing tasks as available unmanned devices. Then, it can schedule the unfinished sub-tasks of the abnormal target unmanned devices to the available unmanned devices. For example, it can send the execution paths that have not yet been executed in the abnormal target unmanned devices to the available unmanned devices, so that the available unmanned devices can continue to execute the sub-tasks based on the corresponding execution paths. Of course, in practical applications, depending on the task type, it may be necessary to remind other personnel to assist the available unmanned devices in taking over the sub-tasks. For example, if the task to be executed is a shooting task, then no other personnel are needed; the available unmanned devices can continue to execute the shooting task according to the remaining execution path after arriving at the corresponding location. As another example, if the task to be executed is a transportation task, then the intelligent scheduling module 204 will also send reminder information to relevant personnel to remind them to come to the scene to assist in the handling of the transported objects, etc. This specification does not impose any special restrictions on this.

[0073] Therefore, the unmanned equipment cluster management system provided in this manual can also determine the operating status of the target unmanned equipment based on the execution data fed back during the execution of the target unmanned equipment. In the event that the operating status of the target unmanned equipment is abnormal, it can schedule available unmanned equipment to take over the target unmanned equipment to continue to execute the corresponding sub-tasks, ensuring that the tasks to be executed are completed on schedule and efficiently, and avoiding situations where tasks cannot be completed on time due to abnormal operation of unmanned equipment.

[0074] In another embodiment of this specification, please refer to Figure 3 The unmanned equipment cluster management system can also optimize the execution of tasks based on the execution data fed back by the target unmanned equipment, so as to adapt to changes in environmental and other factors and ensure the efficient and high-quality completion of the tasks to be executed.

[0075] In this embodiment, the execution data fed back by the target unmanned device may also include the current environmental information it has detected. This current environmental information may include the actual terrain, current wind speed, current temperature, etc., which it has collected. The feedback receiving module 205 may also compare the current environment with the initial environmental information collected by the aforementioned information collection module 201 to determine whether the task environment for the task to be performed has undergone a significant change. The criteria for considering a significant environmental change can be pre-set based on the degree of impact of the environmental change on the unmanned device; this specification does not impose any special limitations on this.

[0076] Specifically, the initial environmental information may be obtained from the Internet or the Internet of Things (IoT), or uploaded by relevant personnel, and may differ from the actual environment. For example, the information acquisition module 201 may obtain a sunny weather forecast for the task location from the Internet or IoT, but during the execution of a sub-task by the target unmanned equipment, the weather may suddenly change, with strong winds and impending thunderstorms. After the target unmanned equipment feeds back the collected current weather information to the unmanned equipment cluster management system, the feedback receiving module 205 can determine that the task environment has undergone a significant change. As another example, relevant personnel may know that the task location is mountainous and upload initial environmental information about the mountainous area. Later, an asphalt road may be laid on the mountain. After the target unmanned equipment feeds back the collected video or images to the unmanned equipment cluster management system, the feedback receiving module 205 can also determine that the task environment has undergone a significant change.

[0077] In this embodiment, after the feedback receiving module 205 determines that the task environment has undergone a significant change, the intelligent decision-making module 202 can identify the remaining tasks that have not yet been completed in the tasks to be executed based on the execution data fed back by each target unmanned device. For example, objects that have not yet been transported to their destination, areas that have not yet been searched, etc. Then, the remaining tasks can be re-splittered based on the remaining tasks, the unmanned device cluster information, and the current environment information to divide the remaining tasks into multiple sub-remaining tasks, and the device information corresponding to each sub-remaining task can be determined. Specifically, the sub-remaining tasks can still be split with the goal of maximizing task completion efficiency, minimizing the energy consumption of the unmanned device cluster, and minimizing the negative impact of the environment on the unmanned device cluster.

[0078] In this embodiment, after the intelligent decision-making module 202 completes the re-splitting of the remaining tasks, the intelligent planning module 203 can re-plan the tasks based on the splitting results. Then, the intelligent scheduling module 204 can schedule the adapted unmanned equipment to execute the remaining tasks based on the re-planned execution path.

[0079] For example, suppose the target unmanned equipment is a drone, and the task to be performed is an inspection task. During the execution of the task, the weather suddenly changes, strong winds blow, and thunderstorms are about to occur. After the drone feeds back the current weather data to the unmanned equipment cluster management system, the feedback receiving module 205 can determine that the task environment has changed drastically. The intelligent decision module 202 re-divides the task, dividing the remaining task (the area that has not yet been inspected) into multiple sub-remaining tasks, and determines that the equipment information corresponding to these sub-remaining tasks is unmanned vehicles. The intelligent scheduling module 204 schedules the appropriate unmanned vehicles to replace the drone to continue to perform the inspection task, so as to avoid the drone from falling or other safety risks due to the extreme weather.

[0080] Therefore, the unmanned equipment cluster management system provided in this specification can also determine whether the task environment has changed significantly based on the execution data fed back by the target unmanned equipment during the execution of the task. When it is determined that the task environment has changed significantly, the task can be re-divided and the equipment information corresponding to the remaining sub-tasks can be re-determined. In this way, the appropriate unmanned equipment can be scheduled to continue to execute the task in a timely manner according to the changes in environmental information, which improves the task completion efficiency, reduces the wear and tear of unmanned equipment, and improves the robustness and flexibility of the system.

[0081] In another embodiment of this specification, the unmanned equipment cluster management system can also maintain a corresponding expert knowledge base. When making intelligent decisions, the intelligent decision module 202 can obtain expert knowledge related to the task to be executed, the unmanned equipment cluster, and the initial environment from the expert knowledge base, and then, based on the guidance of the expert knowledge, split the task to be executed and determine the equipment information corresponding to the sub-tasks.

[0082] The expert knowledge base can be a knowledge graph that integrates knowledge from multiple fields, including the performance of unmanned equipment, environmental information (such as climate and terrain), task characteristics, and industry standards. Specifically, semantic modeling can be performed on information from these fields to extract knowledge and experience related to the implementation of unmanned equipment tasks, and a knowledge graph including entities such as unmanned equipment, environment, and tasks can be constructed.

[0083] For example, the knowledge graph may include unmanned equipment entities, whose attributes may include the type of task they are suited for, their load capacity, speed limit, power consumption, and wind speed variation.

[0084] In this embodiment, initially, the expert knowledge base can be configured based on the parameters of the unmanned equipment and some industry experience. Subsequently, the unmanned equipment cluster management system can update the expert knowledge base based on the actual situation during task execution to retain the latest knowledge.

[0085] For details, please refer to Figure 3 The unmanned equipment cluster management system may also include a knowledge maintenance module 207, which can update the expert knowledge in the expert knowledge base according to the execution data fed back by the target unmanned equipment.

[0086] For example, in the initial state, the relationship between power consumption and wind speed recorded in the expert knowledge base for model A drone (such as how much power consumption increases when the wind speed exceeds a certain value) becomes inapplicable as the drone's battery performance declines with use. This relationship can no longer provide accurate data for the intelligent decision-making module. Therefore, the knowledge maintenance module 207 can re-determine the relationship between power consumption and wind speed based on the drone's feedback execution data during mission execution, thereby updating the corresponding expert knowledge in the expert knowledge base and providing accurate reference data for the intelligent decision-making module.

[0087] Similarly, in addition to the intelligent decision-making module, the intelligent planning module 203 can also refer to relevant expert knowledge in the knowledge base when planning tasks. This manual does not impose any special restrictions on this.

[0088] Therefore, the unmanned equipment cluster management system described in this manual can also maintain an expert knowledge base, thereby providing expert experience for task decomposition and planning, and improving the accuracy of task decomposition and planning. Simultaneously, the knowledge maintenance module can update the expert knowledge base in a timely manner based on the execution data fed back by the unmanned equipment during task execution. Compared to traditional static expert bases, this enables dynamic intelligent reasoning, thus ensuring that the system maintains adaptability and efficiency when facing new tasks and new environments.

[0089] In another embodiment of this specification, the unmanned equipment cluster management system can also send the task plan of each sub-task to local or remote unmanned equipment for physical simulation operation after determining the task plan of each sub-task, or send it to virtual unmanned equipment for virtual simulation. Specifically, the unmanned equipment can implement and verify the task in virtual simulation scenarios and real-world scenarios through digital twin technology, and the aforementioned task division and task planning can be optimized based on the verification results.

[0090] In one example, before scheduling unmanned equipment to perform a corresponding task, the intelligent scheduling module 204 can schedule virtual unmanned equipment to perform virtual simulation in a virtual simulation scenario to verify and optimize the task specifications. After the virtual simulation is completed, it can schedule local or remote unmanned equipment to perform the corresponding task.

[0091] In another example, based on scheduling virtual unmanned equipment to perform virtual simulations in a virtual simulation scenario, local or remote unmanned equipment can also be scheduled to perform corresponding tasks in a real-world scenario. Please refer to [reference needed]. Figure 4 Virtual simulation scenes and real-world scenes can be mapped and synchronized with each other, and can also communicate and interact with each other.

[0092] For example, unmanned devices in a real-world scenario can send their execution data to a virtual simulation scenario, which can then update its displayed virtual instances based on the execution data. For instance, if an unmanned device collects meteorological information such as wind speed in the real-world environment and synchronizes it to the virtual simulation scenario, the corresponding meteorological information can be simulated within that virtual simulation scenario. As another example, the virtual simulation scenario can also send optimized task plans (such as optimized execution paths) to unmanned devices in the real-world scenario after task planning optimization, so that the unmanned devices can execute the corresponding tasks according to the optimized task plan. The real-world scenario includes both local and remote real-world scenarios.

[0093] Therefore, it can be seen that by using the unmanned equipment cluster management system provided in this manual, the virtual simulation scene can interact with the real simulation scene, and thus, on the basis of dynamic monitoring of the task, the unmanned equipment in the real scene can also be dynamically adjusted and controlled.

[0094] The aforementioned virtual simulation scenario can be implemented through the execution and display module of the unmanned equipment cluster management system.

[0095] In another embodiment of this specification, please refer to Figure 5 The intelligent decision-making module in the unmanned equipment cluster management system can break down the task to be executed and determine the equipment information corresponding to the sub-tasks in the following ways:

[0096] Step 502: Obtain the optimization function, which is a weighted function of multiple sub-functions, wherein each sub-function corresponds to one or more sub-tasks, including an efficiency item corresponding to task completion efficiency, an energy consumption item corresponding to cluster energy consumption, an environmental impact item corresponding to negative impact, and a resource utilization item corresponding to the available resource utilization rate.

[0097] In this embodiment, the optimization function can be pre-constructed. Since the intelligent decision-making module can break down the task to be executed into multiple sub-tasks, the optimization function can be constructed as a weighted function of multiple sub-functions. Each sub-function can correspond to one or more of the sub-tasks, and the weight of each sub-function in the weighted function can also be pre-set.

[0098] In this embodiment, each sub-function may include an efficiency term representing the efficiency of completing external tasks, an energy consumption term corresponding to cluster energy consumption, an environmental impact term corresponding to the negative impact of the environment, and a resource utilization term corresponding to the utilization rate of available resources. That is, the sub-function includes terms corresponding to the optimization objective.

[0099] In this embodiment, the following optimization function can be constructed:

[0100]

[0101] in, It is an optimization function, w i f is the weight of the i-th subtask. i It is the sub-function corresponding to the i-th sub-task, and each sub-function f i All of them can include the aforementioned efficiency term, energy consumption term, and environmental impact term, where i can take values ​​from 1 to N, and N is the number of sub-functions to be weighted.

[0102] In the optimization function, T can represent the time model, which represents the time variables and functions of the unmanned equipment cluster operation, and can be used to construct efficiency terms, etc.

[0103] In the optimization function, E can represent the environmental model, indicating the negative impact of parameters such as terrain, wind speed, and climate on the unmanned equipment cluster, and can be used to construct environmental impact items, etc.

[0104] In the optimization function, J can represent the task model, indicating task priority, quantity, etc., and can be used to construct efficiency terms, energy consumption terms, etc.

[0105] In the optimization function, D can represent the entity model of the unmanned equipment, representing the equipment performance, specifications, parameters, quantity, unmanned equipment entity, etc., and can be used to construct efficiency items, energy consumption items, and environmental impact items, etc.

[0106] In the optimization function, S can represent the unmanned equipment simulation model. Similar to D, it can also represent equipment performance, specifications, parameters, quantity, unmanned equipment virtual simulation, etc. It can be used to construct efficiency items, energy consumption items, and environmental impact items, etc.

[0107] In the optimization function, C can represent the control model, which represents the state, behavior, and interaction of the unmanned equipment, and can be used to construct efficiency terms, energy consumption terms, and environmental impact terms, etc.

[0108] In the optimization function, K can represent the knowledge base model, indicating the decision-making and logic of cluster operations, and can be used to construct efficiency items, energy consumption items, and environmental impact items, etc.

[0109] In the optimization function, R can represent the available resource model for unmanned equipment.

[0110] Of course, in other examples, other optimization options can be set according to actual needs, and this manual does not impose any special restrictions on this.

[0111] Step 504: With the goal of minimizing the optimization function, and with the constraints that the negative impact of the initial environmental information on the device cluster is less than the environmental impact threshold, the execution time of the task to be executed is less than the time threshold, the device information meets the device restrictions corresponding to the unmanned device cluster, and the available resources used by the task do not exceed the available resource restrictions, the optimization function is solved to obtain the sub-task and the device information and available resources required for each sub-task.

[0112] In this embodiment, the negative impact of the initial environmental information on the device cluster may include safety impact, energy consumption impact, and equipment wear and tear impact. The device information conforming to the device constraints corresponding to the unmanned device cluster can be the device's own constraints (e.g., range not exceeding a certain kilometer), industry constraints (e.g., maximum driving speed limited in a certain area), or cluster constraints (e.g., the number of drones included in the cluster). The fact that the available resources used by the task do not exceed the available resource limits can be a constraint of the available resources themselves, such as the unmanned device charging station having 3 charging stations, capable of charging a maximum of 3 unmanned vehicles.

[0113] In this embodiment, the optimization function can be solved based on these constraints. For example, the Lagrange multiplier method or the penalty function method can be used to solve the optimization function. By solving, the subtasks of the task to be executed and the device information corresponding to each subtask can be obtained, as shown in the following formula:

[0114]

[0115] subject to g(E)≤G,

[0116] h(D)≤H,

[0117] l(T)≤L,

[0118] r≤R

[0119] in, Let g(E)≤G represent the minimization optimization function, g(E)≤G means that the negative impact of the initial environmental information on the device cluster is less than the environmental impact threshold, h(D)≤H means that the device information meets the device constraints corresponding to the unmanned device cluster, l(T)≤L means that the execution time of the task to be executed is less than the time threshold, and r≤R means that the available resources have not exceeded the available resource limit.

[0120] Of course, in other examples, other constraints may be set as needed, and this specification does not impose any special restrictions on them.

[0121] Therefore, this specification demonstrates that by solving a multi-objective optimization function, the task to be executed is broken down into multiple sub-tasks, and the corresponding device information and available resources for each sub-task are obtained. This allows for a trade-off among multiple objectives, enabling the breakdown of complex tasks and improving the efficiency and flexibility of subsequent task execution.

[0122] In another embodiment of this specification, a method for managing unmanned equipment clusters is also provided. This method can be applied to an unmanned equipment cluster management system. Please refer to [reference needed]. Figure 6 The method may include the following steps:

[0123] Step 602: Collect information on the task to be executed, the unmanned equipment cluster, the available resources of the unmanned equipment, and the initial environment information of the task to be executed.

[0124] Step 604: Based on the task to be executed, the unmanned equipment cluster information, the unmanned equipment available resource information, and the initial environment information, with the goals of maximizing task completion efficiency, maximizing available resource utilization, minimizing unmanned equipment cluster energy consumption, and minimizing the negative impact of the environment on the unmanned equipment cluster, the task to be executed is divided into multiple sub-tasks, and the equipment information and available resources of each sub-task are determined.

[0125] Step 606: For each subtask, determine the task plan corresponding to the subtask based on its required device information and the available resources of the subtask. The task plan includes the correspondence between the available resources of the subtask, the execution path and the required device, wherein the available resources of the subtask are located on the execution path corresponding to the required device or the distance between the available resources of the subtask and the execution path is within a preset distance.

[0126] Step 608: Obtain the device status of the unmanned devices in the unmanned device cluster, and according to the device status and the target unmanned device adapted by the task planning and scheduling, execute the sub-task using the available resources of the corresponding sub-task based on the corresponding execution path.

[0127] Step 610: Dynamically receive the execution data fed back by the target unmanned device during the execution of the sub-task.

[0128] Step 612: Based on the execution data, synchronously display the corresponding virtual execution instance in the virtual simulation scene.

[0129] In this embodiment, the specific implementation of the above process can be referred to the foregoing embodiments, and will not be repeated here.

[0130] For example, the task to be executed and the device information and available resources for the subtask can be determined in the following manner:

[0131] Obtain an optimization function, which is a weighted function of multiple sub-functions, wherein each sub-function corresponds to one or more sub-tasks, including an efficiency item corresponding to the task completion efficiency, an energy consumption item corresponding to the cluster energy consumption, an environmental impact item corresponding to the negative impact, and a resource utilization item corresponding to the available resource utilization rate.

[0132] With the objective of minimizing the optimization function, and constrained by the following conditions: the negative impact of the initial environmental information on the device cluster is less than the environmental impact threshold, the execution time of the task to be executed is less than the time threshold, the device information meets the device restrictions corresponding to the unmanned device cluster, and the available resources used by the task do not exceed the available resource restrictions, the optimization function is solved to obtain the sub-task and the device information and available resources required for each sub-task.

[0133] For example, the management method for an unmanned equipment cluster may further include: determining the operating status of a corresponding target unmanned equipment based on the execution data; if the operating status of the target unmanned equipment is abnormal, determining available unmanned equipment, and scheduling the sub-tasks that the abnormal target unmanned equipment has not completed to the available unmanned equipment, so that the available unmanned equipment can continue to execute the sub-tasks based on the corresponding execution path.

[0134] For example, the execution data also includes the current environment information detected by the target unmanned device. The management method of the unmanned device cluster may further include: determining whether the task environment of the task to be executed has changed significantly based on the current environment information and the initial environment information; if the task environment has changed significantly, determining the remaining unexecuted tasks in the task to be executed, and re-splitting the remaining tasks based on the remaining tasks, the unmanned device cluster information and the current environment information, and determining the equipment information required for each sub-remaining task after re-splitting.

[0135] For example, the management method for unmanned equipment clusters can also obtain expert knowledge related to the task to be executed, the unmanned equipment cluster, and the initial environment from an expert knowledge base; the process of the intelligent decision module splitting the task to be executed and determining the equipment information includes: splitting the task to be executed and determining the equipment information and the available resources of the sub-tasks based on the expert knowledge with the objectives of maximizing task completion efficiency, maximizing the utilization rate of available resources, minimizing the energy consumption of the unmanned equipment cluster, and minimizing the negative impact of the environment on the unmanned equipment cluster.

[0136] For example, the management method for an unmanned equipment cluster may further include: updating the expert knowledge in the expert knowledge base based on the execution data.

[0137] For example, the expert knowledge comes from the unmanned equipment mission implementation knowledge graph. The knowledge graph is constructed from unmanned equipment-related knowledge, experience, industry standards, environmental information (terrain, obstacles, climate, etc.), and unmanned equipment specifications, including unmanned equipment entities, mission entities, and environmental entities.

[0138] The specific implementation process of the above method can also be referred to the aforementioned unmanned equipment cluster management system embodiment, and will not be repeated here.

[0139] Figure 7 This is a schematic structural diagram of a device provided in an exemplary embodiment. The physical carrier of the aforementioned unmanned equipment cluster management system can be... Figure 7 The device shown. Please refer to... Figure 7 At the hardware level, the device includes a processor 702, an internal bus 704, a network interface 706, memory 708, and non-volatile memory 710, and may also include other hardware required for its functions. One or more embodiments of this specification can be implemented in software, for example, the processor 702 reads the corresponding computer program from the non-volatile memory 710 into memory 708 and then runs it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0140] Based on the same concept as the methods described above, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described in any of the above embodiments by executing the executable instructions.

[0141] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0142] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

Claims

1. A cluster management system for unmanned equipment, characterized in that, The system includes: The information acquisition module is used to collect information on the task to be executed, the unmanned equipment cluster, the available resources of the unmanned equipment, and the initial environmental information of the task to be executed. The intelligent decision-making module is used to divide the task to be executed into multiple sub-tasks based on the task to be executed, the information of the unmanned equipment cluster, the information of available resources of the unmanned equipment, and the information of the initial environment, with the goal of maximizing task completion efficiency, maximizing the utilization rate of available resources, minimizing the energy consumption of the unmanned equipment cluster, and minimizing the negative impact of the environment on the unmanned equipment cluster, and to determine the equipment information and available resources of each sub-task to execute. The intelligent planning module is used to determine the task plan corresponding to each subtask based on its required device information and the available resources of the subtask. The task plan includes the correspondence between the available resources of the subtask, the execution path and the required device, wherein the available resources of the subtask are located on the execution path corresponding to the required device or the distance between the subtask and the execution path is within a preset distance. The intelligent scheduling module is used to obtain the device status of unmanned devices in the unmanned device cluster, and according to the device status and the task planning, the target unmanned device is scheduled to execute the sub-task based on the corresponding execution path and the available resources of the corresponding sub-task. The feedback receiving module is used to dynamically receive the execution data fed back by the target unmanned device during the execution of the sub-task; The execution display module is used to synchronously display the corresponding virtual execution instance in the virtual simulation scene based on the execution data; The intelligent decision-making module uses the following method to break down the task to be executed and determine the device information and available resources for the sub-task: Obtain an optimization function, which is a weighted function of multiple sub-functions, wherein each sub-function corresponds to one or more sub-tasks, including an efficiency item corresponding to the task completion efficiency, an energy consumption item corresponding to the cluster energy consumption, an environmental impact item corresponding to the negative impact, and a resource utilization item corresponding to the available resource utilization rate. With the objective of minimizing the optimization function, and constrained by the following conditions: the negative impact of the initial environmental information on the device cluster is less than the environmental impact threshold, the execution time of the task to be executed is less than the time threshold, the device information meets the device restrictions corresponding to the unmanned device cluster, and the available resources used by the task do not exceed the available resource restrictions, the optimization function is solved to obtain the sub-task and the device information and available resources required for each sub-task.

2. The system according to claim 1, characterized in that, The feedback receiving module is also used to determine the operating status of the corresponding target unmanned equipment based on the execution data; The intelligent scheduling module is also used to determine available unmanned equipment when the operating state of the target unmanned equipment is abnormal, and to schedule the sub-tasks that the abnormal target unmanned equipment has not completed to the available unmanned equipment, so that the available unmanned equipment can continue to execute the sub-tasks based on the corresponding execution path.

3. The system according to claim 1, characterized in that, The execution data also includes the current environmental information detected by the target unmanned device; The feedback receiving module is also used to determine whether the task environment of the task to be executed has changed significantly based on the current environment information and the initial environment information; The intelligent decision-making module is also used to determine the remaining unexecuted tasks in the tasks to be executed when the task environment undergoes a significant change, and to re-divide the remaining tasks based on the remaining tasks, the unmanned equipment cluster information and the current environment information, and to determine the equipment information required for each sub-remaining task after re-dividement.

4. The system according to claim 1, characterized in that, The intelligent decision-making module is also used to obtain expert knowledge related to the task to be executed, the unmanned equipment cluster, and the initial environment from the expert knowledge base; The process by which the intelligent decision-making module breaks down the task to be executed and determines the device information includes: Based on the expert knowledge, the tasks to be executed are broken down with the objectives of maximizing task completion efficiency, maximizing the utilization rate of available resources, minimizing the energy consumption of the unmanned equipment cluster, and minimizing the negative impact of the environment on the unmanned equipment cluster, and the equipment information and available resources of the sub-tasks are determined.

5. The system according to claim 4, characterized in that, The system also includes: The knowledge maintenance module is used to update the expert knowledge in the expert knowledge base based on the execution data.

6. A method for managing a cluster of unmanned equipment, characterized in that, The method includes: Collect information on tasks to be executed, unmanned equipment cluster information, available resources of unmanned equipment, and the initial environment information of the tasks to be executed; Based on the task to be executed, the unmanned equipment cluster information, the unmanned equipment available resource information, and the initial environment information, with the goals of maximizing task completion efficiency, maximizing available resource utilization, minimizing unmanned equipment cluster energy consumption, and minimizing the negative impact of the environment on the unmanned equipment cluster, the task to be executed is divided into multiple sub-tasks, and the equipment information and available resources of each sub-task are determined. For each subtask, a task plan is determined based on the required device information and the available resources of the subtask. The task plan includes the correspondence between the available resources of the subtask, the execution path, and the required device. The available resources of the subtask are located on the execution path corresponding to the required device or the distance between the available resources and the execution path is within a preset distance. Obtain the device status of unmanned devices in the unmanned device cluster, and according to the device status and the target unmanned device adapted by the task planning and scheduling, execute the sub-task using the available resources of the corresponding sub-task based on the corresponding execution path; Dynamically receive execution data fed back by the target unmanned device during the execution of the sub-task; Based on the execution data, the corresponding virtual execution instance is synchronously displayed in the virtual simulation scene; The task to be executed and the device information and available resources for the sub-tasks are determined in the following manner: Obtain an optimization function, which is a weighted function of multiple sub-functions, wherein each sub-function corresponds to one or more sub-tasks, including an efficiency item corresponding to the task completion efficiency, an energy consumption item corresponding to the cluster energy consumption, an environmental impact item corresponding to the negative impact, and a resource utilization item corresponding to the available resource utilization rate. With the objective of minimizing the optimization function, and constrained by the following conditions: the negative impact of the initial environmental information on the device cluster is less than the environmental impact threshold, the execution time of the task to be executed is less than the time threshold, the device information meets the device restrictions corresponding to the unmanned device cluster, and the available resources used by the task do not exceed the available resource restrictions, the optimization function is solved to obtain the sub-task and the device information and available resources required for each sub-task.

7. An electronic device, characterized in that, include: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method as claimed in claim 6 by executing the executable instructions.

8. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method as described in claim 6.

9. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the method as described in claim 6.

Images