Implementation method, device and equipment for unified control system of heterogeneous unmanned platform

Abstract

The invention provides an implementation method, device and equipment for a unified control system of heterogeneous unmanned platforms and a medium, and the method comprises the steps: carrying out the communication based on a unified control protocol through first communication equipment and second communication equipment of the heterogeneous unmanned platforms, obtaining the platform information of each unmanned platform, and enabling the first communication equipment to be deployed at a control terminal; based on the platform information of each unmanned platform, task planning of the heterogeneous unmanned platform is carried out, and the execution state of the task is tracked; various task loads carried by the heterogeneous unmanned platform are monitored, and sensing information collected by the task loads is obtained; and integrating the platform information, the execution state and the perception information, and displaying through a map interface.

Description

Technical Field

[0001] The present disclosure relates to the technical field of unmanned platform control, and particularly relates to a method, device, equipment and medium for implementing a unified control system for heterogeneous unmanned platforms. Background Art

[0002] The unmanned platform control system is a comprehensive system that conducts command sending and data transmission between the unmanned platform and the control terminal or other operating platforms according to the agreed communication system and protocol content, and completes functions such as unmanned platform control, providing basic support for unmanned platform mission planning, motion control, perception information transmission, and collaborative work with manned and unmanned platforms.

[0003] Currently, the air and ground unmanned platform control systems usually adopt a method of one set of control equipment controlling one set of unmanned platforms. The control protocols, control links, control software, and control terminals of unmanned platforms from different manufacturers and of different specifications and models are different. There are problems restricting integrated control such as incompatible protocols, frequency band interference, inability to expand, and inconvenient operation when used in coordination, which hinder the interconnection and interoperability between multiple unmanned platforms. Moreover, with the booming development of the unmanned system technology field, a large number of new unmanned platforms have emerged, and requirements for the expandable control of various new heterogeneous multi-unmanned platforms have also been put forward. Summary of the Invention

[0004] (I) Technical Problems to be Solved

[0005] In view of the above problems, the present disclosure provides a method, device, equipment and medium for implementing a unified control system for heterogeneous unmanned platforms, so as to at least partially solve the technical problem of poor collaborative ability of the unmanned platform system caused by the great difficulty in integrated control of heterogeneous unmanned platforms in the prior art.

[0006] (II) Technical Solutions

[0007] On the one hand, the present disclosure provides a method for implementing a unified control system for heterogeneous unmanned platforms, including: communicating with the second communication device of the heterogeneous unmanned platform through the first communication device based on a unified control protocol to obtain the platform information of each unmanned platform, where the first communication device is deployed on the control terminal; performing mission planning for the heterogeneous unmanned platforms based on the platform information of each unmanned platform, and tracking the execution status of the mission; monitoring various mission payloads carried by the heterogeneous unmanned platforms to obtain the perception information collected by the mission payloads; integrating the platform information, the execution status, and the perception information, and displaying them through a map interface.

[0008] According to an embodiment of the present disclosure, the message frame of the control protocol adds at least one ID field on the basis of the general message frame, and the ID field is used to characterize and distinguish the attributes of the heterogeneous unmanned platforms.

[0009] According to an embodiment of the present disclosure, it further includes: constructing a platform list for displaying the platform information; querying or filtering the unmanned platforms in the platform list that meet preset conditions based on the platform information; and / or sending control instructions to control each of the unmanned platforms based on the platform list.

[0010] According to an embodiment of the present disclosure, the task planning for the heterogeneous unmanned platforms based on the platform information of each of the unmanned platforms includes: determining the starting point and ending point of the task based on the platform information of each of the unmanned platforms; manually marking a path on the map interface according to the starting point and ending point of the task to obtain a planned path; or automatically planning a planned path from the starting point to the ending point of the task by using the environmental information of the unmanned platform.

[0011] According to an embodiment of the present disclosure, the task planning for the heterogeneous unmanned platforms based on the platform information of each of the unmanned platforms further includes: decomposing the tasks of the unmanned platforms based on the platform information of each of the unmanned platforms to obtain at least two subtasks; controlling the unmanned platforms to execute the subtasks sequentially or in parallel until all the subtasks are completed.

[0012] According to an embodiment of the present disclosure, it further includes: obtaining the alarm information of the unmanned platform and the mission payload based on the perception information, and adjusting the task planning of the heterogeneous unmanned platforms according to the alarm information.

[0013] According to an embodiment of the present disclosure, it further includes: performing basic plotting operations on a map interface that integrates the platform information, the execution status, and the perception information to draw corresponding identification information; performing perspective analysis on the map interface to determine the line-of-sight obstruction situation between any two points on the map interface; and adjusting the task according to the identification information and the line-of-sight situation.

[0014] A second aspect of the present disclosure provides an implementation device for a unified control system of heterogeneous unmanned platforms, including: an acquisition module for communicating with a second communication device of a heterogeneous unmanned platform through a first communication device based on a unified control protocol to obtain the platform information of each unmanned platform, where the first communication device is deployed on a control terminal; a planning module for performing task planning for the heterogeneous unmanned platforms based on the platform information of each of the unmanned platforms and tracking the execution status of the tasks; a perception module for monitoring various mission payloads carried by the heterogeneous unmanned platforms to obtain the perception information collected by the mission payloads; and a display module for integrating the platform information, the execution status, and the perception information and displaying them through a map interface.

[0015] A third aspect of the present disclosure provides an electronic device, including: one or more processors; a storage device for storing one or more computer programs, which, when executed by the one or more processors, cause the one or more processors to execute the respective steps in the method for implementing the unified control system of heterogeneous unmanned platforms.

[0016] A fourth aspect of the present disclosure provides a computer-readable storage medium, on which executable instructions are stored, and when the instructions are executed by a processor, the respective steps in the method for implementing the unified control system of heterogeneous unmanned platforms are realized.

[0017] (III) Advantageous Effects

[0018] The method, device, equipment and medium for implementing the unified control system of heterogeneous unmanned platforms provided by the present disclosure unify the information interaction protocol of heterogeneous unmanned platforms at the communication mechanism level through a standardized and extensible unified control protocol, realize the integrated control of heterogeneous unmanned platforms, and incorporate any type of unmanned platform into the scope of integrated control in the form of a control plug-in. In addition, according to actual application requirements, the basic functions involved in the unified control of heterogeneous unmanned platforms and payloads are designed, which can be used as a reference for the design and development of any unmanned platform control system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] To more fully understand the present disclosure and its advantages, reference will now be made to the following description in conjunction with the accompanying drawings, in which:

[0020] Figure 1 Schematically shows a schematic diagram of an existing unified control system of heterogeneous unmanned platforms provided by an embodiment of the present disclosure;

[0021] Figure 2 Schematically shows a flowchart of the method for implementing the unified control system of heterogeneous unmanned platforms provided by an embodiment of the present disclosure;

[0022] Figure 3 Schematically shows a schematic diagram of the unified control system of heterogeneous unmanned platforms provided by an embodiment of the present disclosure;

[0023] Figure 4 Schematically shows a block diagram of the structure of the device for implementing the unified control system of heterogeneous unmanned platforms provided by an embodiment of the present disclosure;

[0024] Figure 5 Schematically shows a hardware structure diagram of an electronic device. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for the sake of explanation, numerous specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present disclosure.

[0026] The terms used herein are merely for describing specific embodiments and are not intended to limit the present disclosure. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0028] Some block diagrams and / or flowcharts are shown in the accompanying drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flowcharts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when executed by the processor, these instructions can create a device for implementing the functions / operations illustrated in these block diagrams and / or flowcharts.

[0029] The inventors have found through research that in the related art, a complete unmanned platform control system includes a control terminal, a communication device, an unmanned platform (including a control unit), and a mission payload, etc., as Figure 1 shown. However, since each system is independently equipped with a communication device and uses its own unique control protocol, this has led to serious compatibility problems between systems, thereby increasing the difficulty of interaction between systems and at the same time restricting the interconnection and interoperability between multiple unmanned platforms. In addition, when multiple unmanned platforms are required to cooperate to complete a task, the operator may need to operate multiple control terminals simultaneously, which not only increases the operation complexity but also reduces the efficiency and accuracy of task execution.

[0030] Therefore, the embodiments of the present disclosure provide a method for implementing a unified control system for heterogeneous unmanned platforms to solve the deficiencies of the prior art.

[0031] As Figure 2 shown, the flowchart of the method for implementing a unified control system for heterogeneous unmanned platforms includes S1 to S4.

[0032] In operation S1, the first communication device and the second communication device of the heterogeneous unmanned platform communicate based on a unified control protocol through the first communication device deployed on the control terminal to obtain the platform information of each unmanned platform.

[0033] Through the communication between the first communication device deployed on the control terminal and the second communication devices respectively deployed on each heterogeneous unmanned platform, an effective connection and information interaction between the control terminal and multiple heterogeneous unmanned platforms are realized. Among them, the first communication device, as a key component of the control terminal, is responsible for establishing communication links with each unmanned platform, sending control commands, and receiving feedback information. The second communication devices are respectively integrated on different types of heterogeneous unmanned platforms such as unmanned aerial vehicles, unmanned vehicles, and robots. They also follow the unified control protocol, can accurately parse the commands from the control terminal, and transmit important platform information such as the real-time status, position information, and sensor data of the unmanned platform back to the control terminal.

[0034] In operation S2, based on the platform information of each of the unmanned platforms, task planning for the heterogeneous unmanned platforms is performed, and the execution status of the tasks is tracked.

[0035] Based on the platform information of each unmanned platform, such as the type, performance parameters, current position, and task requirements of the unmanned platform, tasks such as maneuvering and perception of the unmanned platform can be planned to generate a task list. This task list can include not only the task overview of each controllable unmanned platform, such as basic information like task name, task priority, and expected completion time, but also details of the execution status of the tasks (such as to be assigned, to be executed, in execution, paused, and completed) and task numbers. At the same time, during the task execution process, the execution status of the tasks can be tracked to obtain and display the latest status of each task in real time.

[0036] In operation S3, various task payloads carried by the heterogeneous unmanned platforms are monitored to obtain the perception information collected by the task payloads.

[0037] Unmanned platforms can carry various types of mission payloads (mission payloads refer to those devices equipped on unmanned platforms to complete specific tasks), including but not limited to optoelectronic payloads, communication detection payloads, radar payloads, and environmental monitoring payloads, etc. Each payload has specific functions and data acquisition capabilities. For example, optoelectronic payloads can acquire video information, and communication detection payloads can acquire information of communication devices, etc. By monitoring various mission payloads carried on heterogeneous unmanned platforms, it can ensure that the unmanned platforms can efficiently execute diverse tasks and obtain key information in real time. For example, payload status monitoring can reflect the working status of the payload in real time, such as whether it is online and whether the working parameters are normal, etc.; payload data acquisition and transmission monitoring can ensure that the data collected by the payload can be accurately and timely transmitted to the control terminal.

[0038] In addition, the perception information of various mission payloads carried on the unmanned platform can be displayed in a single window or multiple windows. Among them, single-window display is suitable for scenarios with fewer types of information or scenarios that require centralized display, while multi-window display is suitable for scenarios with more types of information or scenarios that require simultaneous attention to multiple aspects.

[0039] In operation S4, the platform information, the execution status, and the perception information are integrated and displayed through a map interface.

[0040] During the integration process, first, it can be ensured that the platform information collected from each unmanned platform is accurate and real-time. This information includes but not limited to key parameters such as the current position, speed, heading, remaining battery power or fuel quantity of the unmanned platform. Then, the platform information can be combined with the execution status information of the task and the perception information, and a unified display of the platform can be presented based on the map. In this process, the execution status can be intuitively displayed through icons, color changes, or dynamic markers on the map; the payload perception information can be associated with the corresponding positions on the map and displayed through overlay layers, pop-up windows, or dynamic icons, etc.

[0041] In addition, environmental information can also be integrated. Environmental information can include terrain and landforms, building distributions, road networks, and weather conditions, etc. Exemplarily, when planning the flight route of an unmanned platform, operators can refer to the terrain information in the environmental information on the map and choose to avoid complex terrains such as high mountains or canyons to ensure the safe flight of the unmanned platform.

[0042] Figure 3 Schematically shows a schematic diagram of the unified control system for heterogeneous unmanned platforms provided by an embodiment of the present disclosure.

[0043] As Figure 3As shown in the figure, the standardized and scalable unified control system for heterogeneous unmanned platforms provided by the embodiments of the present disclosure is deployed on a control terminal. Based on a unified control protocol and relying on the communication link provided by communication equipment, it realizes the unified control of unmanned platforms and mission payloads. Specifically, the unified control system for heterogeneous unmanned platforms mainly includes a unified control protocol and a unified control software module. Among them, the control software module is a control software function module designed based on the unified control protocol in the integrated control system, which realizes functions such as mission planning, control, and status monitoring of unmanned platforms and payloads, and may include modules such as platform management, mission management, monitoring management, and map display.

[0044] It can be understood that the unified control protocol is the basis for the integrated control of heterogeneous unmanned platforms and also the basis for multi-platform collaborative applications. Any platform that conforms to the protocol standard can be connected to the integrated control system. In addition, the unified control function design not only stipulates the basic function set of the unmanned control system but also supports function reconstruction and extension, thus serving as a reference for the design and development of any unmanned platform control system, software, and plugins.

[0045] In the embodiments of the present disclosure, the message frame of the control protocol adds at least one ID field on the basis of the general message frame, and the ID field is used to characterize and distinguish the attributes of the heterogeneous unmanned platforms.

[0046] The control protocol stipulates the transmission specifications of task instructions, perception information, etc. between the communication equipment of the unmanned platform and the communication equipment of the control terminal. The communication between the communication equipment and the unmanned platform and the control terminal usually mainly uses the UDP protocol or the serial port protocol.

[0047] Exemplarily, the unified control protocol can make adaptive improvement and extension to the general protocol MAVLinkV2 to meet the basic requirements for the transmission and exchange of control instructions and data information of heterogeneous unmanned platforms such as unmanned aerial vehicles, unmanned vehicles, and robots. The general message frame format of this protocol can be customized and extended based on MAVlink V2. Based on the message frame design of MAVLinkV2, the general message frame of this specification is designed. The data length of one frame of data can be at least 12 bytes and at most 280 bytes.

[0048] Since heterogeneous unmanned platforms involve multiple types of platforms such as aerial unmanned platforms, ground unmanned platforms (including unmanned vehicles and robots), and clusters composed of the above unmanned platforms, compared with the original MAVLinkV2 message frame (i.e., a structure used for encapsulating and transmitting data in network communication), multiple ID fields can be added, such as platform type ID, system ID, reserved ID, and message type ID fields. Among them, the platform type ID is used to distinguish platforms such as control platforms, unmanned aerial vehicles, unmanned vehicles, robots, and unmanned clusters; the system ID is used to distinguish IDs for corresponding types of platforms; the reserved ID is used for expandable IDs when the platform ID and system ID do not meet business requirements; the message type ID is used to distinguish the types of communication messages, including common protocols, unmanned aerial vehicle type protocols, unmanned vehicle type protocols, robot type protocols, cluster type protocols, etc.; the message ID is used for specific instruction IDs under the corresponding message type. The overall protocol improves the system / systematicness, readability, flexibility, and scalability of the protocol.

[0049] The message frame structure and its interpretations are shown in Table 1 and Table 2.

[0050] Table Message frame structure

[0051]

[0052] Table Interpretation of message frame structure

[0053]

[0054] It can be understood that through the unified and expandable settings of the above control protocols, it becomes possible to standardize and expand the control of heterogeneous unmanned platforms.

[0055] In the embodiments of the present disclosure, it further includes: constructing a platform list for displaying the platform information; querying or filtering the unmanned platforms in the platform list that meet preset conditions based on the platform information; and / or sending control instructions to control each of the unmanned platforms based on the platform list.

[0056] Specifically, in order to display and manage the controllable unmanned platforms, a platform list can be constructed to show all the platform information that the current operator has the control authority over. In the list, key information such as the model and operating status (such as idle, busy or offline) of each platform can be visually viewed. At the same time, to facilitate the user to quickly locate a specific platform, platform search can be carried out, that is, by entering keywords (such as platform number, name, etc.) or selecting preset conditions (such as platform type) to query the specified platform in the list. In addition, different status platforms can be displayed through status filtering, that is, by selecting preset conditions such as "idle", "busy" or "offline" status, so that the system only displays the platforms that meet the filtering conditions.

[0057] When the user clicks on a certain platform in the platform list, a detailed information page of the platform can be displayed. The page content can include basic configuration information such as the platform number, name, load capacity and mileage, and can also include other detailed information (such as platform location, battery power and maintenance records, etc.).

[0058] Platform control is the control entry for each platform. Instructions can be sent to control each platform, such as starting, stopping, moving, executing tasks, etc. At the same time, real-time feedback of the control results can also be supported to ensure that the user can accurately understand the response of the platform.

[0059] It can be understood that by constructing a platform list and providing corresponding functions such as platform search and platform control, a comprehensive and efficient management platform is provided, which helps to improve the utilization rate and control efficiency of unmanned platforms and reduce the operation cost and maintenance cost.

[0060] Based on the above embodiments, in this embodiment, the task planning of the heterogeneous unmanned platforms based on the platform information of each of the unmanned platforms includes: determining the starting point and ending point of the task based on the platform information of each of the unmanned platforms; manually marking the path on the map interface according to the starting point and ending point of the task to obtain the planned path; or automatically planning the planned path from the starting point to the ending point of the task by using the environmental information of the unmanned platform.

[0061] During the task planning process of heterogeneous unmanned platforms, manual path planning or automatic path planning can be carried out according to the scene requirements.

[0062] Specifically, first, based on the platform information (current location), mission requirements, and relevant constraints (such as time limit, energy limit, etc.) of the unmanned platform, the starting point and ending point of the mission can be determined. The starting point can be the current location of the unmanned platform, and the ending point can be the specific location of the mission objective, such as the material delivery point, the reconnaissance target area, etc. In the manual path planning mode, the operator can fully consider various actual situations, such as the terrain complexity, the distribution of obstacles, and the weather changes, etc., and thus, according to the mission requirements and their own experience, mark the path on the map interface in the form of dotting and drawing lines in sequence to determine the planned path. In addition, to reduce the burden on the operator and improve the efficiency of mission planning, automatic path planning can also be carried out. In the automatic path planning mode, the environmental information of the unmanned platform, such as topographic maps, obstacle distribution maps, etc., and real-time sensor data (such as radar, camera, etc.) can be used to plan an optimal or feasible path from the starting point to the ending point. To achieve automatic path planning, various algorithms can be adopted, such as Dijkstra algorithm, A* algorithm, etc. Among them, the Dijkstra algorithm is suitable for solving the single-source shortest path problem, while the A* algorithm combines heuristic search and cost function and can quickly find the optimal path in more complex scenarios.

[0063] Furthermore, after the path planning is completed, necessary adjustments and optimizations can be made. For example, some points or line segments on the path can be modified to avoid newly emerged obstacles or to utilize a more efficient path. In addition, according to the real-time state of the unmanned platform and the mission requirements, the path planning result can also be dynamically adjusted and optimized.

[0064] It can be understood that through the flexible switching, adjustment, and optimization of manual path planning and automatic path planning, the path planning requirements under different scenarios and mission requirements can be met.

[0065] In some exemplary embodiments, based on the platform information of each of the unmanned platforms, for the mission planning of the heterogeneous unmanned platforms, it further includes: decomposing the mission of the unmanned platform based on the platform information of each of the unmanned platforms to obtain at least two subtasks; controlling the unmanned platforms to execute the subtasks sequentially or in parallel until all the subtasks are completed.

[0066] When conducting mission planning, based on the platform information of each unmanned platform, such as the type, performance, load capacity, and endurance of the platform, etc., the original mission of the unmanned platform can be refined into multiple subtasks, and an appropriate unmanned platform can be assigned to execute each subtask.

[0067] Exemplarily, since different types of unmanned platforms (such as unmanned aerial vehicles, unmanned vehicles, and unmanned ships, etc.) have different performances and characteristics, during task decomposition, it is possible to ensure that the decomposed subtasks match the characteristics of the platform according to the platform type. For example, unmanned aerial vehicles are more suitable for performing aerial reconnaissance tasks, while unmanned vehicles are more suitable for ground patrol or material transportation. In addition, each unmanned platform has its specific performance parameters and limiting conditions, such as flight speed, endurance time, and load capacity, etc. For example, for tasks that require carrying materials or long-duration flights, during task decomposition, it can be ensured that the allocation of subtasks does not exceed the performance range or limiting conditions of the platform.

[0068] Furthermore, it is also possible to perform task decomposition according to the specific requirements of the task (such as time limit, accuracy requirements, etc.) and constraint conditions (such as terrain, weather, etc.), so as to generate a series of more refined, adaptable, and specific-condition-satisfying subtasks. These subtasks can be simple action instructions (such as takeoff, landing, moving to a specified position, etc.), or complex task combinations (such as performing reconnaissance, material delivery, target tracking, etc.).

[0069] After the task decomposition is completed, these subtasks can be serial tasks arranged in chronological order, or parallel tasks that can be performed simultaneously. For serial tasks that need to be executed in a specific order, the serial execution order of the subtasks can be set and executed sequentially. For tasks that can be processed in parallel, multiple unmanned platforms can be allowed to execute different subtasks simultaneously to improve the execution efficiency of the overall task.

[0070] It can be understood that through reasonable task decomposition and effective control execution strategies, it can be ensured that the unmanned platform can efficiently complete the task and improve the execution efficiency and quality of the overall task.

[0071] In the embodiments of the present disclosure, it further includes: performing basic plotting operations on the map interface that integrates the platform information, the execution status, and the perception information to draw corresponding identification information; performing perspective analysis on the map interface to determine the line-of-sight obstruction situation between any two points on the map interface; and adjusting the task according to the identification information and the line-of-sight situation.

[0072] On the integrated map interface that combines platform information, execution status, and perception information, basic plotting operations such as points, lines, and polygons can be performed to draw clear and accurate identification information, which includes but is not limited to target locations, action routes, safe areas, and obstacle distributions, thus meeting the basic needs of most mission planning. At the same time, through advanced line-of-sight analysis and profile analysis functions, a deeper understanding of the map interface can be achieved. Among them, line-of-sight analysis can determine the line-of-sight obstruction between any two points on the map to evaluate the line-of-sight range of the unmanned platform, determine the location of communication relay points, and plan obstacle avoidance routes, etc. Profile analysis can display key information such as terrain undulations and obstacle heights on the map interface, so as to more intuitively understand the impact of terrain conditions on mission execution.

[0073] After obtaining the identification information and line-of-sight conditions, the mission can be adjusted in a timely manner based on this information. For example, if it is found that the line of sight on a certain action route is severely obstructed, the route can be re-planned to avoid unnecessary difficulties for the unmanned platform during mission execution. Similarly, if a certain target location is outside the safe area, the user can also adjust the target location accordingly to ensure the safety of the unmanned platform.

[0074] In addition, various common data formats related to unmanned platforms can be loaded and displayed, including image data, point cloud data, and 3D model data, etc., which helps users to more comprehensively understand the mission environment and provides richer information support for mission planning. For example, by loading image data, users can intuitively see the terrain and building distributions in the mission area; by loading point cloud data, users can obtain detailed information such as terrain undulations and obstacle distributions in the mission area; and loading 3D model data enables users to understand the mission environment in a more three-dimensional way and provides a more intuitive reference for mission planning.

[0075] It can be understood that by providing basic and advanced plotting functions and supporting the loading and display of multiple data formats, a comprehensive, flexible, and easy-to-operate mission planning and execution platform is provided for users.

[0076] In some exemplary embodiments, it further includes: obtaining the warning information of the unmanned platform and mission payload based on the perception information, and adjusting the mission planning of the heterogeneous unmanned platform according to the warning information.

[0077] By performing real-time analysis and processing on the perceived information, such as environmental data collected by sensors, motion state data of the unmanned platform, and working state data of the mission payload, etc., abnormal situations of the unmanned platform and the mission payload can be detected in a timely manner, and corresponding alarm information can be returned. The alarm information can include alarm type, alarm level, alarm time, alarm location (such as longitude and latitude coordinates), alarm reason, and recommended countermeasures, etc. Among them, the alarm reason includes but is not limited to power system failure of the unmanned platform, navigation system abnormality, communication link interruption, mission payload damage or performance degradation, etc.

[0078] After receiving the alarm information, the corresponding processing mechanism can be triggered immediately. For example, alarm notifications can be sent to users in multiple ways such as audible, visual, and electronic alarms, text messages, and emails to ensure that users can timely understand the abnormal situations of the unmanned platform and the mission payload. In addition, according to the type and severity of the alarm information, the task planning of the heterogeneous unmanned platform can be automatically or recommended for the user to adjust. For example, if a power system failure of the unmanned platform is detected, it can be recommended that the user return the unmanned platform to the base for repair or adjust the mission route to avoid further damage to the unmanned platform. For some emergency situations, such as the unmanned platform getting out of control or the mission payload being severely damaged, the emergency processing mechanism can be immediately started, including emergency shutdown, remotely taking over the control right of the unmanned platform, etc.

[0079] It can be understood that by obtaining the alarm information and adjusting the task planning of the heterogeneous unmanned platform according to the alarm information, the safety and efficiency of the unmanned platform during the mission execution can be ensured.

[0080] Figure 4 The structural block diagram of the implementation device of the unified control system for heterogeneous unmanned platforms provided by the embodiments of the present disclosure is schematically shown.

[0081] As Figure 4 shown, the implementation device 400 of the unified control system for heterogeneous unmanned platforms in this embodiment includes an acquisition module 401, a planning module 402, a perception module 403, and a display module 404.

[0082] The acquisition module 401 is used to communicate with the second communication device of the heterogeneous unmanned platform through the first communication device based on a unified control protocol to obtain the platform information of each unmanned platform, and the first communication device is deployed on the control terminal.

[0083] The planning module 402 is used to perform the task planning of the heterogeneous unmanned platform based on the platform information of each of the unmanned platforms and track the execution status of the task.

[0084] The perception module 403 is used to monitor various mission payloads carried by the heterogeneous unmanned platform and obtain the perception information collected by the mission payload.

[0085] The display module 404 is configured to integrate the platform information, the execution status, and the perception information, and display them through a map interface.

[0086] It can be understood that the acquisition module 401, the planning module 402, the perception module 403, and the display module 404 can be implemented in one module, or any one of them can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 401, the planning module 402, the perception module 403, and the display module 404 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on a substrate, a system in a package, an application specific integrated circuit (ASIC), or can be implemented in any other reasonable way of integrating or packaging circuits, etc., in hardware or firmware, or implemented in an appropriate combination of software, hardware, and firmware. Alternatively, at least one of the acquisition module 401, the planning module 402, the perception module 403, and the display module 404 can be at least partially implemented as a computer program module, which can execute the functions of the corresponding module when the program is run on a computer.

[0087] Figure 5 A hardware structure diagram of an electronic device provided by an embodiment of the present disclosure is schematically shown.

[0088] As Figure 5 shown, the electronic device described in this embodiment includes: The electronic device 500 includes a processor 510 and a computer-readable storage medium 520. The electronic device 500 can execute the method described above with reference to Figure 2 to implement the detection of a specific operation.

[0089] Specifically, the processor 510 can include, for example, a general microprocessor, an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (e.g., an application specific integrated circuit (ASIC)), etc. The processor 510 can also include on-board memory for caching purposes. The processor 510 can be a single processing unit or multiple processing units for executing different actions of the method flow according to the embodiment of the present disclosure described with reference to Figure 2 above.

[0090] A computer-readable storage medium 520 can be, for example, any medium capable of containing, storing, transmitting, propagating, or transporting instructions. For example, the readable storage medium can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, components, or propagation media. Specific examples of the readable storage medium include: magnetic storage devices, such as magnetic tapes or hard disk drives (HDDs); optical storage devices, such as compact discs (CD-ROMs); memories, such as random access memories (RAMs) or flash memories; and / or wired / wireless communication links.

[0091] The computer-readable storage medium 520 can include a computer program 521, and the computer program 521 can include code / computer-executable instructions that, when executed by the processor 510, cause the processor 510 to execute, for example, the method flows and any variations thereof described above in connection with Figure 2 those described.

[0092] The computer program 521 can be configured to have computer program code that includes, for example, computer program modules. For example, in an exemplary embodiment, the code in the computer program 521 can include one or more program modules, such as module 521A, module 521B,.... It should be noted that the way of dividing the modules and the number of modules are not fixed, and those skilled in the art can use appropriate program modules or combinations of program modules according to the actual situation. When these combinations of program modules are executed by the processor 510, the processor 510 can execute, for example, the method flows and any variations thereof described above in connection with Figures 2 - 3 those described.

[0093] According to an embodiment of the present disclosure, at least one of the acquisition module 401, the planning module 402, the perception module 403, and the display module 404 can be implemented as a computer program module described with reference to Figure 5 those described, and when executed by the processor 510, can implement the corresponding operations described above.

[0094] The present disclosure also provides a computer-readable medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer-readable medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0095] Those skilled in the art can understand that the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0096] The specific embodiments described above further elaborate on the objectives, technical solutions, and beneficial effects of the present disclosure. It should be understood that the above are only specific embodiments of the present disclosure and are not used to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A method for implementing a unified control system for heterogeneous unmanned platforms, characterized in that: include: Communicate with a second communication device of a heterogeneous unmanned platform through a first communication device based on a unified control protocol to obtain platform information of each unmanned platform, wherein the first communication device is deployed on a control terminal; Based on the platform information of each of the unmanned platforms, perform task planning for the heterogeneous unmanned platforms and track the execution status of the tasks; Monitor various mission payloads carried by the heterogeneous unmanned platform and obtain perception information collected by the mission payloads; The platform information, the execution status and the perception information are integrated and displayed through a map interface.

2. The method for implementing a unified control system for heterogeneous unmanned platforms according to claim 1, characterized in that: The message frame of the control protocol adds at least one ID field on the basis of the general message frame, and the ID field is used to characterize and distinguish the properties of the heterogeneous unmanned platform.

3. The method for implementing a unified control system for heterogeneous unmanned platforms according to claim 1, characterized in that: Also includes: Constructing a platform list, wherein the platform list is used to display the platform information; Based on the platform information, query or filter the unmanned platforms in the platform list that meet preset conditions; and / or Based on the platform list, control instructions are sent to control each of the unmanned platforms.

4. The method for implementing a unified control system for heterogeneous unmanned platforms according to claim 1, characterized in that: The performing of mission planning of the heterogeneous unmanned platforms based on the platform information of each of the unmanned platforms includes: Determining the starting point and the end point of the task based on the platform information of each of the unmanned platforms; Manually mark the path on the map interface according to the starting point and end point of the task to obtain a planned path; or The environmental information of the unmanned platform is used to automatically plan a planned path from the starting point to the end point of the task.

5. The method for implementing a unified control system for heterogeneous unmanned platforms according to claim 1 or 4, characterized in that: The performing task planning of the heterogeneous unmanned platform based on the platform information of each of the unmanned platforms further includes: Based on the platform information of each of the unmanned platforms, decomposing the task of the unmanned platform to obtain at least two subtasks; The unmanned platform is controlled to execute the subtasks sequentially or in parallel until all the subtasks are completed.

6. The method for implementing a unified control system for heterogeneous unmanned platforms according to claim 1, characterized in that: Also includes: Based on the perception information, warning information of the unmanned platform and the mission payload is obtained, and according to the warning information, the mission planning of the heterogeneous unmanned platform is adjusted.

7. The method for implementing a unified control system for heterogeneous unmanned platforms according to claim 1, characterized in that: Also includes: Performing basic plotting operations on a map interface that integrates the platform information, the execution status, and the perception information to draw corresponding identification information; Performing perspective analysis on the map interface to determine the obstruction of sight between any two points on the map interface; The task is adjusted according to the identification information and the line of sight condition.

8. A device for realizing a unified control system for heterogeneous unmanned platforms, characterized in that: The device comprises: An acquisition module, configured to communicate with a second communication device of a heterogeneous unmanned platform through a first communication device based on a unified control protocol to acquire platform information of each unmanned platform, wherein the first communication device is deployed on a control terminal; A planning module, used to plan the tasks of the heterogeneous unmanned platforms based on the platform information of each of the unmanned platforms, and to track the execution status of the tasks; A perception module, used to monitor various mission payloads carried by the heterogeneous unmanned platform and obtain perception information collected by the mission payloads; The display module is used to integrate the platform information, the execution status and the perception information, and display them through a map interface.

9. An electronic device, comprising: one or more processors; a storage device for storing one or more computer programs, It is characterized in that when the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, characterized in that: When the instruction is executed by a processor, the processor executes the method according to any one of claims 1 to 7.

Images