Comprehensive situation display system and method of unmanned system management and control center

Abstract

The invention provides a comprehensive situation display system of an unmanned system management and control center, and the system comprises a general protocol access module which is used for accessing related data of an unmanned aerial vehicle and an unmanned vehicle which operate in a whole-space unmanned system into a situation display system based on an open type unmanned system access information format; the grid subdivision situation base is used for carrying out situation summarization on related information under a unified space-time framework to form a basis of the full-space unmanned system situation display system; the operation supervision module and the situation analysis module are used for performing whole-process and full-state monitoring management on the unmanned system in combination with the operation management and control requirements of the unmanned system, and providing situation support for operation management and control personnel in combination with a situation analysis means; the display module is used for displaying information sent by the operation supervision module and the situation analysis module; meanwhile, the invention also provides a comprehensive situation display method of the unmanned system management and control center.

Description

Technical Field

[0001] The present application belongs to the field of unmanned system command and control and operation management technology, and in particular, relates to a comprehensive situation display system and method for an unmanned system control center. Background Art

[0002] The full-space unmanned system operation control center is the center and brain of the full-space unmanned system operation, providing real-time and efficient airspace resource allocation, operation control scheduling and system operation monitoring for heterogeneous multi-domain unmanned equipment such as operating drones, unmanned vehicles, and unmanned ships.

[0003] Currently, there are some pilot applications of drones, unmanned vehicles and unmanned ships in their respective industries, but there is a lack of application scenarios for coordinated operation in the same area. How to meet the needs of operation managers to grasp the operation status under different operating conditions of sea, land and air in the same area is the basis for ensuring the safe operation of the full-space unmanned system. Summary of the invention

[0004] The purpose of the invention is to solve the problem of grasping the air-ground joint operation situation of the full-space unmanned system operation control center of the sea, land and air during the operation control process, and propose an air-ground integrated full-space unmanned system comprehensive situation display system, which solves the problems of unified information access of heterogeneous multi-domain unmanned platforms, air-ground integrated collaborative operation analysis, and full-element display of the full-space unmanned system, providing situation support for the collaborative operation of the full-space unmanned system.

[0005] In a first aspect, the present application provides a comprehensive situation display system for an unmanned system control center, the system comprising:

[0006] The universal protocol access module is used to access the relevant data of drones and unmanned vehicles running in the full-space unmanned system into the situation display system based on the open unmanned system access information format;

[0007] The grid-divided situation base is used to summarize relevant information in a unified space-time framework, forming the basis of the full-space unmanned system situation display system;

[0008] The operation supervision module and situation analysis module are used to monitor and manage the unmanned system in the whole process and in all states according to the operation control requirements of the unmanned system, and provide situation support for the operation control personnel by combining situation analysis methods;

[0009] The display module is used to display the information sent by the operation supervision module and the situation analysis module.

[0010] Preferably, the open unmanned system access information format includes:

[0011] Information architecture, which specifies the types of information that the unmanned system situation accesses;

[0012] The information definition method specifies the field length, field type, and field format of each type of information during the expression process.

[0013] Preferably, the information types of the unmanned system situation access include platform information, mission information, location information, status information, and load information.

[0014] Preferably, the gridded situation base is obtained by the following steps:

[0015] Acquiring data, and generating a three-dimensional data model of a target area based on the data;

[0016] Gridding geographic space;

[0017] Spatially align and adapt the 3D data model, 2D map, and 2D water map, grid the ground coordinates and water surface coordinates, and fuse the 3D data model with the 2D map according to the grid size and longitude and latitude values ​​to achieve unified coding;

[0018] Grid coding technology is used to give every inch of space in the pilot area a unique spatial coding identifier that can be calculated and easily retrieved.

[0019] Preferably, the operation supervision module is also used for full-task and full-state situation monitoring.

[0020] Preferably, the full-mission, full-state situation monitoring includes airspace / regional / water area supervision and drone / vehicle / ship monitoring.

[0021] Preferably, the situation analysis module is also used for full-cycle situation statistical analysis, specifically including: unmanned system statistics, equipment information display, planned task display, historical task display, and task statistics.

[0022] In a second aspect, the present application also provides a comprehensive situation display method for an unmanned system control center, the method comprising:

[0023] The general protocol access module is based on the open unmanned system access information format, and connects the relevant data of drones and unmanned vehicles running in the full-space unmanned system to the situation display system;

[0024] The grid-divided situation base summarizes relevant information in a unified space-time framework, forming the basis of the full-space unmanned system situation display system;

[0025] The operation supervision module and situation analysis module are combined with the unmanned system operation control requirements to monitor and manage the unmanned system in the entire process and in all states, and provide situation support for the operation control personnel by combining situation analysis methods;

[0026] The display module displays the information sent by the operation supervision module and the situation analysis module.

[0027] Beneficial technical effects of the present invention:

[0028] (1) Based on the air-ground integrated grid partitioning technology, a unified situation time benchmark is constructed to solve the different requirements for operation planning maps brought about by different unmanned platforms such as drones and unmanned vehicles due to their different working mechanisms and operation modes. The operation of unmanned systems is uniformly displayed under the same space-time framework to ensure the orderly operation of various drone platforms;

[0029] (2) An open unmanned system access information format was designed to meet control needs. This solved the problem of different drone and unmanned vehicle manufacturers being unable to access each other due to the inconsistency of information formats and standard protocols, and achieved unified access to all unmanned vehicles operating in the full-space unmanned system.

[0030] (3) Aiming at the operational control needs of fully controlled unmanned systems, we will build full-state and full-mission process situation control capabilities to achieve full-state supervision of unmanned system mission conditions, operating parameters, operating routes, and equipment status information;

[0031] (4) An unmanned system operation status analysis module was designed. According to the unmanned system operation control requirements, the module uses big data technology to achieve unified analysis and display of data such as operating hot spots and airspace usage during the operation of the unmanned system. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A diagram showing the composition of a comprehensive situation display system for an unmanned system control center provided in an embodiment of the present application;

[0033] Figure 2 A schematic diagram of grid division provided in an embodiment of the present application;

[0034] Figure 3 A schematic diagram of the information protocol architecture provided for an embodiment of the present application;

[0035] Figure 4 A diagram of the full-task, full-state situation monitoring functionality provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] It should be noted that the present application belongs to the field of command and control and operation management of unmanned systems. It mainly solves the problem of grasping the operation status of heterogeneous multi-domain unmanned systems such as drones and unmanned vehicles during their operation in the same area, and provides management support for operation management personnel. Specifically, it is an air-ground integrated full-space unmanned system comprehensive situation display system.

[0037] See also Figure 1 - Figure 4 The integrated air-ground, full-space unmanned system comprehensive situation display system constructed by this system realizes for the first time the unified access, full-element display, and full-process monitoring of heterogeneous multi-domain unmanned systems such as drones and unmanned vehicles under a unified time and space reference framework, providing an integrated means of situation grasp for the operation and maintenance personnel of the control center.

[0038] The application provides a comprehensive situation system for unmanned system control center to meet the operation and management needs of unmanned system in the future national construction of sea, land and air full-space unmanned system operation and control process. It is necessary to coordinate the operation of low-altitude airspace resources and ground road network resources under the same control framework and integrate the operation situation of unmanned systems. Based on the regional two-dimensional and three-dimensional maps, a standard air-ground integrated access protocol is designed to solve the unified access problem of heterogeneous multi-domain unmanned systems.

[0039] By adopting grid subdivision technology, cross-domain unmanned systems such as drones and unmanned vehicles are displayed in a unified time and space benchmark, and support is provided for operation analysis based on regional grids, providing management and control personnel with unified data analysis capabilities. Finally, a full-space and full-factor display platform integrating mission planning, equipment status, load status, and supporting equipment is constructed.

[0040] This invention solves the problem of unified access and unified representation of heterogeneous multi-domain unmanned systems such as multiple UAV platforms and multiple unmanned vehicle platforms in the full-space unmanned system, as well as the full-element display of the full-space unmanned system, providing intuitive and comprehensive decision-making information for the operation and control of the unmanned system.

[0041] The system consists of

[0042] The full-space unmanned system situation display system mainly includes a grid-divided situation base, a general protocol access module, a situation analysis module, an operation supervision module and a display module. The system consists of Figure 1 shown.

[0043] The general protocol access module is based on the open unmanned system access information format, and connects the relevant data of drones and unmanned vehicles running in the full-space unmanned system to the situation display system; based on the grid-divided situation base, the relevant information is summarized in a unified time and space framework, forming the basis of the full-space unmanned system situation display system; the operation supervision module and the situation analysis module are combined with the unmanned system operation control requirements. The unmanned system is monitored and managed throughout the entire process and in all states, and situation analysis methods are used to provide situation support for operation control personnel, and finally displayed uniformly through the display module.

[0044] Among them, the grid division situation base

[0045] 1) Data preparation

[0046] Through the use of drone oblique photography technology, 1:500 high-precision drone DEM data, 0.1-meter drone orthophoto data, 3cm oblique 3D model data, overall planning data, building data, park component facility data, basic topographic map data, and POI data of the target area are collected, and general 3D modeling technology is used to generate a high-precision 3D model of the target area.

[0047] 2) Grid division

[0048] The geographic space is gridded, and the division rules are as follows:

[0049] ① The first layer: According to the latitude and longitude grid of the target area, grid division is carried out in the longitude and latitude directions. The target area is divided into grids, and each grid is divided into 5.394′x 5.394′ grids (about 10Km x10Km). For irregular edge areas, expansion and filling are adopted. Numbering rules: Increasing from 00 in the direction of longitude from small to large, increasing from 00 in the direction of latitude from small to large, and increasing from 00 in the direction of height from small to large according to 10Km.

[0050] ② The second layer: Divide each horizontal grid of the first layer into 0.5394′x 0.5394′ grids (about 1Km x1Km). The total number of grids is 1000. The numbering rules are as follows: they increase from 0 to the top in the longitude direction, from 0 to the top in the latitude direction, and from 0 to the top in the altitude direction of 10Km.

[0051] ③The third layer: Divide each horizontal grid of the second layer into 0.05394′x 0.05394′ grids (about 100m x100m). The total number of grids is 1000. The numbering rules are as follows: they increase from 0 to the top in the longitude direction, from 0 to the top in the latitude direction, and from 0 to the top in the height direction of 100m.

[0052] ④ The fourth layer: Divide each horizontal grid of the third layer into 0.005394′x 0.005394′ grids (about 10x 10m). The total number of grids is 1000. The numbering rules are as follows: they increase from 0 to the top in the longitude direction, from 0 to the top in the latitude direction, and from 0 to the top in the height direction of 10m.

[0053] ⑤Fifth layer: Divide each horizontal grid of the fourth layer into 0.0010788′x 0.0010788′ grids (about 2m x2m). The total number of grids is 125. The numbering rule is: increase from 0 to the top in the longitude direction, increase from 0 to the top in the latitude direction, and increase from 0 to the top in the height direction of 2m from the small to the large direction.

[0054] 3) 2D map / water map fusion

[0055] Spatially align and adapt the three-dimensional data model, two-dimensional map, and two-dimensional water map, grid the ground coordinates and water surface coordinates, and merge the three-dimensional model and two-dimensional map according to the grid size and longitude and latitude values ​​to achieve unified coding.

[0056] 4) Data Governance

[0057] Grid coding technology is used to give every inch of space in the pilot area a unique spatial coding identification that is calculable and easy to retrieve. This identification system has the characteristics of multi-scale, unique, three-dimensional, and full-time and space, and realizes the refined management of construction elements. Using the spatiotemporal big database technology, relying on the data organization method based on grid space and vector layer, combined with the construction and implementation dimensions of various systems in the park, a unified spatiotemporal data storage service engine is established to assign spatiotemporal attributes for planning and construction projects, and complete the spatiotemporal integration of multi-source heterogeneous data.

[0058] Among them, the open unmanned system access information format for control needs

[0059] The heterogeneous and diverse unmanned equipment such as drones, unmanned vehicles, and unmanned ships operating in the full-space unmanned system come from different manufacturers and production units. The information formats and transmission protocols of the unmanned systems produced and developed by each unit are different. If various information protocols are expressed in a unified way, it will bring huge challenges to future operation control. The control center needs to develop a large number of information protocol parsing functions to ensure the access of different manufacturers. Therefore, the open unmanned system access information format for control needs is based on the unmanned system control needs, and the information architecture and general representation mode are designed to provide information access basis for operation status display.

[0060] The open unmanned system access information format for management and control needs is a set of communication protocol formats designed according to operational management and control needs, for the access of unmanned systems such as drones and unmanned vehicles operating within the management and control system. It enables each node of the drone system to provide the required information accurately, efficiently and precisely in accordance with unified standards and specifications, thereby providing an information basis for situation display.

[0061] Information architecture: It specifies the types of information required for unmanned system situation access, including platform information, mission information, location information, status information, and payload information. It also clarifies the data necessary for unmanned system management and control, including the platform's own basic data, mission execution data, location information, operating status data, and on-board optoelectronic payload data.

[0062] The information definition method stipulates the basic information such as field length, type, format, etc. of various types of information during the expression process. Taking the information status of the unmanned system platform as an example, the relevant information definition is shown in Table 1.

[0063] Table 1 Unmanned system status information representation

[0064]

[0065] All types of unmanned systems within the management and control system follow the common information format and connect information to the management and control system in accordance with unified standards, providing information support for ensuring the operation and control of the system.

[0066] Among them, full-task, full-state situation monitoring

[0067] Full-mission, full-state situation monitoring includes airspace / region / water supervision, drone / vehicle / ship monitoring. Based on the basic airspace map, it dynamically displays the current drone situation in the airspace (normal drone number, drone type, number of drones below 120 meters and within 120 meters and 300 meters, drone mission types such as logistics cruise, etc.), unmanned flying car situation, conflict and warning situation, airspace usage (comparison between declaration and actual flight), and historical route display.

[0068] The full-task, full-state situation monitoring function consists of Figure 4 shown.

[0069] The software interface has four panels: Statistics, Equipment, Plan, and History. Equipment and Plan are lists. Click an item in the list to locate the device on the map and display the real-time information of the device.

[0070] The history panel is a list of historical tasks. Click on an item to enter the track playback of the task, and display the real-time information at that time. On the right is the return video panel. No operation is required. It displays part of the return video and refreshes automatically every minute. Next to the video panel is the map operation tool, which can be used to control the map. The weather effect is real-time weather. On the left side of the device perspective, you can search for equipment or hangars based on keywords and types. After clicking on a device or hangar, the real-time video of the device or hangar can be played on the right, and the item being played will be marked.

[0071] Among them, full cycle situation analysis

[0072] The full-cycle situation statistical analysis includes: unmanned system statistics, equipment information display, planned task display, historical task display, and task display.

[0073] 1) Statistics of unmanned systems

[0074] Unmanned system statistics include airspace / region / water statistics, route statistics, and equipment statistics. Charts are used to display the number and type of airspace / region / water areas, and the number and proportion of routes are counted by type. The types and numbers of drones / vehicles / ships are also displayed, and visual statistical analysis is performed at different height dimensions.

[0075] 2) Equipment information display

[0076] The equipment information display includes equipment list display and hangar list display. The equipment information display includes a map showing the real-time location of the equipment of each manufacturer connected to the system. The display content includes:

[0077] a) Access device data

[0078] b) Equipment type: various drones, unmanned vehicles, and unmanned ships;

[0079] c) Status: Online or offline viewing

[0080] d) Click on a flying device to switch the page view to that device's view, showing the device's real-time trajectory, and basic information such as device name, current mission, location, altitude, speed, status, and returned video.

[0081] Hangar list display shows the hangar list of different types of locomotives / vehicles / ships from different manufacturers.

[0082] 3) Planned task display

[0083] Scheduled task display includes scheduled task query and running task tracking.

[0084] The main contents of the planned task display include: 1) Supporting the display of equipment according to the task dimension, and divided into two panels: planned tasks and historical records, supporting retrieval by task name and task type; 2) Displaying a list of tasks in cruise and standby status, and arranging them in order of the planned start time.

[0085] The main contents of running mission tracking include: 1) Click on a cruise mission, the view will switch to the equipment executing the mission, and a pop-up window will display the current mission and the details of the equipment executing the mission; 2) Click on a pending mission, and a pop-up window will prompt: the mission has not yet started!

[0086] 4) Historical mission display

[0087] Historical task display includes historical task query and historical video playback.

[0088] The historical task query displays the completed historical tasks, arranged in order of actual completion time, including the current day, the past 7 days, and the past 30 days, and includes relationships; the historical video playback displays the task route trajectory as well as the synchronization parameters and returned video by clicking.

[0089] 5) Task Statistics

[0090] Task statistics include statistical analysis of the number of trips, mileage, and execution times, displayed in the form of a bar chart.

Claims

1. A comprehensive situation display system for an unmanned system control center, characterized in that: The system comprises: The universal protocol access module is used to access the relevant data of drones and unmanned vehicles running in the full-space unmanned system into the situation display system based on the open unmanned system access information format; The grid-divided situation base is used to summarize relevant information in a unified space-time framework, forming the basis of the full-space unmanned system situation display system; The operation supervision module and situation analysis module are used to monitor and manage the unmanned system in the whole process and in all states according to the operation control requirements of the unmanned system, and provide situation support for the operation control personnel by combining situation analysis methods; The display module is used to display the information sent by the operation supervision module and the situation analysis module.

2. The system according to claim 1, characterized in that The open unmanned system access information format includes: Information architecture, which specifies the types of information that the unmanned system situation accesses; The information definition method specifies the field length, field type, and field format of each type of information during the expression process.

3. The system according to claim 2, characterized in that The information types of the unmanned system situation access include platform information, mission information, location information, status information, and payload information.

4. The system according to claim 1, characterized in that The gridded situation base is obtained by the following steps: Acquiring data, and generating a three-dimensional data model of a target area based on the data; Gridding geographic space; Spatially align and adapt the 3D data model, 2D map, and 2D water map, grid the ground coordinates and water surface coordinates, and fuse the 3D data model with the 2D map according to the grid size and longitude and latitude values ​​to achieve unified coding; Grid coding technology is used to give every inch of space in the pilot area a unique spatial coding identifier that can be calculated and easily retrieved.

5. The system according to claim 1, characterized in that The operation supervision module is also used for full-task and full-state situation monitoring.

6. The system according to claim 5, characterized in that The full-mission, full-state situation monitoring includes airspace / regional / water area supervision and drone / vehicle / ship monitoring.

7. The system according to claim 6, characterized in that The situation analysis module is also used for full-cycle situation statistical analysis, specifically including: unmanned system statistics, equipment information display, planned task display, historical task display, and task statistics.

8. A comprehensive situation display method for an unmanned system control center, characterized in that: The method comprises: The general protocol access module is based on the open unmanned system access information format, and connects the relevant data of drones and unmanned vehicles running in the full-space unmanned system to the situation display system; The grid-divided situation base aggregates relevant information in a unified space-time framework, forming the basis of the full-space unmanned system situation display system; The operation supervision module and situation analysis module are combined with the unmanned system operation control requirements to monitor and manage the unmanned system in the entire process and in all states, and provide situation support for the operation control personnel by combining situation analysis methods; The display module displays the information sent by the operation supervision module and the situation analysis module.

Images