An Augmented Reality-Based Unmanned Aerial Vehicle (UAV) Inspection Platform and Method for Airport Airspace Clearance Zones

Abstract

This invention relates to the field of unmanned aerial vehicle (UAV) inspection technology, specifically to an augmented reality-based UAV inspection platform and method for airport airspace. The platform comprises interconnected server modules, a GIS engine module, a UAV module, a flight control and interaction module, a data interface module, a business management module, and a UAV application module. By utilizing augmented reality technology to provide intuitive information overlay and precise guidance for UAV inspections, it achieves efficient and comprehensive inspection of airport airspace and possesses the ability to proactively handle intrusion targets such as unauthorized UAVs. This effectively solves the problems of low efficiency and incomplete coverage in traditional manual inspections, insufficient accuracy and poor real-time performance of remote sensing monitoring methods, high costs of long-term periodic updates, and passive defense against intrusion targets such as unauthorized UAVs.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) inspection technology, and in particular to an augmented reality-based UAV inspection platform and method for airport airspace. Background Technology

[0002] The rapid recovery of my country's civil aviation industry and the continuous expansion of urban construction have placed increasing pressure on airspace management at civil airports. Currently, most airports still rely primarily on manual inspections for airspace management. This method is not only inefficient (obstacles are usually scattered, and most of the time is wasted on the road, leaving very little time for effective inspections), but its effectiveness is also questioned by both the Civil Aviation Administration of China (CAAC) and airport authorities. The main reasons for the criticism are: it cannot cover areas inaccessible to personnel; it is difficult to spot illegal structures at the top of obstacles and small protrusions far from the roadside when looking up from the ground (illegal structures are the main cause of abnormally high obstacles); and a leader of an airport's airspace management department also reported that due to insufficient manpower, even with the participation of leaders in inspections, it is difficult to meet the inspection frequency recommended by the CAAC, making inspections a significant burden.

[0003] Some large international airports have established airspace management systems based on satellite remote sensing imagery, including our company's previous generation product, the "Airport Airspace Management System." This system uses processed satellite remote sensing data such as orthophoto maps (DOM), digital elevation models (DEM), and digital surface models (DSM) combined with restriction surface models and a 3D GIS engine to monitor the elevation of obstacles in the airspace. This method allows users to have a comprehensive understanding of the obstacles in the airspace, observe areas that cannot be covered by manual inspections, and discover illegal rooftop constructions that are difficult to detect by manual inspections. However, it also has problems such as the inability to detect small objects, the elevation monitoring accuracy not meeting the 0.5-meter level elevation measurement accuracy required by the authorities within the airspace restriction surface in Annex 14 (civilian-grade satellite remote sensing data elevation accuracy is usually only about 3 meters), high single-phase cost, and difficulty in achieving the monthly update frequency required by the authorities. Therefore, even with the establishment of such an airspace management system, it cannot completely replace traditional manual inspections.

[0004] In addition, airspace management departments still rely heavily on visual inspection to detect unauthorized drones and airborne objects. When unauthorized drones are discovered, they can only call the police and search for the pilots. In most cases, the pilots cannot be found. Moreover, because they lack law enforcement authority, even if the pilots are found, they cannot take coercive measures and can only hand them over to the airport police. When airborne objects are discovered, they can only be reported, and they cannot quickly eliminate the adverse effects. In terms of operations, they are relatively passive and lack effective means.

[0005] In response to the above difficulties and pain points, and based on existing implementation cases of drone inspection solutions for power transmission lines based on augmented reality (AR) technology, this patent proposes a design scheme for a drone inspection platform for airport airspace based on augmented reality (AR) technology. Summary of the Invention

[0006] The purpose of this invention is to provide an augmented reality-based unmanned aerial vehicle (UAV) inspection platform and method for airport airspace, which solves the problems of low efficiency, incomplete coverage, and inability to effectively handle unauthorized UAVs and airborne objects in existing airport airspace inspections, as well as the problems of insufficient accuracy, low update frequency, and high cost of traditional remote sensing monitoring methods.

[0007] To achieve the above objectives, the present invention provides an augmented reality-based unmanned aerial vehicle (UAV) inspection platform for airport airspace, comprising a server module, a GIS engine module, a UAV module, a flight control and interaction module, a data interface module, a business management module, and a UAV application module. The data interface module is connected to the server module, the server module is connected to the GIS engine module, the business management module, and the flight control and interaction module, and the flight control and interaction module is connected to the UAV application module and the UAV module. The server module is used to store and manage airport basic parameters, airspace restriction surface models, obstacle data, UAV and personnel files, and to perform fusion processing of multi-source detection data, and distribute the processed data to the business management module and the flight control and interaction module. The GIS engine module is used to construct and render a three-dimensional scene of the airport airspace based on geographic information data. The three-dimensional scene includes terrain, images, restriction surfaces and obstacle models. The data interface module is used to access external data sources such as air traffic control systems, ADS-B automatic dependent surveillance broadcast systems, and UOM unmanned aerial vehicle cloud systems; The business management module is used to provide a human-computer interaction interface for three-dimensional scenes, realize the creation, dispatch and monitoring of patrol tasks, and manage obstacle, construction project and personnel and equipment files; The flight control and interaction module is used to control the flight of the drone and generate an augmented reality interactive interface based on the data received from the server module. The drone application module is used to realize augmented reality-based inspection check-in, electronic fence alarm for no-fly zones, flight collision avoidance warning, and interception and guidance of non-cooperative drones; The drone module is used to perform aerial flight missions and is equipped with sensing devices to collect images and location data.

[0008] The drone module includes a multi-rotor drone and a vertical take-off and landing fixed-wing drone; The multi-rotor drone is used to perform patrol, interception, and bird control tasks within the airport's red line area; The vertical takeoff and landing fixed-wing UAV is used to perform patrol missions in the long-distance airspace outside the airport.

[0009] The augmented reality-based airport airspace drone inspection platform also includes a detection and perception module, which is connected to the server module. The detection and sensing module is used to detect low-altitude targets within the airspace using radar, optoelectronic equipment, and radio monitoring equipment, and to send the detection data to the server module for fusion processing.

[0010] The augmented reality interactive interface generated by the flight control and interaction module is used to overlay and display obstacle labels, airspace restriction surfaces, electronic fences for no-fly zones, suggested patrol routes, and real-time detected low-altitude target cursors.

[0011] The platform also includes a handheld terminal module, which is connected to the server module. The handheld terminal module is used to provide a two-dimensional electronic map interface to realize on-site data entry, height limit calculation and manual inspection task management.

[0012] On the other hand, the present invention also includes an augmented reality-based unmanned aerial vehicle (UAV) inspection method for airport airspace, comprising the following steps: Construct a three-dimensional scene of the airport's airspace, which includes terrain, images, airspace restriction surfaces, and obstacle models; Access to external data sources such as air traffic control systems, ADS-B automatic dependent surveillance broadcast systems, and UOM unmanned aerial vehicle cloud systems; Low-altitude targets within the airspace are detected using radar, optoelectronic equipment, and radio monitoring equipment. The detected data is then fused with external data to generate real-time situational information about the airspace. Create and assign patrol tasks, which include patrol routes, mandatory obstacles to be patrolled, and crew information; During the drone's flight, an augmented reality interactive interface is generated and displayed based on the mission information and real-time situational information received from the server. Through an augmented reality interactive interface, it performs augmented reality-based inspection check-in, no-fly zone electronic fence alarms, flight collision avoidance warnings, and interception and guidance of non-cooperative drones.

[0013] The specific steps for performing augmented reality-based inspection check-in include: Establish a visual space model based on the parameters of the drone camera; When a target obstacle enters the visible space model and remains within the preset time, the inspection and check-in of the obstacle is automatically completed.

[0014] The present invention provides an augmented reality-based unmanned aerial vehicle (UAV) inspection platform and method for airport airspace, which has the following technical advantages and effects: 1. The business functions have a wider coverage, which can change the current situation where manual inspections are the main method, the airspace management department needs to spend a lot of manpower and time on inspections, and it is impossible to intercept illegal drones and airborne objects to quickly eliminate their impact.

[0015] 2. Patrols from a high-altitude perspective can achieve comprehensive coverage of the airspace inspection area without blind spots, easily detect illegal rooftop constructions, and identify rooftop lightning rods by magnifying observation with an airborne high-definition camera equipped with optical zoom and digital zoom capabilities.

[0016] 3. High inspection efficiency: Drones fly at high altitudes unaffected by ground traffic conditions, offering a very wide field of view. With the help of AR display indicators and positioning, they can quickly locate target obstacles requiring inspection. Combined with the "gaze check-in" function, the inspection status of obstacles can be quickly checked off, even simultaneously checking off multiple obstacles within the field of view. With clickable interactive AR tags, the inspection crew can quickly record inspection information. Furthermore, the drone's flight speed is significantly faster than that of a car, greatly reducing the time required for airspace inspections. If the drone flies close to the airspace restriction surface displayed by AR, it can easily detect any excessively tall obstacles in the surrounding area. If the drone's autopilot function is activated, the pilot can also assist in the inspection, further improving efficiency.

[0017] 4. The platform integrates ground detection equipment and, combined with AR display guidance, enables drones to intercept intrusive objects that are currently uninterceptable, such as unauthorized drones and airborne debris. (Intercepting these objects near airports requires consideration of safety issues. Drones equipped with capture nets are a safe interception method, but require precise guidance and operation. Pilots often find it extremely difficult to spot distant drones or objects the size of sky lanterns using their own drone's image transmission; therefore, it is necessary to combine positioning data from other detection systems and accurately indicate the target's location and distance in the pilot's interface to achieve interception.) It can also drive away high-altitude birds that are currently unmanageable. (Actual research shows that airports' methods for dealing with unauthorized drones are limited to issuing alarms and capturing pilots.) 5. Based on the platform's strong integrability, we can further explore integrating bird control systems into this platform to expand bird-related services, establish a bird database, integrate insect detection data and ecological environment survey data, and automatically control bird deterrence equipment using the detection data provided by the platform's detection equipment. We can also add automatic collection and identification functions for ground vegetation and vegetation types (relying on photoelectric equipment) to two types of patrol drones, and add aerial ecological environment collection tasks to form a complete and comprehensive airport low-altitude safety protection solution. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0019] Figure 1 This is a schematic diagram of the structure of the augmented reality-based unmanned aerial vehicle (UAV) inspection platform for airport airspace of the present invention.

[0020] Figure 2 This is a flowchart of the augmented reality-based unmanned aerial vehicle (UAV) inspection method for airport airspace of the present invention.

[0021] Figure 3 This is a schematic diagram of the data source for the three-dimensional visualization scene of the clearance in this invention.

[0022] Figure 4 The full lifecycle operation process of different types of patrol tasks of this invention is shown in the figure.

[0023] Figure 5 This is a schematic diagram of the platform architecture of the present invention.

[0024] In the diagram: 101-Server module, 102-GIS engine module, 103-UAV module, 104-Flight control and interaction module, 105-Data interface module, 106-Business management module, 107-UAV application module, 108-Detection and perception module, 109-Handheld terminal module, 110-Multi-rotor UAV, 111-Vertical take-off and landing fixed-wing UAV. Detailed Implementation

[0025] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0026] Please see Figures 1 to 5 ,in Figure 1 This is a structural diagram of an augmented reality-based drone inspection platform for airport airspace. Figure 2 This is a flowchart of an augmented reality-based drone inspection method for airport airspace clearance zones. Figure 3This is a schematic diagram illustrating the data source for a 3D visualization scene of the airspace. Figure 4 The diagram shows the full lifecycle operation process for different types of patrol tasks. Figure 5 This is a schematic diagram of the platform architecture.

[0027] This invention provides an augmented reality-based unmanned aerial vehicle (UAV) inspection platform and method for airport airspace, comprising a server module 101, a GIS engine module 102, a UAV module 103, a flight control and interaction module 104, a data interface module 105, a business management module 106, and a UAV application module 107. The data interface module 105 is connected to the server module 101, the server module 101 is connected to the GIS engine module 102, the business management module 106, and the flight control and interaction module 104, and the flight control and interaction module 104 is connected to the UAV application module 107 and the UAV module 103. The server module 101 is used to store and manage airport basic parameters, airspace restriction surface model, obstacle data, UAV and personnel files, and to perform fusion processing of multi-source detection data, and distribute the processed data to the business management module 106 and the flight control and interaction module 104. The GIS engine module 102 is used to construct and render a three-dimensional scene of the airport airspace based on geographic information data. The three-dimensional scene includes terrain, images, restriction surfaces and obstacle models. The data interface module 105 is used to access external data sources such as air traffic control systems, ADS-B automatic dependent surveillance broadcast systems, and UOM unmanned aerial vehicle cloud systems. The business management module 106 is used to provide a human-computer interaction interface for three-dimensional scenes, realize the creation, dispatch and monitoring of patrol tasks, and manage obstacle, construction project and personnel and equipment files; The flight control and interaction module 104 is used to control the flight of the drone and generate an augmented reality interactive interface based on the data received from the server module 101. The drone application module 107 is used to realize augmented reality-based inspection check-in, electronic fence alarm for no-fly zones, flight collision avoidance warning, and interception and guidance of non-cooperative drones. The unmanned aerial vehicle module 103 is used to perform aerial flight missions and is equipped with sensing devices to collect images and location data.

[0028] Specifically, the server module 101 is used to collect, summarize, process, forward, and store data. All detection data must first be summarized to the platform server and then processed on the platform server. It can render the 3D scene of the airport airspace and various models in the scene, and at the same time provide B / S mode access for no less than 50 concurrent users. It supports all functions of the platform and has efficient large-scale data processing capabilities and strong real-time performance.

[0029] The GIS engine module 102 is used to construct a 3D scene of the airport airspace, including elements such as DOM, DEM, electronic map, airspace restriction surface model, UAV electronic fence, obstacle white model, and labels. The obstacle white model and labels support interaction. Users can access the 3D scene via PC or access the electronic map via mobile device. They can view manned aircraft and operating UAVs within the airspace and their flight paths. Clicking the UAV icon can view UAV information, as well as the flight crew and crew member information. Users can also view bird activity and real-time UAV detection status displayed as labels or icons overlaid in the 3D scene / electronic map.

[0030] The flight control and interaction module 104 is used to control the drone and simultaneously render the AR flight and monitoring interface. The vertical take-off and landing fixed-wing drone 111 used for field patrols requires at least one pilot and one observer to operate the rotatable high-definition zoom camera and record the patrol situation. It also integrates information such as the drone's current position, heading, motion parameters, and the direction and angle of the observation lens, enabling the AR displayed content to align with corresponding elements in the real world.

[0031] The data interface module 105 is used to interface with air traffic control systems, ADS-B, airport geographic information systems, and other airport systems. Because drone inspections in airport airspace involve operations in controlled airspace and dangerous areas within the red line, all drone activities must be subject to unified scheduling by air traffic control units. After accessing ADS-B message data, relying on the platform's computing power, it can issue warnings to drones operating in risk areas when manned aircraft approach. Combined with AR tag displays and AR-predicted aircraft flight trajectories, it helps drone pilots avoid manned aircraft. It also connects to the UOM system to obtain operational data from cooperative drones around the airport; this data needs to be intelligently fused with detection data provided by detection equipment.

[0032] The business management module 106 can be accessed via a B / S mode platform using desktop or mobile workstations. Users include leaders and inspectors from the airport monitoring and management department. It utilizes a 3D GIS engine to construct a 3D scene, enabling 3D visualization of the airspace clearance zone. The module allows for adding, deleting, modifying, and querying information on obstacles and basic airport information within the airspace clearance zone. It can also create task templates and assign tasks to inspection personnel, monitor ongoing tasks and operational drones, view historical inspection logs and various data collected during inspections, and assess height restrictions at any location and altitude within the scene based on airspace limitations.

[0033] The UAV application module 107 consists of a ground station, related control and processing computers, and UAVs. It needs to be connected to air traffic control communication. The server module 101 provides the data required for all functions such as AR display, electronic fence, collision warning, and airspace inspection. All complex data processing procedures are completed on the platform server. This module is only responsible for performing necessary simple processing on the finished data to ensure the normal operation of each function.

[0034] In this embodiment, a 3D scene of the airspace is constructed through the GIS engine module 102, and multi-source data is acquired through the data interface module 105 and the detection and perception module 108. Data fusion processing is then completed in the server module 101. Management personnel formulate inspection plans and issue tasks through the business management module 106. During task execution, the flight control and interaction module 104 generates an augmented reality interface to provide pilots with intuitive flight guidance. The various functions of the UAV application module 107 work collaboratively during flight to achieve efficient and accurate airspace inspection. Through the collaborative work of the above modules, this invention realizes the digitalization, intelligentization, and visualization of airport airspace inspection, significantly improving inspection efficiency and safety.

[0035] The unmanned aerial vehicle (UAV) module 103 includes a multi-rotor UAV 110 and a vertical take-off and landing fixed-wing UAV 111. The multi-rotor UAV 110 is used to perform patrol, interception and bird control tasks within the airport's red line area; The vertical takeoff and landing fixed-wing UAV 111 is used to perform patrol missions in the long-distance airspace outside the airport.

[0036] In this embodiment, the multi-rotor UAV 110 is used to perform tasks such as patrolling within the airport's red line area and some adjacent airport areas, intercepting and countering unauthorized UAV flights, high-altitude bird control, and driving away intruders from perimeter areas. Typically, at least two or more UAVs are required. An RTK module needs to be installed. It needs to have near-range omnidirectional obstacle avoidance capabilities. The vertical takeoff and landing fixed-wing UAV 111 is used to perform long-distance field inspection tasks, such as monthly inspections, special inspections, and key area inspections of the airspace inspection area. The remote control and image transmission reception range must cover the entire airspace of concern and some surrounding areas. For safety, the range must be at least 55*3=165 kilometers or more. This type of UAV must be jointly controlled by one pilot and one observer. It is equipped with at least two sets of high-definition cameras, one set for normal flight by the pilot and the other set controlled by the observer. It is mounted on a PTZ camera platform and equipped with a gimbal. It has a full downward field of view under the fuselage and a partial upward field of view. It has higher resolution and 3x or more optical zoom + digital zoom capability. It can stably and clearly take pictures of obstacles at high altitudes. It is equipped with an RTK module and has close-range omnidirectional obstacle avoidance function.

[0037] Secondly, the augmented reality-based airport airspace drone inspection platform also includes a detection and perception module 108, which is connected to the server module 101. The detection and sensing module 108 is used to detect low-altitude targets in the airspace through radar, optoelectronic equipment and radio monitoring equipment, and send the detection data to the server module 101 for fusion processing.

[0038] In this embodiment, the detection and sensing module 108 includes a detection radar, a radio monitoring device, and an optoelectronic detection device. The detection radar is used to search for low-altitude targets over a wide area and determine their positions. The optoelectronic device is guided by this platform to identify and verify the targets and characterize them. The radio monitoring device can further determine whether the target is a drone and its detailed information and flight parameters. The detection data is directly transmitted to the server module 101 and, after integration, can be overlaid on the drone's operating interface in the form of an AR cursor to guide the working drone to perform countermeasures or bird scare.

[0039] Furthermore, this invention includes methods for acquiring low-altitude objects such as radar, optoelectronic devices, radio detection devices, and UOM system data access. However, the raw data acquired by these methods each have their own characteristics, and some objects can only be acquired through some of these methods. Details are as follows: radar Features: wide-range rapid scanning, long detection range, ability to locate targets, difficulty in identifying specific object types, high false alarm rate. The original detected target is located in polar coordinates of azimuth, distance, and altitude. The location of the detected target can also be converted into world coordinates of latitude, longitude, and elevation based on the radar installation position and angle.

[0040] Detectable targets include: drones (cannot identify whether they are cooperative drones), birds, and metal-coated airborne objects. After processing, the trajectory of the object can be plotted, and its speed and direction of motion can be obtained.

[0041] Undetectable targets: non-metallic airborne objects, and undetectable cooperative and non-cooperative drones.

[0042] Optoelectronic equipment Features: It includes detection and identification modes in both infrared and visible light, enabling it to identify specific object types and locate targets. It has a low false alarm rate, but its single-unit scanning rate and detection range are lower than radar. Therefore, it requires a larger number of devices deployed in conjunction with radar to compensate for these shortcomings. The typical approach is for radar to first detect the target and then guide the optoelectronic equipment for identification and verification. The monitoring images from the optoelectronic equipment can also be viewed manually by accessing the video data from specific devices.

[0043] Detectable targets: drones (cannot identify whether they are cooperative drones), birds, airborne objects, smoke and other visible targets.

[0044] Undetectable targets: Unable to distinguish between cooperative and non-cooperative drones.

[0045] Radio detection equipment Features: Real-time flight parameters are obtained by parsing the communication message protocol between the drone flight controller and the drone, and the data is completely authentic and reliable.

[0046] Detectable targets: UAVs with known flight control message protocols.

[0047] Undetectable targets: fiber optic communication drones, 4G / 5G communication drones, homemade drones, birds, and airborne objects; cannot accurately distinguish between cooperative and non-cooperative drones.

[0048] UOM system data Features: It receives real-time flight parameters reported by the drone flight control system backend, and the system stores all registration and licensing information of the drone, and can also check whether it has obtained authorization to fly in no-fly zones. The data is completely true and reliable.

[0049] Detectable target: Cooperative drones.

[0050] Undetectable targets: non-cooperative drones, birds, and airborne objects.

[0051] The detection data obtained through the above methods each have their own characteristics and missing parts. They need to be centrally fused and processed, then categorized according to UAVs, birds, and airborne objects. The display specifications and recording fields for data of the same type of target need to be unified, using the WGS-84 coordinate system and the 85 elevation three-dimensional world coordinate system. Simultaneously, false targets should be removed, and the same target detected by multiple devices should be merged. The fusion process is based on a comprehensive analysis and evaluation of the spatial location and motion characteristics of the detected targets submitted by all devices to identify any potentially overlapping targets. Suspected overlapping targets are then merged to minimize erroneous detections such as unmerged identical targets or indistinguishable, closely spaced targets. The fusion methods for different types of targets are as follows: drones Prioritize the union of UOM and radio detection equipment data and UAV flight data from this platform, using the S / N code as the primary key for merging. The data should primarily consist of these two types of UAVs. If a UAV that was not detected by UOM or radio detection equipment is detected through radar detection + radar-guided optoelectronic system target identification + optoelectronic system autonomous detection (identified as a UAV and cannot be merged with the detection results of these two methods), then this type of UAV is considered a non-cooperative UAV. If a non-cooperative UAV or an unauthorized cooperative UAV enters the warning zone, core zone, no-fly zone, or the area covered by Annex 14 of the "Guidelines for the Safety and Control of Unmanned Aerial Vehicles at Civil Transport Airports", it is considered an unauthorized UAV, and the system needs to trigger an alarm. The AR display label, the PC-side 3D scene, and the handheld terminal display label are all displayed in red and highlighted.

[0052] Birds and metal-coated airborne objects Metal-coated airborne objects are typically hydrogen balloons covered with aluminum foil. They are detected through a combination of large-area scanning by drone radar and identification / verification by optoelectronic devices. Their speed and direction of movement are calculated based on their trajectory. If the optoelectronic devices detect targets missed by radar, they can be displayed accordingly.

[0053] Non-metal-free coated airborne materials It can only be searched and located using photoelectric equipment, and its speed, direction of movement, and trajectory can be determined through real-time tracking.

[0054] In addition, the augmented reality interactive interface generated by the flight control and interaction module 104 is used to overlay and display obstacle labels, airspace restriction surfaces, electronic fences for no-fly zones, suggested patrol routes, and real-time detected low-altitude target cursors.

[0055] In this embodiment, both the vertical takeoff and landing fixed-wing UAV 111 and the multi-rotor UAV 110 adopt an AR-based flight interface, which can overlay and display the airspace restriction surface, the electronic fence of the no-fly zone, the name and icon of the obstacle (the different colors, font formats and additional symbols of the label indicate whether it is a published obstacle, whether it is an obstacle under construction or rectification, whether it is over-altitude, whether it is a mandatory patrol, and whether it has been patrolled and checked in), the name and icon of the construction equipment (the color of the label needs to be used to distinguish whether it is in the usable period), the checkpoint (the color of the label needs to be used to distinguish whether it has been checked in), the return point and icon, the suggested patrol route, the key patrol area (the color of the label needs to be used to distinguish whether it has been patrolled and checked in), the cursor and name and distance of the UAVs detected by the detection equipment and the cooperative UAVs obtained by the UOM (the color of the label needs to be used to indicate whether it is in violation and the degree of violation, and the cooperative and non-cooperative UAVs need to be distinguished), and the cursor and flight number, aircraft type and distance of the manned aircraft, etc. The required data is collected and summarized by the server module 101 from the source and then provided. When a drone flies close to the restricted area, excessively high obstacles and their names (if the obstacle has been entered into the system) can be easily detected. For the VTOL fixed-wing UAV 111, the observer can click on the obstacle label to enter obstacle information in a pop-up window, or enter other information such as special situations and add tags through the system terminal. They can also click on key inspection areas to view notes and precautions. For the multi-rotor UAV 110, the pilot must hover to perform these operations. The ground station flight interface allows for the selection of all, a major type, or a single subtype of restricted areas for display and hiding. Both the pilot and observer can control this independently in their respective flight display interfaces. Suggested inspection routes can be displayed or hidden in the pilot's control interface.

[0056] Meanwhile, the platform also includes a handheld terminal module 109, which is connected to the server module 101; The handheld terminal module 109 is used to provide a two-dimensional electronic map interface to realize on-site data entry, height limit calculation and manual inspection task management.

[0057] In this embodiment, the handheld terminal module 109 is a retained traditional functional module, used as an aid to UAV inspections. It is used by leaders and inspectors of the airport airspace management department when performing inspection tasks, and pilots do not have access to it. The handheld terminal module 109 is a smartphone or tablet computer, which needs to have a GNSS positioning module, a front-facing camera, and the ability to access the operator's wireless network. The APP page of the handheld terminal displays a two-dimensional electronic map. The map type includes orthophoto maps (DOM) from ordinary satellite remote sensing image data or ordinary electronic maps (similar to the modes of commonly used navigation software such as Gaode Maps and Baidu Maps). The map scene can display the user's current location and the direction they are facing. The electronic map displays all obstacles in the form of icons. Clicking an icon brings up a pop-up bubble displaying all information about that obstacle. At the bottom of the page, buttons for "On-site Inspection Data Input" and "Height Restriction Calculation" appear. Clicking these allows users to input relevant information collected during inspections or upload photos taken on-site. Upon completion, the inspection data and upload time are uploaded to the platform, and the platform's data is refreshed. Clicking the "Height Restriction Calculation" button directly uses the obstacle's coordinates and inspection height to calculate the height restriction. The evaluation results are displayed similarly to those on the PC. Users can view all their inspection tasks for the day. As an inspection team leader, users can select and control the start and end of tasks. Special situations can be reported during inspections by marking points, entering descriptions, and taking photos on-site. New obstacles can be created. After filling in all relevant information except coordinates according to the obstacle database fields, an obstacle icon is generated on the screen. Dragging the icon to the corresponding location on the map locks the icon's position after 15 minutes (subsequent position updates must be performed on the PC). Simultaneously, the system records the location coordinates and uploads them to the platform server. It supports long-pressing any point on the map and then using two methods in the pop-up window: using the current location's altitude plus the true height of obstacles, or directly entering the altitude to be evaluated. The evaluation results are displayed similarly to the PC version. Obstacles can be searched and located by name and coordinates, with fuzzy matching supported when searching by name. Navigation is supported for any obstacle and checkpoints in patrol tasks. On the handheld device, users can also view shared content with access permissions in the platform's information publishing and sharing function (at least supporting preview images and PDF document attachments), and can also fill in relevant descriptions and upload shared content from their local device.

[0058] Furthermore, the business management module 106 includes a three-dimensional airspace visualization unit, a scene management unit, an inspection business management unit, an obstacle management unit, a construction project management unit, an airport parameter management unit, a personnel file management unit, and a drone file management unit; The aforementioned 3D visualization unit for airspace is built upon a GIS engine (such as the common Cesium engine). The data consists of orthophotos (DOM) and digital elevation models (DEM) generated by remote sensing technology. It can also display digital surface models (DSM) and ordinary electronic maps, and can display latitude, longitude, and elevation in real time according to the mouse position. DOM, DEM, and DSM data are typically obtained from satellite remote sensing imagery through manual processing, or from UAV remote sensing or aerial remote sensing imagery data through manual processing. The imaging methods are diverse, including oblique photography, radar, and lidar. Generally, some reputable professional surveying and mapping companies, surveying and mapping bureaus, and some commercial satellite companies can provide finished DOM, DEM, and DSM data in TIFF format. However, these TIFF format data products need to be sliced ​​using professional software (such as CesiumLab) before they can be imported into various systems or platforms using GIS engines (including this platform). In the 3D scene, drones detected by the detection equipment (including drones acquired through UOM data, which need to be merged), airborne objects, birds, etc., can be displayed by overlaying cursors. The cursor supports click interaction. The scene can also display white or detailed models of buildings and structures, which also support click interaction. The white or detailed models can also be colored to indicate their over-altitude status or whether they are airport-announced obstacles. The visualization scene also includes some auxiliary functions, such as retrieval (search and location by obstacle name or input coordinates, supporting fuzzy matching), manual creation of white models (supports binding to obstacles in the existing obstacle list), height restriction calculation (after clicking any location in the scene, the altitude to be calculated or the true height of the obstacle to be calculated is entered in the pop-up window, based on all types of restriction surfaces and sub-surfaces, and indicating which sub-surfaces are exceeded when the evaluation result is over-altitude), distance measurement, area measurement, etc. The scene can control the visibility of each type of airspace restriction surface and its sub-surfaces, each key inspection area, all no-fly zones, all white and detailed models of buildings and structures, and patrol trajectories.

[0059] The data structures and sources required for the scenario are as follows: Figure 3 As shown.

[0060] The scene management unit is used to update the 3D scene and store historical scenes. Each scene consists of an orthophoto (DOM), a digital elevation model (DEM), and a digital surface model (DSM), which can be selected and switched arbitrarily. Scene data already in the database can also be added, deleted, modified, and queried. On a separate page, a change patch detection algorithm can be used to select which areas between two image periods show changes, and the changed areas are displayed using a 3D scene highlight and semi-transparent overlay. The scene management page can also add calculation functions for mutual conversion between multiple coordinate systems, such as WGS-84 and airport polar coordinates. The calculation method is as follows: first, determine the WGS-84 coordinates of the airport reference point, then calculate the horizontal distance between the input coordinates and the airport reference point, and the azimuth angle of the input coordinate point relative to the airport reference point to achieve WGS-84 to airport polar coordinates conversion; conversely, determine the position of the point to be calculated in the scene based on the airport reference point position + the input distance and relative azimuth angle, and then read the latitude and longitude coordinates to achieve airport polar coordinates to WGS-84 coordinates conversion.

[0061] The patrol operation management unit is used to manage the tasks of patrol personnel (including crew observers), pilots, and other personnel. This module is not open to patrol personnel accounts and specifically includes the following functions: Task Template Management The patrol missions include two types: drone patrols and traditional manual patrols. Drone patrols are the primary method, while traditional manual patrols are secondary. Traditional manual patrols are only used as an emergency backup plan when drones cannot be used due to severe weather or airspace control, and are a traditional function that is retained. Since patrol tasks are typically repetitive, task templates can be pre-set. These templates include selecting the task type, patrol method, drawing a recommended patrol route (for UAV patrol tasks, the recommended route must be drawn as a 3D route; route nodes are generated by clicking on each node in the 3D scene and inputting the altitude; the tail node is fixed at the return-to-home altitude directly above the ground; a 3D preview of all nodes and the routes connecting them is available, and individual nodes can be selected for adjustment; for example, if the suggested patrol route node for a vertical take-off and landing fixed-wing UAV 111 is less than 30 meters above the ground or the calculated turning radius may exceed the UAV's maneuverability, a pop-up warning should be displayed), selecting mandatory obstacles (allowing navigation to the 3D scene or selection from the obstacle list), marking checkpoints (for checking objects or locations not already in the database; generally used for temporary or additional checks; can be quickly modified and adjusted; check notes can be added), and filling in the task description. Existing templates support CRUD operations.

[0062] (2) Staff scheduling management Administrators (leaders) can set work schedules for each inspector (observer) and pilot. The scheduling method must be consistent with the specific airport used, and scheduling must be done for at least the current month and the following month. Special statuses such as leave, meetings, absences, and secondments are allowed. If the work schedule for everyone for the following month has not been completed by the end of the month, the system must remind the administrator (leader) to complete the scheduling.

[0063] (3) Task dispatch You can select an existing task template and modify it slightly (without changing the template content). You can then select members from the executors to assign tasks, but only those on duty during the task's start and end time can be selected. For patrol tasks using fixed-wing drones, a mandatory two-person team is required, meaning one pilot with fixed-wing drone operating qualifications and one ordinary patrolman as an observer are needed. For patrol tasks using multi-rotor UAV 110, a pilot with multi-rotor UAV 110 operating qualifications is mandatory; ordinary patrolmen are optional. Traditional manual patrol tasks require two ordinary patrolmen: one team leader and one team member. For drone-based patrol tasks, the corresponding pilot can only select and operate the drone by logging into their account to unlock the ground station after receiving the task assignment notification. Pilots without a task cannot use the ground station, and pilots with a task can only select drones that meet the task requirements and their own pilot qualifications. Ordinary patrolmen do not need to perform many operations and must follow the pilot's instructions. For traditional manual patrol tasks, patrol team members' handheld terminals will receive the task notification, which is then confirmed and initiated by the patrol team leader. When issuing a mission to intercept unauthorized drones, simply click on the unauthorized drone to be intercepted in the 3D scene (multiple targets can be selected). The system will automatically select an on-duty and available pilot from the pilot drop-down list. You can also modify the default selected pilot. Then you can issue the mission directly. The unauthorized drone to be intercepted will be highlighted in the pilot's video transmission interface from the start of the mission. If it is out of view, its relative position and distance will be indicated by an arrow and its name will be attached.

[0064] Published task work orders will be uploaded to the platform's database of published but not yet completed or completed but awaiting review. All fields of the work order will be uploaded completely so that the UAV ground station system and handheld terminal can read task details, obtain recommended routes, checkpoints, mandatory obstacles, and other task information. It also supports recording checkpoint check-in times, actual task start times, etc. It includes all fields of the task ledger database and needs to include suggested patrol trajectories and actual patrol trajectory records. After the task is completed and approved, it will be removed from the database and directly added to the task ledger database.

[0065] (4) Task monitoring The system displays all assigned tasks and completed patrols that have not yet been approved by the administrator (leader) in a list format. Users can view task name, task type, actual start time, actual end time (if completed), planned start and end times, execution time, member names, progress of mandatory obstacle inspections, checkpoint check-in progress, inspected obstacles, task team members, and incident records. When an incident is reported, an alarm will pop up on the administrator's (leader's) account interface, allowing access to a 3D scene to view the location and trajectory of the drone or patrol personnel. Pilots or patrol team leaders can manually terminate tasks. For patrol tasks that have taken too long to complete, leaders can expedite the process with a single click; this will trigger a prompt on both the ground station screen and handheld terminal.

[0066] (5) Task Ledger Completed and approved patrol tasks are stored in the task log database. Records include task name, task type, actual start time, actual end time, planned start and end times, total execution time, member names, mandatory obstacles to be checked and patrol completion rate, checkpoints and check-in progress, obstacles already checked and patrol time, checkpoint check-in time, task team members, special incident records, reviewer, and review comments. However, patrol routes are not recorded. All stored task logs can be deleted and queried, and are retained for 5 years, after which they are automatically deleted.

[0067] (6) Mapping of key inspection areas The system allows users to mark key inspection areas, name them, add notes on inspection precautions, and view whether they have been checked in this month and the check-in time (automatically refreshed and reset at 00:00:00 on the 1st of each month), as well as the check-in members (including the names of all crew or shift members). It also supports adding, deleting, modifying, and querying operations.

[0068] (7) No-fly zone management To set up a circular or polygonal no-fly zone, you can either input coordinates or mark points in the 3D scene to determine the center, then input the radius. To create a polygonal no-fly zone, input the coordinates of the polygon's fixed points or mark points in the 3D scene sequentially. After creating the no-fly zone, you can set its name, validity period (it will be automatically deleted after the expiration date; the start date can be set to take effect immediately, and the end date can be set to be permanently valid), and no-fly period (you can set it to be valid for the entire period).

[0069] The full lifecycle operation process for different types of patrol tasks, such as Figure 4 As shown: The obstacle management unit manages all obstacles in the platform through a list, recording detailed information about each obstacle, including name, coordinates, whether it is a natural or man-made obstacle, obstacle type (hills, tall vegetation, ordinary buildings, lightning rods, communication towers, etc.), current status (fixed, under construction (man-made obstacles only), rectification), associated construction project, management unit and contact information, whether the obstacle is under control, whether it has been publicized, whether it has been filed, whether it is under special attention, approved height, measured height (can be directly updated during inspection), whether it exceeds the height limit (including normal, warning, and excessive height statuses, automatically assessed, modification prohibited), last inspection time, obstacle identification status (none, markers, marker lights), whether it was checked in this month (refreshed at 00:00:00 on the 1st of each month to indicate no), and remarks, covering all airspace management business needs. All obstacle information can be added, deleted, modified, and queried, and can be exported in Excel format or imported in batches using standard Excel templates. Users can select any obstacle from the list and then jump to the 3D scene to locate its position. All obstacles support recording historical inspection heights, and can be combined with the height of the restricted surfaces at their location to observe the height change trend through a line graph, as well as whether the height exceeds the limit and which restricted surfaces it exceeds.

[0070] The project management unit includes project management and construction equipment management. Project management is manually maintained and includes information such as project name, construction unit, contact person and contact information, construction status (planned, pending, under construction, completed), construction data files, associated obstacles, and associated construction equipment. Associated obstacles and associated construction equipment are displayed in separate lists, with the displayed fields being the same as those on their respective management pages. The unit supports adding, deleting, modifying, and querying data for project information, obstacles within project information, and construction equipment within project information.

[0071] The construction equipment management section displays all construction equipment in a list format. The displayed fields include name, associated construction project, approved height, actual measured height, whether it exceeds the height limit (including three states: normal, warning, and exceeding the height limit, which are automatically evaluated and cannot be modified), planned installation time, planned evacuation time, and permitted usage period (which can be set separately for weekdays, weekends, holidays, and some special dates).

[0072] The airport parameter management unit includes basic airport information, reference points, a runway list, detailed runway parameters, a list and detailed parameters of navigation facilities, a list and detailed parameters of radar systems, a list and detailed parameters of navigation stations, and a list and detailed parameters of lighting facilities. It allows for CRUD operations (add, delete, modify, and query) and serves as the basis for generating airspace restriction surfaces. The module page allows setting the warning and alarm heights for each restriction surface, enabling batch settings for a type of restriction surface or individual settings for a single sub-surface.

[0073] The personnel file management unit is not open to inspector accounts and is maintained by the administrator (leader). It contains information on all accounts (including the leader's account) and can be synchronized from the OA system or created by the administrator. It provides CRUD operations and records basic personnel information. In addition to the basic information, the pilot's account also needs to record the pilot's license type, permitted aircraft type, validity period, and violation / dangerous driving records (which can be manually deleted by the administrator (leader)).

[0074] The UAV file management unit displays all VTOL fixed-wing UAVs 111 and multi-rotor UAVs 110 included in the platform in real time in a list format. The recorded content includes: name, type classification (VTOL fixed-wing, multi-rotor), S / N code, model, manufacturer, manufacturing date, purchase date, fuselage photo, current status (airworthy, on mission, unairworthy, under maintenance, under repair, scrapped), reason for unairworthiness (only for unairworthy status), current mission name (only for on mission), current pilot name (only for on mission), last flight time (based on motor start time), total flight hours, last maintenance time, hours remaining until the next maintenance (when the remaining hours are 0, the UAV becomes unairworthy, but is not affected while flying in the air), current remaining battery cycles (not displayed for hybrid or gasoline-powered UAVs; when the remaining cycles are 0, the motor is automatically locked and flight is not allowed, the status is set to unairworthy, and a prompt is displayed on the pilot's operating interface), and abnormal status (obtained in real time through the UAV self-check function; if there is an abnormal status that affects flight safety, the UAV's status is set to unairworthy). Only drone pilots who are correctly registered as required and whose status is airworthy can make selections in the control equipment at the ground station.

[0075] Finally, the drone application module 107 receives all the data required for AR display, electronic fence, collision warning, and airspace inspection functions directly from the server module 101. All complex data processing procedures are completed on the platform server. This module is only responsible for performing necessary simple processing on the finished data to ensure the normal operation of each function. The specific processing steps are as follows: (1) Constructing a 3D model of the AR display object Based on different AR display object categories, the system filters and extracts all the information required for AR display functionality, such as the object's name, coordinates, and elevation, from the corresponding database and received task data. Then, it creates and updates dynamic 3D models of all AR display objects in real-time based on their coordinates and elevations. Note that AR display objects requiring clickable interaction, such as obstacles and construction equipment, need their corresponding primary key information in the platform database recorded to enable CRUD operations on these records. This step can be understood as creating a virtual world containing only the objects needed for AR display, using the same world coordinate system, and aligned with the corresponding real-world height in real-time.

[0076] (2) Establish the three-dimensional polar coordinate system of the camera Based on the real-time position, attitude (heading, slope, pitch angle), altitude, camera lens pitch angle, camera lens deflection angle, lens field of view, lens focal length, resolution, and other parameters of the piloted UAV, the spatial range and projection plane of the lens's field of view are modeled, and a polar coordinate system is established with the lens as the origin and the positive Z-axis direction directly in front of the field of view.

[0077] (3) Projection display Calculate the position of each AR display object in the polar coordinate system established in step (1) based on the coordinate system transformation formula and projection matrix. Calculate the projection position and shape of the AR display object on the projection plane (for non-point objects). Overlay the cursor or highlight plane, and simultaneously display the distance of the AR display object in the polar coordinate system as the actual spatial distance. This step can be understood as highlighting and semi-transparently displaying objects in the virtual world to achieve a visualization effect, and then overlaying them onto the real world. Ideally, the display position of the AR display object in the virtual world is aligned with the observation position of the corresponding object in the real world.

[0078] Pilots need to log in with their real names (facial recognition) to use this module. Those without a valid license matching the aircraft type compatible with the ground station cannot log in. Drones can only be selected and operated during patrol missions or maintenance test flights. During patrol missions, this module must ensure that only drones currently in airworthy condition can be selected, only drones licensed by the pilot can be selected, only drones meeting mission requirements can be selected, and only drones compatible with the ground station can be selected. During maintenance test flights, drones in all states can be selected, but this module must still ensure that only drones licensed by the pilot can be selected, only drones compatible with the ground station can be selected, and both the flight interface and observer interface must display all content except for the suggested patrol route. The functionality of the RTK component can be tested, and all patrol check-in related functions are locked. This module should also support direct access to the obstacle database for real-time data updates during aerial patrols, and should also support access to and uploading of shared content to the platform's information publishing and sharing functions.

[0079] The drone application module 107 includes an automatic cruise unit, an obstacle elevation measurement unit, an inspection and check-in unit, an illegal drone interception unit, a no-fly zone electronic fence unit, and a collision warning unit. The automatic cruise unit can be set to automatically cruise the UAV, allowing it to fly along a fixed patrol route or automatically return to home, thereby reducing the pilot's workload and enabling the pilot to assist the observer in patrols. For multi-rotor UAVs 110 patrolling within the airfield, this allows the pilot to focus on the patrol task. Fixed-route automatic cruise typically needs to be activated near the suggested patrol route. The system will automatically control the UAV to gradually approach and fly along the suggested patrol route, gradually bringing its flight altitude to within 10 meters below the lowest limit surface of the airspace restriction surface or the altitude specified in the drawn route (for vertical take-off and landing fixed-wing UAVs 111, this requires prioritizing an elevation at least 30 meters above the ground level). If obstacles are present, it can automatically avoid them, automatically giving way to manned aircraft and other operational or cooperative UAVs. One-key return to home can be activated from any location. After activation, it will control the UAV to reach the preset return altitude, then fly at an altitude directly above the return point before landing vertically. However, the automatic cruise function must be able to be quickly deactivated manually at any time to deal with emergencies and cannot replace the pilot's role. The relevant algorithms for the automatic cruise function are usually provided by the UAV manufacturer.

[0080] The obstacle elevation measurement unit allows the drone to hover near the obstacle, ensuring the camera angle from the pilot's perspective is horizontal. The center of the field of view (the crosshair needs to be set in the image transmission interface) is aligned with the highest point of the obstacle, and the onboard RTK module is activated to complete the obstacle elevation measurement. Based on the measurement accuracy of existing onboard RTK modules and the current drone hovering accuracy, this method's measurement accuracy meets the 0.5-meter obstacle elevation measurement accuracy requirement within the area specified in Annex 14 of the "Civil Airport Flight Zone Technical Standards".

[0081] The inspection and check-in unit focuses on obstacles rather than specific areas or routes, so the check-in process primarily targets obstacles. A "staring check-in" method is used for obstacles: the user enters a certain distance and keeps the obstacle within a certain field of view (based on the observer's perspective on the VTOL fixed-wing UAV 111) close to the center for a certain period. This is considered a completed inspection and automatic check-in. If a magnified view is used, the check-in distance can be extended accordingly, but it must be ensured that small, lightning rod-like appendages of the target obstacle are clearly visible within this distance. For the VTOL fixed-wing UAV 111, the observer's camera must have the ability to automatically track any target for extended and detailed inspection of obstacles. The check-in method for checkpoints is the same as for obstacles. For key inspection areas, the UAV needs to be operated to enter the relevant area to complete the check-in. Managers are allowed to set mandatory obstacles for each inspection when assigning tasks in the airspace inspection and business management interface. The AR tags for these obstacles must be highlighted, and the total number and the number of times they have been checked must be displayed in the pilot and observer interfaces. The pilot interface can also display these obstacles through an expandable and collapseable list. Clicking on an obstacle in the list will briefly indicate its location and distance via an arrow on the flight interface. Non-mandatory obstacles can also be checked; they are not displayed in the list but can be found through search and have their location indicated. Real-time photo and video recording are supported, and photos can be uploaded to a designated obstacle photo list in the obstacle database before the task ends, or uploaded through the information publishing and sharing function. Unuploaded photos and videos are automatically deleted after the task ends. Single-operated multi-rotor drones (110) are prohibited from uploading photos or videos while in non-hovering flight. The AR tag color and other methods should also indicate whether an obstacle or key inspection area has been checked.

[0082] The data calculation process involved in the gaze check-in method is as follows: In the camera's three-dimensional polar coordinate system, a quadrangular pyramid model is established based on the magnification, the camera's field of view, and the preset check-in distance at that magnification. This can be understood as all objects inside the pyramid being visible, and even details such as objects like lightning rods being clearly visible. Once the actual position of the AR display object in the AR virtual world enters the quadrangular pyramid model, the timer starts (and resets to zero when it leaves). After reaching the preset time (such as 1 second, there is no unified standard, and it is determined by the user department), it can be marked as checked in.

[0083] For check-in of key inspection areas, the coverage area can be determined when drawing the key inspection areas on the PC login platform. If the platform detects that the current location reported by the drone or handheld terminal is within the scope of a certain inspection area, the platform will automatically mark the inspection area as checked in.

[0084] The interception unit for unauthorized drones utilizes a more maneuverable multi-rotor drone 110, primarily employing physical interception methods such as attaching a capture net. This net-mounted interception method is a safe approach, preventing uncontrolled drone falls that could damage ground aircraft or infrastructure, but it requires continuous and precise guidance. Since it's difficult to detect distant drones via image transmission, the platform collects information on unauthorized drones located by radar, optoelectronic devices, and radio detection equipment. This information, combined with the multi-rotor drone 110's position and flight data, is displayed as an AR cursor on the image transmission interface, showing its distance and indicating it as a non-cooperative drone. Drones intruding into the airport's designated countermeasures zone are displayed in a collapsible list on the pilot's interface, allowing for guidance. Once guidance is activated, arrows continuously indicate the target's position and distance to guide the interception. The interception method involves the pilot, guided by the AR display, precisely maneuvering to capture the unauthorized drone in the capture net, then returning with the captured drone. Furthermore, for dangerous high-altitude birds detected by the detection system, this method can also guide the multi-rotor UAV 110 to launch bird-repelling munitions or release bird-repelling agents to drive them away. At the start of the mission, the ground station system will query all illegal UAVs that need to be intercepted through a work order, and automatically activate the AR-enhanced display of these illegal UAVs and provide arrow + name + distance guidance outside the field of view. This method will continue throughout the mission cycle. If a new illegal UAV is detected during the interception process, the pilot can also click on its tag to add it to the interception list, and can also delete it from the interception list if it is necessary to stop the interception.

[0085] The electronic fence unit for the no-fly zone is drawn, managed, and stored in the platform database within the business management module 106. If a drone is less than a certain distance from the no-fly zone, a continuous warning should be displayed on the pilot's interface. For vertical takeoff and landing fixed-wing drones 111, if they are too close to the no-fly zone and there is a high risk of crossing the boundary, the drone will automatically turn away in the nearest direction. For multi-rotor drones 110, they will automatically brake and lock their flight direction before touching the restriction surface to ensure they do not cross the boundary. When performing a task to intercept unauthorized drone flights, only the no-fly zone electronic fence will be displayed, and no automatic avoidance actions will be triggered. For safety reasons, unauthorized entry into the no-fly zone will only trigger automatic departure without forced landing. The system backend should automatically record any unauthorized entry into the no-fly zone (excluding entry during the interception of unauthorized drone flights), recording which pilot entered which no-fly zone at which time.

[0086] The calculation method involved in this section is as follows: After the no-fly zone is drawn, the system will set a buffer zone at a certain distance from the edge of the no-fly zone. The thickness of the buffer zone needs to fully consider the deceleration performance of the drone in the local environment and varies depending on the type of drone. When the drone enters the buffer zone, the velocity component in the direction perpendicular to the edge of the no-fly zone needs to be reduced by the drone flight control system. The closer to the edge of the no-fly zone, the smaller the allowed velocity component, until it is 0 at the edge of the no-fly zone. The automatic avoidance direction is: draw a perpendicular line from the current position of the drone to the edge of the nearest no-fly zone and parallel to the horizontal plane. The direction of the perpendicular line away from the no-fly zone is the automatic avoidance direction.

[0087] The collision avoidance warning unit is used to predict future flight trajectories and assess collision risks based on current flight data of the platform's drones, flight data of other cooperative drones, and real-time flight data of manned aircraft. If the risk is high, the system should display a warning message and indicate a suggested avoidance direction through the pilot's video transmission interface. If the collision risk is extremely high, forced automatic avoidance should be triggered. If forced automatic avoidance is triggered, the pilot information, trigger time, and trigger location should be recorded in the background. This function is similar to the TCAS system commonly equipped on civil aircraft and can be improved and developed based on the functionality of this system. In addition, both types of drones need to have the function of detecting and alerting surrounding obstacles, that is, similar to the omnidirectional obstacle avoidance function of ordinary consumer drones. When performing a mission to intercept unauthorized drones, the collision avoidance function for drone-type targets and the omnidirectional obstacle avoidance function of the drone should be temporarily disabled. If there are nearby obstacles, only an alert should be given, and automatic obstacle avoidance should not be performed.

[0088] If not developed based on the TCAS system, the data processing method must include all three rules simultaneously: The first rule: Based on the flight data of other drones and manned aircraft held by the platform, the flight path is plotted. A safety radius is set according to the aircraft type. A cylindrical area model is established with the flight path direction as the height, taking the flight distance within the current speed warning time and danger warning time (which needs to be comprehensively evaluated and set according to the drone's performance, the user department, etc.). At the same time, the cylindrical area is divided into a warning zone and a danger zone according to the flight distance within the expected impact time. When the drone enters the corresponding cylindrical area, the corresponding reaction is triggered (warning zone: only a prompt is given, no forced avoidance is required, AR enhancement display is used, and the relative position and spatial straight-line distance of all drones / manned aircraft outside the field of view are indicated in the AR flight interface with orange arrows and names; danger zone: AR enhancement display is used, forced avoidance is required, and the relative position and spatial straight-line distance of all drones / manned aircraft outside the field of view are indicated in the AR flight interface with red arrows and names). The suggested avoidance direction and the forced avoidance direction are unified in that the drone's heading axis is away from the axis of the cylindrical area.

[0089] The second rule: Using the local drone as the center point, extend a cylindrical height in both the vertical and horizontal directions by the dangerous height difference (determined based on the drone model, performance characteristics, and user department requirements). Then, draw a cylindrical area with a horizontal dangerous distance (determined based on the drone model, performance characteristics, and user department requirements) as the radius. If there is a drone or manned aircraft in this area and the first rule has not been triggered, only a warning will be displayed: "Drone / manned aircraft nearby, please fly with caution." The AR display will be enhanced, and the relative positions and straight-line distances of all drones / manned aircraft outside the field of view will be indicated in the AR flight interface with yellow arrows, along with their names.

[0090] The third rule: When carrying out the interception mission of unauthorized drones, the first two rules are only valid for manned aircraft.

[0091] This invention discloses an augmented reality-based UAV inspection platform for airport airspace. The platform uses a server module 101 as a data hub, connecting the entire chain from external data access, multi-source detection and perception, business logic processing to AR visualization guidance and task execution feedback, forming a complete business closed loop of "perception-decision-execution-evaluation," significantly improving the intelligence level and decision-making efficiency of airport airspace management. Through the flight control and interaction module 104 and the UAV application module 107, augmented reality technology is deeply applied to UAV flight control and inspection operations. It not only provides pilots with an intuitive and information-rich AR flight interface, reducing operational difficulty and cognitive load, but also reshapes the efficient and accurate UAV inspection process through functions such as "gaze check-in" and AR-guided interception. A collaborative air-ground security system has been constructed: the platform combines the mobile patrol capabilities of drones with the wide-area surveillance capabilities of a fixed ground detection network, and through data fusion and AR visualization, it achieves timely detection, precise positioning, intelligent identification, and effective handling of various low-altitude targets such as cooperative / non-cooperative drones, birds, and airborne objects within the airspace, forming a proactive and three-dimensional new paradigm for airport low-altitude security. Integrated functions such as electronic fences for no-fly zones, multi-rule collision avoidance warnings, and strict drone and personnel file management jointly construct a multi-layered security defense line, ensuring the safe operation of drones in complex airport environments and meeting the extremely high safety requirements of the civil aviation sector. In summary, this invention, through systematic integration and technological innovation, effectively solves the shortcomings of traditional manual patrols and existing technologies in airport airspace management in terms of efficiency, accuracy, coverage, and proactive defense capabilities, providing reliable technical support for achieving digital, intelligent, and visualized refined management of airport airspace.

[0092] On the other hand, please see Figure 2 The present invention also includes an augmented reality-based unmanned aerial vehicle (UAV) inspection method for airport airspace, comprising the following steps: S1: Construct a three-dimensional scene of the airport's airspace, which includes terrain, images, airspace restriction surfaces, and obstacle models; S2: Access external data sources such as air traffic control systems, ADS-B automatic dependent surveillance broadcast systems, and UOM unmanned aerial vehicle cloud systems; S3: Detect low-altitude targets within the airspace using radar, optoelectronic equipment, and radio monitoring equipment, and fuse the detected data with the external data to generate real-time situational information of the airspace. S4: Create and dispatch patrol tasks, which include patrol routes, mandatory obstacles to be patrolled, and crew information; S5: During the flight of the drone, an augmented reality interactive interface is generated and displayed based on the mission information and real-time situation information received from the server; S6: Through an augmented reality interactive interface, it performs augmented reality-based inspection check-in, no-fly zone electronic fence alarms, flight collision avoidance warnings, and interception and guidance of non-cooperative drones.

[0093] Specifically, S61: Establish a visual space model based on the parameters of the drone camera; S62: When a target obstacle enters the visible space model and remains within the range for a preset time, the inspection and check-in of the obstacle is automatically completed.

[0094] In this embodiment, a 3D scene including terrain, imagery, and airspace restriction surfaces is first constructed based on a GIS engine. Data from air traffic control, ADS-B, and UOM systems is accessed through the data interface module 105, while detection data acquired by radar, photoelectric, and radio monitoring equipment is integrated. The server module 101 performs multi-source data fusion processing to generate a real-time situation. The business management module 106 creates patrol tasks based on the real-time situation and dispatches them to designated flight crews. After receiving the task data, the flight control and interaction module 104 generates an AR interactive interface, which overlays obstacle tags, restriction surfaces, electronic fences, and detection target cursors in real time. The UAV application module 107 performs AR inspection check-in. By establishing a camera polar coordinate system and a four-sided pyramid visual model, the check-in is automatically completed when an obstacle remains within the field of view for a preset time. At the same time, the AR cursor guides the interception of non-cooperative UAVs, and the combination of multi-rule collision avoidance warning and electronic fence monitoring ensures flight safety.

[0095] The above-disclosed embodiments are merely one or more preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art can understand that all or part of the processes for implementing the above embodiments and equivalent changes made in accordance with the claims of this application still fall within the scope of this application.

Claims

1. An airport clearance area unmanned aerial vehicle inspection platform based on augmented reality, characterized in that, The platform comprises a server module, a GIS engine module, a UAV module, a flight control and interaction module, a data interface module, a business management module and a UAV application module, the data interface module is connected with the server module, the server module is connected with the GIS engine module, the business management module and the flight control and interaction module, the flight control and interaction module is connected with the UAV application module and the UAV module; The server module is configured to store and manage airport basic parameters, clearance limit surface models, obstacle data, UAV and personnel archives, and perform fusion processing of multi-source detection data, and distribute the processed data to the business management module and the flight control and interaction module; The GIS engine module is configured to construct and render a three-dimensional scene of the airport clearance area based on geographic information data, and the three-dimensional scene comprises terrain, images, limit surfaces and obstacle models; The data interface module is configured to access external data sources such as air traffic control systems, ADS-B broadcast automatic dependent surveillance systems and UOM UAV cloud systems; The business management module is configured to provide a human-computer interaction interface of the three-dimensional scene, realize creation, dispatch and monitoring of patrol tasks, and manage obstacle, construction project and personnel equipment archives; The flight control and interaction module is configured to control flight of the UAV, and generate an augmented reality interaction interface based on data received from the server module; The UAV application module is configured to realize augmented reality-based patrol check-in, no-fly zone electronic fence alarm, flight collision avoidance warning and interception and guidance of non-cooperative UAVs; The UAV module is configured to perform aerial flight tasks and carry sensing devices to collect image and position data.

2. The augmented reality-based airport clearance area UAV patrol platform of claim 1, wherein The UAV module comprises a multi-rotor UAV and a vertical take-off fixed-wing UAV; The multi-rotor UAV is configured to perform patrol, interception and bird repelling tasks within the airport red line range; The vertical take-off fixed-wing UAV is configured to perform patrol tasks in a long-distance clearance area outside the airport.

3. The augmented reality based airport clearance area drone inspection platform of claim 1, wherein, The augmented reality-based airport clearance area UAV patrol platform further comprises a detection and perception module connected with the server module; The detection and perception module is configured to detect low-altitude targets in the clearance area through radar, photoelectric devices and radio monitoring devices, and send detection data to the server module for fusion processing.

4. The augmented reality based airport clearance area drone inspection platform of claim 1, wherein, The augmented reality interaction interface generated by the flight control and interaction module is configured to superimposedly display obstacle labels, clearance limit surfaces, no-fly zone electronic fences, suggested patrol routes and real-time detected low-altitude target cursors.

5. The augmented reality based airport clearance area drone inspection platform of claim 1, wherein, The platform further comprises a handheld terminal module connected with the server module; The handheld terminal module is configured to provide a two-dimensional electronic map interface to realize on-site data entry, height limit calculation and manual patrol task management.

6. An airport clearance area unmanned aerial vehicle inspection method based on augmented reality, using the airport clearance area unmanned aerial vehicle inspection platform based on augmented reality according to any one of claims 1-5, characterized in that, The method comprises the following steps: constructing a three-dimensional scene of the airport clearance area, wherein the three-dimensional scene comprises terrain, images, clearance limit surfaces and obstacle models; Access external data sources such as air traffic control system, ADS-B broadcast automatic dependent surveillance system, and UOM unmanned aerial vehicle cloud system; Detect low-altitude targets in the clear zone through radar, photoelectric equipment, and radio monitoring equipment, and fuse the detection data with the external data to generate real-time situation information of the clear zone; Create and dispatch a patrol task, which includes a patrol route, mandatory obstacles, and execution crew information; During the flight of the unmanned aerial vehicle, generate and display an augmented reality interaction interface based on the task information received from the server and the real-time situation information; Through the augmented reality interaction interface, perform augmented reality-based patrol check-in, no-fly zone electronic fence warning, flight collision avoidance warning, and interception guidance for non-cooperative unmanned aerial vehicles.

7. The augmented reality based airport clearance area drone inspection method of claim 6, wherein, The specific steps for performing augmented reality-based patrol check-in include: Establish a visual space model according to the parameters of the unmanned aerial vehicle camera; When a target obstacle enters the range of the visual space model and lasts for a preset time, automatically complete the patrol check-in for the obstacle.

Images